JP2017118200A - Display device, camera position control method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To settle even attendants other than a speaking attendant within an imaging range of a camera even while the attendant speaks in the vicinity of an electronic blackboard.SOLUTION: A Web camera 130 is mounted on the back of a display 3 of an electronic blackboard. The Web camera 130 is moved from side to side along an upper end of the display and moves an imaging range. In the case where a touch operation of a writing person on a region at a left side of the display 3 is detected by a touch panel in a view from a front side of the display 3, a control part of the electronic blackboard moves the Web camera 130 to a right side in the view from the front side of the display 3. Thus, an imaging range (h) of the Web camera 130 is moved to the right side in the view from the front side of the display 3.SELECTED DRAWING: Figure 20

Description

  The present invention relates to a display device, a camera position control method, and a program.

  A display device such as an electronic blackboard equipped with a touch panel that detects a touch (touch operation) on the display screen can enlarge various images such as documents displayed on the display screen by bringing a finger or the like into contact with the display screen. Since these various images can be easily written, they are rapidly spreading. In addition, there are many controllers that comply with server and client communication standards such as OPS (Open Profiling Standard). By applying the controller to the electronic blackboard, various images displayed on the display screen can be transmitted and received between the electronic blackboards. Systems that do this are also becoming popular.

  In addition, there is a conference system that performs a conference with a remote location by transmitting and receiving images and sound between terminals at a remote location via a communication network such as the Internet. In this conference system, a terminal in a conference room where attendees attend a conference converts an image obtained by capturing the conference room with a camera and a voice of a conversation made in the conference room into digital data. , And send it to a conference room terminal in a remote location. A conference room terminal at a remote location displays the received image on a display screen of the terminal, or outputs the received sound from a speaker. Thereby, the attendee who exists in a meeting room can hold a meeting in the state near an actual meeting. There is also an electronic blackboard that uses a conference system.

  By the way, the electronic blackboard to which the conference system is applied has to manually move the camera when the attendee in the conference room does not enter the imaging range of the camera. An electronic blackboard to which a conference system is applied has a camera control method for moving an imaging range of a camera in a direction where there are attendees speaking in a conference room. However, in this camera control method, when an attendee speaks in the vicinity of the electronic blackboard when the camera is attached to the top of the display screen of the electronic blackboard, the attendee occupies most of the imaging range, Other attendees will not enter the imaging range.

  The present invention has been made in view of the above, and even if an attendee speaks in the vicinity of the electronic blackboard, an attendant other than the attendant also enters the imaging range of the camera, A camera position control method and program are provided.

  In order to solve the above-described problems and achieve the object, the present invention is provided with a display screen, a detection unit that detects a touch operation on the display screen, and an imaging range in front of the display screen, And a camera that can move the imaging range in the left-right direction of the display screen, a transmission unit that transmits a captured image obtained by imaging the front of the display screen by the camera to an external device, and a detection unit in the left-right direction of the display screen And a control unit that moves the imaging range in a direction opposite to the detection position where the touch operation is detected.

  According to the present invention, even when writing is performed in the vicinity of the display device, another attendee attending a meeting or the like can be imaged.

FIG. 1 is a diagram illustrating an example of the overall configuration of the image processing system according to the present embodiment. FIG. 2A is a diagram illustrating an example of a hardware configuration of the electronic blackboard according to the present embodiment. FIG. 2B is a diagram for explaining an example of a configuration of a camera included in the electronic blackboard according to the present embodiment. FIG. 3 is a diagram illustrating an example of functional blocks of the electronic blackboard according to the present embodiment. FIG. 4 is a diagram illustrating an example of functional blocks of a file processing unit included in the electronic blackboard according to the present embodiment. FIG. 5 is a diagram illustrating an example of functional blocks of a communication control unit included in the electronic blackboard according to the present embodiment. FIG. 6 is a conceptual diagram showing an example of page data stored in the electronic blackboard according to the present embodiment. FIG. 7 is a conceptual diagram showing an example of stroke arrangement data stored in the electronic blackboard according to the present embodiment. FIG. 8 is a conceptual diagram showing an example of coordinate array data stored in the electronic blackboard according to the present embodiment. FIG. 9 is a conceptual diagram showing an example of media data stored in the electronic blackboard according to the present embodiment. FIG. 10 is a diagram showing an example of a remote license management table stored in the electronic blackboard according to the present embodiment. FIG. 11 is a diagram showing an example of an address book management table stored in the electronic blackboard according to the present embodiment. FIG. 12 is a diagram showing an example of backup data stored in the electronic blackboard according to the present embodiment. FIG. 13 is a diagram illustrating an example of a connection destination management table stored in the electronic blackboard according to the present embodiment. FIG. 14 is a diagram illustrating an example of a participating site management table stored in the electronic blackboard according to the present embodiment. FIG. 15 is a diagram illustrating an example of operation data used for displaying an image on the electronic blackboard according to the present embodiment. FIG. 16 is a diagram illustrating an example of an image displayed on the electronic blackboard according to the present embodiment. FIG. 17 is a sequence diagram showing processing of the electronic blackboard according to the present embodiment. FIG. 18 is a sequence diagram showing processing of the electronic blackboard according to the present embodiment. FIG. 19 is a diagram for explaining an example of a method for controlling the imaging range of a camera in a conventional electronic blackboard. FIG. 20 is a diagram for explaining an example of a method of controlling the imaging range of the camera using the electronic blackboard according to the present embodiment. FIG. 21 is a flowchart illustrating an example of a flow of a method for controlling the imaging range of the camera using the electronic blackboard according to the present embodiment.

  Hereinafter, embodiments of a display device, a camera position control method, and a program will be described in detail with reference to the accompanying drawings. In the following description, an example in which a display device, a camera position control method, and a program are applied to an image processing system will be described.

<System overview>
FIG. 1 is a diagram illustrating an example of the overall configuration of the image processing system according to the present embodiment. As shown in FIG. 1, in the present embodiment, the image processing system includes two electronic blackboards 2a and 2b and electronic pens 4a and 4b associated therewith in order to simplify the description. However, you may have three or more electronic blackboards, electronic pens, etc.

  As shown in FIG. 1, the image processing system 1 includes a plurality of electronic blackboards 2a and 2b (an example of a display device), a plurality of electronic pens 4a and 4b, USB (Universal Serial Bus) memories 5a and 5b, a notebook PC (Personal Computer) 6a, 6b, television (video) conference terminals 7a, 7b, and PC 8. The electronic blackboards 2a and 2b and the PC 8 are connected via a communication network 9 so that they can communicate with each other. Furthermore, the plurality of electronic blackboards 2a and 2b are provided with displays 3a and 3b (an example of a display screen), respectively.

  The electronic blackboard 2a displays various images such as a stroke image drawn by an event generated by the electronic pen 4a (touch operation by the pen tip of the electronic pen 4a or the pen bottom of the electronic pen 4a on the display 3a). 3a can be displayed. The electronic blackboard 2a changes various images displayed on the display 3a based on not only the electronic pen 4a but also events (gestures such as enlargement, reduction, and page turning) generated by the user's hand Ha or the like. You can also The electronic blackboard 2a includes a camera ca that is provided so as to be able to capture an imaging range in front of the display 3a and that can move the imaging range in the left-right direction of the display 3a.

  The electronic blackboard 2a can be connected to a USB memory 5a. The electronic blackboard 2a reads an electronic file such as PDF (Portable Document Format) from the USB memory 5a, or the electronic blackboard 2a records an electronic file on the USB memory 5a. You can do it. The electronic blackboard 2a is connected to a notebook 10 via a cable 10a1 capable of communication according to standards such as DisplayPort, DVI (Digital Visual Interface), HDMI (High-Definition Multimedia Interface: registered trademark), and VGA (Video Graphics Array). It is connected to a video output device such as a PC 6a. Then, the electronic blackboard 2a generates an event in response to a touch operation on the display 3a, and transmits event information indicating the event to the notebook PC 6a in the same manner as an event from an input device such as a mouse or a keyboard. The electronic blackboard 2a is connected to a television (video) conference terminal 7a via a cable 10a2 capable of communication according to the above-mentioned standard. The notebook PC 6a and the video conference terminal 7a may communicate with the electronic blackboard 2a by wireless communication complying with various wireless communication protocols such as Bluetooth (registered trademark).

  On the other hand, in the other bases where the electronic blackboard 2b is installed, the electronic blackboard 2b with the display 3b, the electronic pen 4b, the USB memory 5b, the notebook PC 6b, the video conference terminal 7b, the cable 10b1, and the cable 10b2 in the same manner as described above. The camera cb is used. Furthermore, various images displayed on the display 3b can be changed based on an event caused by the user's hand Hb or the like.

  Thereby, various images displayed on the display 3a of the electronic blackboard 2a at one base are also displayed on the display 3b of the electronic blackboard 2b at another base, and conversely, the display 3b of the electronic blackboard 2b at another base. Various images displayed above are displayed on the display 3a of the electronic blackboard 2a at one base. As described above, the image processing system 1 can perform remote sharing processing for sharing the same image at a remote location, and thus is very convenient when used for a conference or the like at a remote location.

  Hereinafter, when an arbitrary electronic blackboard is shown among the plurality of electronic blackboards 2a, 2b, etc., it is indicated as “electronic blackboard 2”. When an arbitrary display is shown among the plurality of displays 3a and 3b, it is indicated as “display 3”. When an arbitrary electronic pen among the plurality of electronic pens 4a and 4b is indicated, it is indicated as “electronic pen 4”. When an arbitrary USB memory is indicated among the plurality of USB memories 5a and 5b, it is indicated as “USB memory 5”. When an arbitrary notebook PC is shown among the plurality of notebook PCs 6a and 6b, it is indicated as “notebook PC 6”. When an arbitrary video conference terminal is indicated among the plurality of video conference terminals 7a and 7b, it is indicated as "video conference terminal 7". Further, in the case where an arbitrary hand among the hands Ha and Hb of a plurality of users is indicated, it is indicated as “hand H”. When an arbitrary cable is shown among the plurality of cables 10a1, 10a2, 10b1, and 10b2, it is indicated as “cable 10”. In addition, when an arbitrary camera among the plurality of cameras ca and cb is indicated, it is indicated as “camera c”.

  In the present embodiment, the electronic blackboard 2 is described as an example of a display device. However, the present invention is not limited to this, and other examples of the display device include electronic signboards (digital signage), sports, and weather forecasts. It may be a telestrator to be used or a remote image (video) diagnostic apparatus. The notebook PC 6 will be described as an example of the information processing terminal, but is not limited thereto. Other examples of the information processing terminal include a desktop PC, a tablet PC, a PDA (Personal Digital Assistant), a digital video camera. It may be a terminal capable of supplying image frames, such as a digital camera or a game machine. Further, the communication network 9 includes the Internet, a LAN (Local Area Network), a mobile phone communication network, and the like. In this embodiment, the USB memory 5 is described as an example of a recording medium. However, the present invention is not limited to this, and various recording media such as an SD card may be used as another example of the recording medium.

<Hardware configuration of electronic blackboard>
Next, the hardware configuration of the electronic blackboard 2 of the present embodiment will be described using FIG. 2A. FIG. 2A is a diagram illustrating an example of a hardware configuration of the electronic blackboard according to the present embodiment.

  As shown in FIG. 2A, the electronic blackboard 2 includes a control unit 121 that controls the entire electronic blackboard 2, an SSD (Solid State Drive) 104 that stores various data such as programs for the electronic blackboard 2, and a communication network 9. A network I / F 105 that controls communication and an external storage controller 106 that controls communication with the USB memory 5 via the USB connector 122 are provided. In the present embodiment, the control unit 121 is an electronic circuit and is used to drive a CPU 101 such as a CPU (Central Processing Unit) 101, an IPL (Initial Program Loader) as a processor that controls the operation of the entire electronic blackboard 2. A ROM (Read Only Memory) 102 that stores various programs such as a whiteboard application and a video conference application, and a RAM (Random Access Memory) 103 used as a work area of the CPU 101 are included.

  Here, the whiteboard application is an application that executes display processing of various images on the display 3 in accordance with a touch operation detected by the touch panel 115 described later. The video conference application transmits a captured image obtained by imaging with the camera c, a sound collected by a microphone 126 (to be described later), and the like to a preset external device (in the present embodiment, another electronic blackboard 2). Or receiving a captured image or sound from the external device.

  The electronic blackboard 2 includes a capture device 111 that is connected to a video output device such as a notebook PC 6, a tablet, or a smartphone via the USB connector 123 and acquires various types of information from the video output device. The capture device 111 displays video information as a still image or a moving image on a display of a video output device such as the notebook PC 6 via a VGA (Video Graphics Array) connector 124. In the present embodiment, the capture device 111 is connected to a video output device such as a notebook PC 6 via the VGA connector 124. However, the present invention is not limited to this. For example, HDMI (High Definition Multimedia Interface) or It may be connected to a video output device such as a notebook PC 6 via a connector according to a standard such as DP (Display Port). In addition, the capture device 111 acquires the sound collected by the microphone 126 from the microphone 126 via the USB connector 125. Here, the microphone 126 is provided so as to collect sounds around the electronic blackboard 2. The electronic blackboard 2 also includes a GPU (Graphics Processing Unit) 112 that generates various images to be displayed on the display 3 and a display controller 113 that outputs the various images generated by the GPU 112 to the display 3 and the video conference terminal 7. ing.

  Furthermore, the electronic blackboard 2 includes a touch panel 115 (an example of a detection unit) that detects a touch operation on the display 3 by an infrared ray blocking method or the like. In the present embodiment, the touch panel 115 includes a sensor controller 114 that controls the operation of the touch panel 115 as a whole, an infrared LED 127 that irradiates the display 3 with infrared rays, and shadows that are generated by the shading of infrared rays by a touch operation on the display 3. It has a camera 128 to be detected and a touch panel driver 129 for detecting a touch operation on the display 3 based on a shadow detection result by the camera 128.

  In addition, the touch panel 115 is not limited to the infrared blocking method, but is a capacitive touch panel that detects a touch operation on the display 3 by detecting a change in capacitance, and a voltage change between two opposing resistance films. A resistive film type touch panel for detecting a touch operation on the display 3 by an electromagnetic wave, an electromagnetic induction type touch panel for detecting a touch operation on the display 3 by detecting electromagnetic induction caused by contact of a contact object with the display unit, etc. Various detection means may be used.

  The electronic blackboard 2 includes an electronic pen controller 116. The electronic pen controller 116 (an example of a detection unit) detects the touch of the pen tip on the display 3 and the touch operation with the pen butt by communicating with the electronic pen 4. The electronic pen controller 116 detects a touch operation on the display 3 by a part gripped by the user of the electronic pen 4 as well as a pen tip and a butt of the electronic pen 4 and other parts of the electronic pen 4. Also good.

  The electronic blackboard 2 includes a camera c. In the present embodiment, the camera c moves the imaging range by moving in the left-right direction of the display 3. Specifically, the camera c includes a web camera 130 and a motor 132. The Web camera 130 is provided so that an imaging range in front of the display 3 can be imaged. In addition, the Web camera 130 can move along the upper end of the display 3 to move the imaging range in front of the display 3. The motor 132 moves the web camera 130 along the upper end of the display 3.

  Further, the electronic blackboard 2 includes a control unit 121, an SSD 104, a network I / F 105, an external storage controller 106, a capture device 111, a GPU 112, a touch panel 115, an electronic pen controller 116, and a camera c as shown in FIG. 2A. A bus line 120 such as an address bus or a data bus is provided. The program for the electronic blackboard 2 may be recorded and distributed on a computer-readable recording medium such as a CD-ROM.

  FIG. 2B is a diagram for explaining an example of a configuration of a camera included in the electronic blackboard according to the present embodiment. As shown in FIG. 2B, in the present embodiment, the Web camera 130 is attached to the back surface of the display 3. In the present embodiment, the web camera 130 is connected to the motor 132 via the spring S and the belt b. The web camera 130 is movable in the left-right direction along the upper end of the display 3 when the motor 132 is driven to rotate. For example, as shown in FIG. 2B (b), the web camera 130 moves to the left as viewed from the front of the display 3 when the belt 132 is wound and the spring S is contracted by the rotation of the motor 132. To do. On the other hand, as shown in FIG. 2B (a), the Web camera 130 moves to the right as seen from the front of the display 3 when the spring 132 is extended by the motor 132 being driven to rotate.

  In the present embodiment, the electronic blackboard 2 moves the imaging range of the Web camera 130 in the left-right direction of the display 3 by moving the Web camera 130, but the imaging range of the Web camera 130 is changed to the display 3. If it moves to the left-right direction, it will not be limited to this.

<Functional configuration of electronic blackboard>
Next, the functional configuration of the electronic blackboard 2 will be described with reference to FIGS. First, the overall functional configuration of the electronic blackboard 2 will be described with reference to FIG. FIG. 3 is a diagram illustrating an example of functional blocks of the electronic blackboard according to the present embodiment.

  The electronic blackboard 2 has each functional configuration shown in FIG. 3 by the hardware configuration and program shown in FIG. 2A. The electronic blackboard 2 can be a “host device” that starts the remote sharing process first, and can also be a “participating device” that participates in the remote sharing process that has already started. Here, the remote sharing process is a process of displaying a similar image on the display 3 between the plurality of electronic blackboards 2. The electronic blackboard 2 roughly includes a client unit 20 and a server unit 90. The client unit 20 and the server unit 90 are functions realized in one housing of the electronic blackboard 2. When the electronic blackboard 2 serves as a host device, the electronic blackboard 2 implements the client unit 20 and the server unit 90. Further, when the electronic blackboard 2 is a participating device, in the electronic blackboard 2, the client unit 20 is realized, but the server unit 90 is not realized. That is, in FIG. 1, when the electronic blackboard 2a is the hosting device and the electronic blackboard 2b is the participating device, the client unit 20 of the electronic blackboard 2a passes through the server unit 90 realized in the same electronic blackboard 2a. Communicate with the client unit 20 of the electronic blackboard 2b. On the other hand, the client unit 20 of the electronic blackboard 2b communicates with the client unit 20 of the other electronic blackboard 2a via the server unit 90 realized in the other electronic blackboard 2a.

[Functional configuration of client unit 20]
Next, the functional configuration of the client unit 20 will be described with reference to FIG. The client unit 20 includes a video acquisition unit 21, a coordinate detection unit 22, an automatic adjustment unit 23, a contact detection unit 24, an event distribution unit 25, an operation processing unit 26, a gesture processing unit 27, a video superimposing unit 28, and an image processing unit 30. And a communication control unit 60.

  The video acquisition unit 21 acquires an image signal that is a signal of an output image displayed on a video output device such as a notebook PC 6 connected to the cable 10. When the video acquisition unit 21 acquires an image signal from the video output device, the video acquisition unit 21 analyzes the image signal and derives image information such as the resolution of the frame of the output image formed by the image signal and the update frequency of the frame. And output to the image processing unit 30. In addition, the video acquisition unit 21 acquires a captured image obtained by imaging of the camera c included in the electronic blackboard 2 and outputs the acquired captured image to the image processing unit 30.

  The coordinate detection unit 22 detects the coordinates of a detection position where an event (such as a touch operation by the user's hand H on the display 3) detected by the user on the display 3 is detected. That is, the coordinate detection unit 22 detects the coordinates of the detection position where the touch operation on the display 3 is detected by the touch panel 115. The coordinate detection unit 22 also detects an area touched by the user's hand H on the display 3. The automatic adjustment unit 23 is activated when the electronic blackboard 2 is activated, and adjusts the image processing parameters of the sensor camera in the optical sensor method of the touch panel 115 so that the touch panel 115 can output an appropriate value to the coordinate detection unit 22.

  The contact detection unit 24 detects the coordinates of a detection position where an event (such as a touch operation with the pen tip of the electronic pen 4 or the pen tip of the electronic pen 4 on the display 3) detected by the user is detected. That is, the contact detection unit 24 detects the coordinates of the detection position where the touch operation of the electronic pen 4 on the display 3 is detected by the electronic pen controller 116. The event distribution unit 25 distributes the coordinates detected by the coordinate detection unit 22 and the coordinates detected by the contact detection unit 24 to each event of stroke drawing, UI operation, and gesture operation.

  Here, in “stroke drawing”, when a stroke image B (see FIG. 16) to be described later is displayed on the display 3, the user touches the electronic pen 4 with the electronic pen 4 in contact with the display 3. This is an event until the electronic pen 4 is finally released from the display 3. By this stroke drawing, for example, alphabets “S” and “T” are drawn on the display 3. The “stroke drawing” includes not only drawing an image but also an event of deleting an already drawn image or editing a drawn image.

  The “UI operation” is an event in which the user performs a touch operation on a predetermined position with the electronic pen 4 or the hand H when a UI image A (see FIG. 16) described later is displayed on the display 3. By this UI operation, for example, the color or width of a line drawn by the electronic pen 4 is set. The “gesture operation” is an event that the user touches or moves the display 3 with the hand H when a stroke image B (see FIG. 16) described later is displayed on the display 3. By this gesture operation, for example, when the user moves the hand H while the hand H is in contact with the display 3, the image can be enlarged (or reduced), the display area can be changed, or the page can be switched. .

  The operation processing unit 26 executes various processes according to the UI element in which the event is generated, from what is determined as the UI operation by the event distribution unit 25. Examples of the UI element include a button, a list, a check box, and a text box. The gesture processing unit 27 executes processing corresponding to what is determined as a gesture operation by the event distribution unit 25.

  The video superimposing unit 28 displays the image superimposed by the display superimposing unit 36 described later on the video output device (display 3 or the like). In addition, the video superimposing unit 28 converts a captured image sent from another video output device (such as the video conference terminal 7) into a picture-in-picture in response to an output image from the video output device (such as the notebook PC 6). To do. Further, the video superimposing unit 28 performs switching for displaying the captured image that is picture-in-picture and displayed on a part of the display 3 on the entire display 3.

  The image processing unit 30 performs a layer superimposition process (see FIG. 16) of an output image displayed on the display 3. The image processing unit 30 includes an image acquisition unit 31, a stroke processing unit 32, a UI image generation unit 33, a background generation unit 34, a layout management unit 35, a display superimposition unit 36, a page processing unit 37, a file processing unit 40, and page data storage. Section 300 and a remote license management table storage section 310.

  The image acquisition unit 31 acquires an output image formed by the image signal acquired by the video acquisition unit 21 or an output image from a video output device connected to another electronic blackboard 2. The image acquisition unit 31 outputs the acquired output image to the page processing unit 37. The output image corresponds to the output image C (see FIG. 16) displayed on the video output device (notebook PC 6 or the like). Further, the image acquisition unit 31 outputs a captured image (captured image obtained by imaging of the camera c of the other electronic blackboard 2) received from the other electronic blackboard 2 by the communication control unit 60 to the display superimposing unit 36. . Furthermore, the image acquisition unit 31 outputs the captured image acquired by the video acquisition unit 21 (the captured image acquired by the imaging of the camera c of the electronic blackboard 2) to the communication control unit 60. The stroke processing unit 32 draws the stroke image B (see FIG. 16), deletes the drawn stroke image B, or is drawn based on the event related to the stroke drawing allocated by the event distribution unit 25. Edit the stroke image B. The results of drawing, deleting, and editing the stroke image B based on the stroke drawing are stored in the operation data storage unit 840 (see FIG. 5) as operation data described later. The UI image generation unit 33 generates a UI (user interface) image A (see FIG. 16) preset on the electronic blackboard 2.

  The background generation unit 34 receives media data (background image D (see FIG. 16)) of the page data read from the page data storage unit 300 by the page processing unit 37 from the page processing unit 37. The background generation unit 34 outputs the received media data to the display superimposing unit 36. The background image D is, for example, plain or grid display.

  The layout management unit 35 provides layout information indicating the layout of various images output from the image acquisition unit 31, the stroke processing unit 32, and the UI image generation unit 33 (or the background generation unit 34) to the display superimposing unit 36. I manage. Thereby, the layout management unit 35 causes the display superimposing unit 36 to display the output image C and the stroke image B (see FIG. 16) at any position in the UI image A and the background image D (see FIG. 16). Can be indicated or hidden.

  The display superimposing unit 36 superimposes various images output from the image acquisition unit 31, the stroke processing unit 32, and the UI image generation unit 33 (background generation unit 34) based on the layout information output from the layout management unit 35. A superimposed image is generated. The display superimposing unit 36 superimposes the captured image acquired by the image acquiring unit 31 on the superimposed image using picture-in-picture.

  The page processing unit 37 stores the stroke image B (see FIG. 16) and the output image C (see FIG. 16) in the page data storage unit 300 as one page data. The stroke image B (see FIG. 16) is a part of page data as the stroke arrangement data indicated by the stroke arrangement data ID (see FIG. 6). The output image C (see FIG. 16) is part of page data as media data indicated by the media data ID (see FIG. 6). When the media data is read from the page data storage unit 300, it is handled as the background image D (see FIG. 16).

  Further, the page processing unit 37 transmits the media data of the page data stored in the page data storage unit 300 to the display superimposing unit 36 via the background generating unit 34, so that the video superimposing unit 28 causes the background image to be transmitted. D (see FIG. 16) can be displayed again on the display 3. The page processing unit 37 can return the stroke arrangement data (each stroke data) in the page data to the stroke processing unit 32 so that the stroke can be re-edited. Further, the page processing unit 37 can delete or duplicate page data.

  That is, the output image C (see FIG. 16) displayed on the display 3 when the page processing unit 37 stores the page data in the page data storage unit 300 is temporarily stored in the page data storage unit 300, and thereafter In the case of reading from the page data storage unit 300, it is read as media data indicating the background image D (see FIG. 16). Then, the page processing unit 37 outputs stroke arrangement data indicating the stroke image B (see FIG. 16) of the page data read from the page data storage unit 300 to the stroke processing unit 32. Further, the page processing unit 37 outputs media data indicating the background image D (see FIG. 16) of the page data read from the page data storage unit 300 to the background generation unit 34.

  The display superimposing unit 36 outputs an output image C from the image acquisition unit 31 (see FIG. 16), a stroke image B from the stroke processing unit 32 (see FIG. 16), and a UI image A from the UI image generation unit 33 (see FIG. 16). ) And the background image D (see FIG. 16) from the background generation unit 34 are superimposed according to the layout designated by the layout management unit 35. As a result, as shown in FIG. 16, the images having the configurations of the UI image A, the stroke image B, the output image C, and the background image D are arranged in the order that the user can see even if the images overlap. Generate.

  Further, the display superimposing unit 36 switches between the output image C (see FIG. 16) and the background image D (see FIG. 16), and is exclusive to the UI image A (see FIG. 16) and the stroke image B (see FIG. 16). It is also possible to superimpose them. For example, when the UI image A (see FIG. 16), the stroke image B (see FIG. 16), and the output image C (see FIG. 16) are displayed, the electronic blackboard 2 and the video output device (notebook PC 6 or the like) When the cable 10 is disconnected, the layout management unit 35 can remove the output image C (see FIG. 16) from the superimposition target and display the background image D (see FIG. 16). In this case, the display superimposing unit 36 also performs display enlargement, display reduction, and display area movement processing.

  The page data storage unit 300 stores the page data shown in FIG. FIG. 6 is a conceptual diagram showing an example of page data stored in the electronic blackboard according to the present embodiment. The page data is page data (stroke arrangement data (each stroke data) and media data) for one page displayed on the display 3.

  Here, the details of the page data will be described with reference to FIGS. As shown in FIG. 6, the page data includes a page data ID for identifying each page data, a start time indicating a time when the display of the page data is started, and a page content is rewritten by a stroke or a gesture. The end time indicating the time of disappearance, the stroke arrangement data ID for identifying the stroke arrangement data generated by the stroke by the electronic pen 4 or the user's hand H, and the media data ID for identifying the media data are stored in association with each other. ing. The stroke arrangement data is data for displaying the stroke image B (see FIG. 16) on the display 3. The media data is data for displaying the background image D (see FIG. 16) on the display 3.

  With such page data, for example, when the user draws the alphabet “S” with the electronic pen 4, the stroke is written with a single stroke, and therefore, one stroke arrangement data ID is shown as one letter of alphabet [S]. However, when the user draws the alphabet “T” with the electronic pen 4, it is written with two strokes, so that one stroke of the alphabet “T” is indicated by two stroke array data IDs.

  The stroke arrangement data includes detailed information shown in FIG. FIG. 7 is a conceptual diagram showing an example of stroke arrangement data stored in the electronic blackboard according to the present embodiment. As shown in FIG. 7, one stroke arrangement data is represented by a plurality of stroke data. One stroke data includes a stroke data ID for identifying the stroke data, a start time indicating the start time of writing one stroke, an end time indicating the end time of writing one stroke, the color of the stroke, A coordinate array data ID for identifying the width of the stroke and the array of the passing points of the stroke is shown.

  Further, the coordinate array data includes detailed information shown in FIG. FIG. 8 is a conceptual diagram showing an example of coordinate array data stored in the electronic blackboard according to the present embodiment. As shown in FIG. 8, the coordinate array data includes each point (X coordinate value, Y coordinate value) through which the stroke passes on the display 3, and a time difference (ms from the start time of the stroke when passing through each point). ), And the writing pressure of the electronic pen 4 at each point. That is, a collection of points represented by the coordinate array data shown in FIG. 8 is a stroke. For example, when the user draws the alphabet “S” with the electronic pen 4, it is drawn with a single stroke, but passes through a plurality of passing points until the drawing of “S” is completed. Information.

  Further, the media data in the page data shown in FIG. 6 includes detailed information shown in FIG. FIG. 9 is a conceptual diagram showing an example of media data stored in the electronic blackboard according to the present embodiment. As shown in FIG. 9, the media data includes the media data ID in the page data shown in FIG. 6, the data type of the media data, the recording time when the page data is stored in the page data storage unit 300 from the page processing unit 37, and the page data. Includes data indicating the position (X coordinate value, Y coordinate value) of the image displayed on the display 3, the size (width, height) of the image, and the contents of the media data. Among these, the position of the image displayed on the display 3 by the page data is displayed by the page data when the coordinates of the upper left corner of the display 3 are (X coordinate value, Y coordinate value) = (0, 0). The position of the upper left corner of the image to be displayed is shown.

  Returning to FIG. 3, the remote license management table storage unit 310 manages license data necessary for executing the remote sharing process. FIG. 10 is a diagram showing an example of a remote license management table stored in the electronic blackboard according to the present embodiment. As shown in FIG. 10, the remote license management table storage unit 310 stores a remote license management table that associates the product ID of the electronic blackboard 2, the license ID used for authentication, and the expiration date of the license ID.

(Functional configuration of the file processing unit 40)
A functional configuration of the file processing unit 40 illustrated in FIG. 3 will be described with reference to FIG. FIG. 4 is a diagram illustrating an example of functional blocks of a file processing unit included in the electronic blackboard according to the present embodiment.

  The file processing unit 40 includes a recovery processing unit 41, a file input unit 42a, a file output unit 42b, a file conversion unit 43, a file transmission unit 44, an address book input unit 45, a backup processing unit 46, a backup output unit 47, and a setting management unit. 48, a setting file input unit 49a, a setting file output unit 49b, and an address book input unit 50. Furthermore, the file processing unit 40 includes an address book management table storage unit 410, a backup data storage unit 420, a setting file storage unit 430, and a connection destination management table storage unit 440.

  Among these, the recovery processing unit 41 detects the abnormal end after the electronic blackboard 2 ends abnormally, and recovers the unsaved page data. For example, in the case of a normal end, page data is recorded as a PDF file in the USB memory 5 via the file processing unit 40, but in the case of an abnormal end such as when the power is down, the page data is stored in the page data storage unit 300. Is still recorded. Therefore, when the power is turned on again, the recovery processing unit 41 recovers by reading the page data from the page data storage unit 300.

  The file input unit 42a reads a PDF file from the USB memory 5 and stores each page in the page data storage unit 300 as page data. The file conversion unit 43 converts the page data stored in the page data storage unit 300 into a PDF format file. The file output unit 42 b records the PDF file output by the file conversion unit 43 in the USB memory 5.

  The file transmission unit 44 transmits the PDF file generated by the file conversion unit 43 by attaching it to an electronic mail. The transmission destination of this file is displayed on the display 3 by the display superimposing unit 36 on the contents of the address book management table stored in the address book management table storage unit 410, and the file transmission unit 44 inputs the touch panel 115 or the like by the user. It is determined by accepting selection of a destination by operating the apparatus. FIG. 11 is a diagram showing an example of an address book management table stored in the electronic blackboard according to the present embodiment. As shown in FIG. 11, the address book management table storage unit 410 stores an address book management table associating a destination name and a mail address of a destination e-mail. Further, the file transmission unit 44 can accept an input of a mail address as a destination by an operation of an input device such as the touch panel 115 by the user.

  The address book input unit 45 reads a list file of e-mail addresses from the USB memory 5 and stores it in the address book management table storage unit 410. The backup processing unit 46 performs backup by storing the file output by the file output unit 42 b and the file transmitted by the file transmission unit 44 in the backup data storage unit 420. If the user does not set backup, backup processing is not performed. FIG. 12 is a diagram showing an example of backup data stored in the electronic blackboard according to the present embodiment. The backup processing unit 46 stores the PDF format backup data in the backup data storage unit 420 as shown in FIG.

  The backup output unit 47 records the backed up file in the USB memory 5. When recording a file in the USB memory 5, the backup output unit 47 performs authentication using a password input for security by the operation of the input device such as the touch panel 115 by the user, and succeeds in the authentication. In this case, the file is recorded in the USB memory 5.

  The setting management unit 48 stores and reads various setting information of the electronic blackboard 2 in the setting file storage unit 430. Examples of the various setting information include network settings, date and time settings, region and language settings, mail server settings, address book settings, connection destination list settings, and backup settings. The network setting is, for example, an IP address setting of the electronic blackboard 2, a net mask setting, a default gateway setting, a DNS (Domain Name System) setting, or the like.

  The setting file output unit 49b stores various setting information of the electronic blackboard 2 in the USB memory 5 as a setting file. The contents of the configuration file cannot be viewed by the user from the viewpoint of security. The setting file input unit 49 a reads a setting file stored in the USB memory 5 and reflects various setting information on various settings on the electronic blackboard 2.

  The address book input unit 50 reads a list file of connection destination IP addresses for remote sharing processing from the USB memory 5 and manages it in the connection destination management table storage unit 440. FIG. 13 is a diagram illustrating an example of a connection destination management table stored in the electronic blackboard according to the present embodiment. As shown in FIG. 13, the connection destination management table stored in the connection destination management table storage unit 440 is that the user of the participation device serves as a sponsor device when the electronic blackboard 2 is a participation device that intends to participate in the remote sharing process. In order to reduce the trouble of inputting the IP address of the electronic blackboard 2 that plays the role of, it is a table that stores in advance the IP address of the electronic blackboard 2 serving as the hosting device. In the connection destination management table, the name of the base where the electronic blackboard 2 as the hosting device that can participate is installed and the IP address of the electronic blackboard 2 as the hosting device are managed in association with each other.

  The connection destination management table may be omitted. However, when the connection destination management table is not stored, the user of the participating device inputs the IP address of the sponsoring device using the input device such as the touch panel 115 in order to start remote request processing with the hosting device. There is a need. Therefore, the user of the participating device acquires the IP address of the hosting device from the user of the hosting device by telephone or e-mail.

(Functional configuration of communication control unit 60)
Next, the functional configuration of the communication control unit 60 will be described with reference to FIG. FIG. 5 is a diagram illustrating an example of functional blocks of a communication control unit included in the electronic blackboard according to the present embodiment.

  The communication control unit 60 controls communication with other electronic whiteboards 2 and communication with the later-described communication control unit 70 in the server unit 90 via the communication network 9. Therefore, the communication control unit 60 includes a remote start processing unit 61, a remote participation processing unit 62, a remote image transmission unit 63, a remote image reception unit 64, a remote operation transmission unit 65, a remote operation reception unit 66, and a participation base management table storage. Part 610.

  Among these, the remote start processing unit 61 requests the server unit 90 of the same electronic blackboard 2 to start a new remote sharing process, and receives the request result from the server unit 90. In this case, the remote start processing unit 61 refers to the remote license management table storage unit 310, and if license information (product ID, license ID, and expiration date) is stored, a request to start remote sharing processing It can be performed. However, if the license information is not stored, a request to start the remote sharing process cannot be made.

  The participation site management table stored in the participation site management table storage unit 610 is a table that stores an electronic blackboard as a participating device currently participating in the remote sharing process when the electronic blackboard is the host device. FIG. 14 is a diagram illustrating an example of a participating site management table stored in the electronic blackboard according to the present embodiment. In this participating site management table, as shown in FIG. 14, the name of the site where the participating electronic blackboard 2 is installed and the IP address of the electronic blackboard 2 are managed in association with each other.

  The remote participation processing unit 62 sends the remote connection processing to the remote sharing processing to the remote connection request receiving unit 71 in the server unit 90 of the electronic blackboard 2 as the host apparatus that has already started the remote sharing processing via the communication network 9. Request participation. Also in this case, the remote participation processing unit 62 refers to the remote license management table storage unit 310. When the remote participation processing unit 62 participates in the remote sharing process that has already been started, the connection destination management table storage unit 440 is referred to obtain the IP address of the electronic blackboard 2 as the participation destination. The connection destination management table may not be referred to by the remote participation processing unit 62, and the IP address of the electronic blackboard 2 as the participation destination may be input by the user operating the input device such as the touch panel 115.

  The remote image transmission unit 63 transmits at least one of the output image C (see FIG. 16) and the captured image sent from the video acquisition unit 21 via the image acquisition unit 31 to the server unit 90. The remote image receiving unit 64 receives at least one of the image data from the video output device connected to the other electronic blackboard 2 and the captured image obtained by the camera c of the other electronic blackboard 2 from the server unit 90. Then, the remote sharing process is enabled by outputting to the display superimposing unit 36 via the image acquiring unit 31.

  The remote operation transmission unit 65 transmits various operation data necessary for remote sharing processing to the server unit 90. Examples of the operation data include adding a stroke, deleting a stroke, editing a stroke (enlarging, reducing, moving), storing page data, creating page data, duplicating page data, deleting page data, and displaying. Data related to the switching of the current page. Further, the remote operation receiving unit 66 receives the operation data input from another electronic blackboard 2 from the server unit 90 and outputs it to the image processing unit 30 to perform remote sharing processing.

[Functional configuration of server part]
Next, the functional configuration of the server unit 90 will be described with reference to FIG. The server unit 90 is provided in each electronic blackboard 2, and any electronic blackboard 2 can serve as a server. Therefore, the server unit 90 includes a communication control unit 70 and a data management unit 80.

(Functional configuration of communication control unit 70)
Next, the functional configuration of the communication control unit 70 will be described with reference to FIG. The communication control unit 70 controls communication with the communication control unit 60 in the client unit 20 in the same electronic blackboard 2 and the communication control unit 70 in the client unit 20 in another electronic blackboard 2 via the communication network 9. The data management unit 80 manages operation data, image data, and the like.

  More specifically, the communication controller 70 includes a remote connection request receiver 71, a remote connection result transmitter 72, a remote image receiver 73, a remote image transmitter 74, a remote operation receiver 75, and a remote operation transmitter 76. have. The remote connection request receiving unit 71 receives a remote sharing processing start request from the remote start processing unit 61 or receives a participation request for remote sharing processing from the remote participation processing unit 62. The remote connection result transmission unit 72 transmits the result of the remote sharing processing start request to the remote start processing unit 61, or transmits the result of the participation request for the remote sharing processing to the remote participation processing unit 62.

  The remote image receiving unit 73 receives image data (at least one of the output image C (see FIG. 16) and the captured image) from the remote image transmitting unit 63 and transmits the image data to a remote image processing unit 82 described later. The remote image transmission unit 74 receives image data from the remote image processing unit 82 and transmits this image data to the remote image reception unit 64. The remote operation receiving unit 75 receives operation data (data such as a stroke image B (see FIG. 16)) from the remote operation transmitting unit 65 and transmits it to a remote operation processing unit 83 described later. The remote operation transmission unit 76 receives operation data from the remote operation processing unit 83 and transmits this operation data to the remote operation reception unit 66.

(Functional configuration of the data management unit)
Next, the functional configuration of the data management unit 80 will be described with reference to FIG. The data management unit 80 includes a remote connection processing unit 81, a remote image processing unit 82, a remote operation processing unit 83, an operation composition processing unit 84, and a page processing unit 85. The data management unit 80 further includes a passcode management unit 810, a participating site management table storage unit 820, an image data storage unit 830, an operation data storage unit 840, and a page data storage unit 850.

  Among these, the remote connection processing unit 81 starts remote sharing processing and ends remote sharing processing. The remote connection processing unit 81 receives the license information received from the remote start processing unit 61 together with the remote sharing processing start request from the remote connection request receiving unit 71 or received from the remote participation processing unit 62 together with the remote sharing processing participation request. Based on the license information, it is confirmed whether there is a license and whether it is within the license period. Further, the remote connection processing unit 81 confirms whether or not the participation request from the other electronic blackboard 2 as the client exceeds the predetermined number of possible participations.

  Further, the remote connection processing unit 81 has the same passcode that is sent by the other electronic blackboard 2 when the participation request for the remote sharing processing is made, and is the same as the passcode managed by the passcode management unit 810. It is determined whether or not there is the same, and if they are the same, participation in the remote sharing process is permitted. This passcode is issued by the remote connection processing unit 81 when a new remote sharing process is started, and the user of the electronic blackboard 2 as a participating device who wants to participate in the remote sharing processing is notified by the electronic blackboard as the hosting device. 2 users are notified by telephone or e-mail. As a result, the user of the participating device who intends to participate in the remote sharing process inputs the pass code to the participating device by using the input device such as the touch panel 115 and requests participation, thereby permitting participation. Prioritizing the user's usability over security, the confirmation of the passcode may be omitted only by confirming the license status.

  When the electronic blackboard 2 is the host device, the remote connection processing unit 81 uses the participation base information included in the participation request sent from the remote participation processing unit 62 of the participating device via the communication network 9 as the server unit. 90 of the participating base management table storage units 820. Then, the remote connection processing unit 81 reads out the remote site information stored in the participating site management table storage unit 820 and transmits it to the remote connection result transmission unit 72. The remote connection result transmission unit 72 transmits the remote base information to the remote start processing unit 61 in the client unit 20 of the same sponsor device. The remote start processing unit 61 stores the remote site information in the participating site management table storage unit 610. As a result, in the hosting device, the remote site information is managed by both the client unit 20 and the server unit 90.

  The remote image processing unit 82 receives the image data (at least one of the output image C (see FIG. 16) and the captured image) acquired by the client unit 20 (image acquisition unit 31) of each electronic blackboard 2 during the remote sharing process. Then, the image data storage unit 830 stores the image data, and the display order of the image data to be remotely shared is determined in the order of time of arrival at the server unit 90 of the electronic blackboard 2 that is the host device. Further, the remote image processing unit 82 (an example of a transmission unit) refers to the participating base management table storage unit 820, and the client units 20 (sponsorships) of all the electronic blackboards 2 (an example of external devices) participating in the remote sharing process. The image data is transmitted to the apparatus (including the client unit 20 of its own electronic blackboard 2) via the communication control unit 70 (remote image transmission unit 74) in the determined order.

  The remote operation processing unit 83 performs various types of operation data (stroke images, etc.) drawn by the client unit 20 of each electronic blackboard 2 (including the client unit 20 of its own electronic blackboard that is the hosting device) during remote sharing processing. B (see FIG. 16)) is received, and the display order of the images to be remotely shared is determined in the order in which they arrived at the server unit 90 of the electronic blackboard 2 that is the host apparatus. The various operation data is the same as the above-described various operation data. Further, the remote operation processing unit 83 refers to the participating site management table storage unit 820, and includes the client units 20 of all the electronic blackboards 2 that are in the process of remote sharing (including the client unit 20 of its own electronic blackboard that is the host device). Send operation data to.

  The operation synthesis processing unit 84 synthesizes the operation data of each electronic blackboard 2 output from the remote operation processing unit 83, stores the operation data as a result of this synthesis in the operation data storage unit 840 and the remote operation processing unit 83. Return to. The operation data is transmitted from the remote operation transmission unit 76 to each of the client unit 20 of the electronic blackboard 2 that is the hosting device and the client unit 20 of the electronic blackboard 2 that is the participating device. An image related to the same operation data is displayed.

  FIG. 15 is a diagram illustrating an example of operation data used for displaying an image on the electronic blackboard according to the present embodiment. As shown in FIG. 15, the operation data includes SEQ (Sequence), the operation name of the operation data, the IP address of the electronic blackboard 2 that is the transmission source of the operation data, and the client unit 20 (or the server unit 90). The port number of the electronic blackboard 2 to which the operation data is transmitted, the port number of the client unit 20 (or the server unit 90), the operation type of the operation data, the operation target of the operation data, Data indicating the contents of the operation data are shown in association with each other.

  For example, in SEQ1, when a stroke is drawn on the client unit 20 (Port No .: 50001) of the electronic blackboard (IP address: 192.0.0.1) that is the hosting device, the electronic blackboard (IP address: 192.0) that is the same hosting device. .0.1) indicates that operation data has been sent to the server unit 90 (Port No .: 50000). In this case, the operation type is “STROKE”, the operation target is the page data ID “p005”, and the data indicating the contents of the operation data is data indicating the stroke. In SEQ2, a client of an electronic blackboard (IP address: 192.0.0.1) that is a participating device from a server unit 90 (Port No .: 50000) of an electronic blackboard (IP address: 192.0.0.1) that is a host device. The part 20 (Port No .: 50001) indicates that the operation data has been sent.

  Since the operation composition processing unit 84 performs composition in the order in which operation data is input to the operation composition processing unit 84, if the communication network 9 is not congested, remote sharing processing is being performed in the order of strokes by the user of each electronic blackboard 2. A stroke image B (see FIG. 16) is displayed on the displays 3 of all the electronic blackboards 2. The page processing unit 85 has the same function as the page processing unit 37 in the image processing unit 30 of the client unit 20, and the server unit 90 also stores the page data illustrated in FIGS. 6 to 8 in the page data storage unit 850. . The page data storage unit 850 has the same contents as the page data storage unit 300 in the image processing unit 30 and thus will not be described.

<Process or Operation of Embodiment>
Subsequently, the processing or operation of the present embodiment will be described with reference to FIGS. FIG. 16 is a diagram illustrating an example of an image displayed on the electronic blackboard according to the present embodiment. 17 and 18 are sequence diagrams illustrating processing of the electronic blackboard according to the present embodiment.

  In the embodiment shown in FIGS. 17 and 18, the electronic blackboard 2a serves as a hosting device (server unit 90 and client unit 20) that hosts the remote sharing process, and the electronic blackboards 2b and 2c perform the remote sharing process. The case where it plays the role as a participating apparatus (client part 20) to participate is demonstrated. Further, here, displays 3a, 3b, 3c are connected to the electronic blackboards 2a, 2b, 2c, respectively, and notebook PCs 6a, 6b, 6c are connected to the electronic blackboards 2a, 2b, 2c, respectively. Moreover, in the electronic blackboards 2a, 2b, and 2c, electronic pens 4a, 4b, and 4c are used, respectively.

(Participation process)
First, a process for the electronic blackboards 2b and 2c to participate in the remote sharing process will be described with reference to FIG. When the user turns on the power switch of the electronic blackboard 2a, the client unit 20 of the electronic blackboard 2a is activated. When the user performs an operation to activate the server unit 90 using an input device such as the touch panel 115, the remote start request unit 61 of the client unit 20 transfers the server unit to the remote connection request receiving unit 71 in the server unit 90 of the same electronic blackboard 2a. An instruction to start 90 processing is output. Thereby, in the electronic blackboard 2a, not only the client unit 20 but also the server unit 90 can start various processes (step S21).

  Next, the UI image generating unit 33 in the client unit 20 of the electronic blackboard 2a generates connection information for establishing a connection with the electronic blackboard 2a, and the video superimposing unit 28 transmits the display superimposing unit 36 from the UI image generating unit 33. The connection information obtained through is displayed on the display 3a (step S22). This connection information includes the IP address of the host device and the passcode generated for the current remote sharing process. In this case, the passcode stored in the passcode management unit 810 is read by the remote connection processing unit 81 shown in FIG. 5 and transmitted in the order of the remote connection result transmission unit 72 and the remote start processing unit 61. Further, the passcode is transmitted from the communication control unit 60 including the remote start processing unit 61 to the image processing unit 30 shown in FIG. 3 and finally input to the UI image generation unit 33. Thereby, the connection information includes the passcode. Then, the connection information is transmitted to the users of the electronic blackboards 2b and 2c by telephone or electronic mail by the user of the electronic blackboard 2a. With the connection destination management table storage unit 440, even if the connection information does not include the IP address of the host device, the participating device can make a participation request.

  Next, in the electronic blackboards 2b and 2c, when the input of connection information is received by the operation of the input device such as the touch panel 115 by each user, the remote participation processing unit 62 in the client unit 20 of each electronic blackboard 2b and 2c Based on the IP address, a pass code is transmitted to the communication control unit 70 in the server unit 90 of the electronic blackboard 2a via the communication network 9 to make a participation request (steps S23 and S24). As a result, the remote connection request receiving unit 71 of the communication control unit 70 receives a participation request (including a passcode) from each of the electronic blackboards 2b and 2c, and outputs this passcode to the remote connection processing unit 81.

  Next, the remote connection processing unit 81 authenticates the passcode received from each of the electronic blackboards 2b and 2c using the passcode managed by the passcode management unit 810 (step S25). And the remote connection result transmission part 72 notifies an authentication result to the client part 20 of each electronic blackboard 2b, 2c (step S26, S27). If the electronic blackboards 2b and 2c are determined to be legitimate electronic blackboards by the authentication in step S25, remote sharing between the electronic blackboard 2a that is the hosting device and the electronic blackboards 2b and 2c that are the participating devices Processing communication is established, and the remote participation processing unit 62 in the client unit 20 of each electronic blackboard 2b, 2c enables the start of remote sharing processing with each of the other electronic blackboards (steps S28 and S29).

(Output image display)
Next, a process for displaying the output image C in the remote sharing process will be described with reference to FIG. First, the electronic blackboard 2b displays the output image C on the display 3b (step S30). Specifically, the image acquisition unit 31 of the electronic blackboard 2b receives the output image C displayed on the notebook PC 6b from the notebook PC 6b via the video acquisition unit 21, and displays the display superimposing unit 36 and the video superimposing unit 28. The display 3b displays the output image C by transmitting to the display 3b.

  Next, the image processing unit 30 including the image acquisition unit 31 of the electronic blackboard 2b transmits the output image C to the remote image transmission unit 63, so that the communication control unit 60 including the remote image transmission unit 63 establishes the communication network 9. Then, the output image C is transmitted to the communication control unit 70 of the electronic blackboard 2a which is the hosting device (step S31). Thereby, the remote image receiving unit 73 of the electronic blackboard 2a receives the output image C and outputs it to the remote image processing unit 82, whereby the remote image processing unit 82 stores the output image C in the image data storage unit 830. .

  Next, the electronic blackboard 2a, which is the hosting device, displays the output image C on the display 3a (step S32). Specifically, the remote image processing unit 82 of the electronic blackboard 2 a outputs the output image C received from the remote image receiving unit 73 to the remote image transmitting unit 74. The remote image transmission unit 74 outputs the output image C to the remote image reception unit 64 in the client unit 20 of the electronic blackboard 2a which is the same host device. The remote image receiving unit 64 outputs the output image C to the display superimposing unit 36. The display superimposing unit 36 outputs the output image C to the video superimposing unit 28. The video superimposing unit 28 outputs the output image C to the display 3a. Thereby, the display 3a displays the output image C.

  Next, the communication control unit 70 including the remote image transmission unit 74 in the server unit 90 of the electronic blackboard 2a as the hosting device transmits the electronic blackboard other than the electronic blackboard 2b that is the transmission source of the output image C via the communication network 9. The output image C is transmitted to the communication control unit 60 of 2c (step S33). Thereby, the remote image receiving part 64 of the electronic blackboard 2c which is a participating apparatus receives the output image C.

  Next, the electronic blackboard 2c displays the output image C on the display 3c (step S34). Specifically, the remote image receiving unit 64 of the electronic blackboard 2c outputs the output image C received in step S33 to the display superimposing unit 36 of the electronic blackboard 2c. The display superimposing unit 36 outputs the output image C to the video superimposing unit 28. The video superimposing unit 28 outputs the output image C to the display 3c. Thereby, the display 3c displays the output image C.

  When not only the output image C but also each data of the UI image A and the stroke image B is input to the video superimposing unit 28, the display superimposing unit 36, as shown in FIG. A superimposed image that is an image in which the image A and the stroke image B are superimposed is generated. The video superimposing unit 28 outputs the superimposed image data to the display 3c. In addition, when a captured image for video conference is sent from the video conference terminal 7 to the video superimposing unit 28, the video superimposing unit 28 captures the video for video conference by picture-in-picture on the superimposed image. The image is superimposed and output to the display 3c.

(Display of superimposed image)
Next, a process for displaying a superimposed image in the remote sharing process will be described with reference to FIG. First, in the electronic blackboard 2b, the user draws a stroke image B on the electronic blackboard 2b using the electronic pen 4b (step S41). Next, as shown in FIG. 16, the display superimposing unit 36 of the electronic blackboard 2b displays a superimposed image obtained by superimposing the stroke image B on the UI image A and the output image C. It is displayed on the display 3b (step S42). Specifically, the stroke processing unit 32 of the electronic blackboard 2 b receives the stroke image B as the operation data from the coordinate detection unit 22 and the contact detection unit 24 via the event distribution unit 25, and sends it to the display superimposing unit 36. Send. Thereby, the display superimposing unit 36 can superimpose the stroke image B on the UI image A and the output image C, and the video superimposing unit 28 can display the superimposed image on the display 3b of the electronic blackboard 2b. it can.

  Next, the image processing unit 30 including the stroke processing unit 32 of the electronic blackboard 2 b transmits the stroke image B to the remote operation transmission unit 65, so that the remote operation transmission unit 65 of the electronic blackboard 2 b passes through the communication network 9. Then, the stroke image B is transmitted to the communication control unit 70 of the electronic blackboard 2a which is the hosting device (step S43). Thereby, the remote operation receiving unit 75 of the electronic blackboard 2a receives the stroke image B and outputs it to the remote operation processing unit 83, so that the remote operation processing unit 83 outputs the stroke image B to the operation composition processing unit 84. . In this way, the stroke image B drawn on the electronic blackboard 2b is sequentially transmitted to the remote operation processing unit 83 of the electronic blackboard 2a that is the hosting device every time it is drawn. This stroke image B is data shown for each stroke data ID shown in FIG. Therefore, for example, as described above, when the user draws the alphabet “T” with the electronic pen 4, the stroke image B indicated by each of the two stroke data IDs is sequentially transmitted. .

  Next, the electronic blackboard 2a, which is the hosting device, displays a superimposed image including the stroke image B sent from the electronic blackboard 2b on the display 3a (step S44). Specifically, the operation composition processing unit 84 of the electronic blackboard 2a synthesizes a plurality of stroke images B sequentially sent via the remote operation processing unit 83 and stores them in the operation data storage unit 840 and remotely. Return to the operation processing unit 83. Thereby, the interval operation processing unit 83 outputs the combined stroke image B received from the operation synthesis processing unit 84 to the remote operation transmission unit 76. The remote operation transmitting unit 76 outputs the combined stroke image B to the remote operation receiving unit 66 in the client unit 20 of the electronic blackboard 2a which is the same hosting device. The remote operation receiving unit 66 outputs the combined stroke image B to the display superimposing unit 36 in the image processing unit 30. Therefore, the display superimposing unit 36 superimposes the combined stroke image B on the UI image A and the output image C. Finally, the video superimposing unit 28 displays the superimposed image on which the stroke image B is superimposed by the display superimposing unit 36 on the display 3a.

  Next, the communication control unit 70 including the remote operation transmission unit 76 in the server unit 90 of the electronic blackboard 2a as the hosting device transmits the electronic blackboard other than the electronic blackboard 2b that is the transmission source of the stroke image B via the communication network 9. The combined stroke image B is transmitted to the communication control unit 60 of 2c (step S45). Thereby, the remote operation receiving part 66 of the electronic blackboard 2c which is a participating apparatus receives the synthesized stroke image B.

  Next, the electronic blackboard 2c displays a superimposed image on the display 3c (step S46). Specifically, the remote operation receiving unit 66 of the electronic blackboard 2c outputs the combined stroke image B received in step S45 to the image processing unit 30 of the electronic blackboard 2c. The display superimposing unit 36 of the image processing unit 30 outputs a superimposed image in which the combined stroke image B is superimposed on the data of the UI image A and the output image C to the video superimposing unit 28. The video superimposing unit 28 outputs the superimposed image to the display 3c. Thereby, the display 3c displays a superimposed image.

  In the above process, the output image C is displayed on the display 3, but a background image D may be displayed instead of the output image C as shown in FIG. Alternatively, the exclusive relationship between the output image C and the background image D may be stopped, and both the output image C and the background image D may be displayed on the display 3 at the same time.

(End of participation)
Next, a process in which the participating device ends participation in the remote sharing process will be described with reference to FIG. In the embodiment shown in FIG. 18, a process for the electronic blackboard 2c to finish participation is shown. First, in the electronic blackboard 2c, when a participation end request is received by the operation of the input device such as the touch panel 115 by the user, the remote participation processing unit 62 sends the communication control unit 70 in the server unit 90 of the electronic blackboard 2a as the hosting device. Then, an end request for participation is made (step S47). Thereby, the remote connection request receiving unit 71 of the communication control unit 70 receives the participation end request from the electronic blackboard 2c, and outputs the participation end request together with the IP address of the electronic blackboard 2c to the remote connection processing unit 81. . Then, based on the IP address sent from the remote connection request receiving unit 71, the remote connection processing unit 81 of the electronic blackboard 2a sends an end request for participation from the participating base management table storage unit 820. And the name of the base where the electronic blackboard 2c is installed are deleted, and the remote connection result transmission unit 72 outputs the IP address of the electronic blackboard 2c and a notification that the electronic blackboard 2c has been deleted.

  Next, the communication control unit 70 including the remote connection result transmission unit 72 instructs the communication control unit 60 in the client unit 20 of the electronic blackboard 2c to end participation via the communication network 9 (step S48). As a result, the remote participation processing unit 62 of the communication control unit 60 in the electronic blackboard 2c disconnects the communication of the remote sharing process, thereby completing the participation process, thereby completing the participation (step S49).

  Next, a method for controlling the imaging range of the camera c by the electronic blackboard 2 according to the present embodiment will be described with reference to FIGS. 19 and 20. FIG. 19 is a diagram for explaining an example of a method for controlling the imaging range of a camera in a conventional electronic blackboard. FIG. 20 is a diagram for explaining an example of a method of controlling the imaging range of the camera using the electronic blackboard according to the present embodiment.

  As shown in FIG. 19A, when the writer's touch operation on the left area of the display 3 is detected by the touch panel 115 when viewed from the front of the display 3, the Web camera 130 is moved from the front of the display 3. When viewed and moved to the left side, most of the imaging range h of the Web camera 130 is occupied by the writer, and the Web camera 130 can image other attendees attending a meeting or the like. Can not. In addition, as shown in FIG. 19B, when the touch panel 115 detects a writer's touch operation on the right area of the display 3 when viewed from the front of the display 3, the Web camera 130 is connected to the display 3. When viewed from the front and moved to the right side, most of the imaging range H of the Web camera 130 is occupied by the writer, and the Web camera 130 images other attendees attending a meeting or the like. I can't.

  Therefore, in the present embodiment, as shown in FIG. 20A, when the touch panel 115 detects a writer's touch operation on the left area of the display 3 when viewed from the front of the display 3, the control unit 121 moves the Web camera 130 to the right side when viewed from the front of the display 3. Thereby, the control unit 121 also moves the imaging range h of the Web camera 130 to the right side when viewed from the front of the display 3. On the other hand, as illustrated in FIG. 20B, when the touch panel 115 detects a writer's touch operation with respect to the area on the right side of the display 3 as viewed from the front of the display 3, the control unit 121 displays the web camera 130. Is moved to the left side as viewed from the front of the display 3. Thereby, the control unit 121 also moves the imaging range h of the Web camera 130 to the left as viewed from the front of the display 3.

  That is, the control unit 121 moves the imaging range h of the Web camera 130 in the left-right direction of the display 3 in the direction opposite to the area where the touch operation is detected by the touch panel 115 (an example of a detection position). Thereby, since most of the imaging range h of the Web camera 130 can be prevented from being occupied by the writer, other attendees attending a meeting or the like can also be imaged.

  Next, a flow of a method for controlling the imaging range of the camera in the electronic blackboard according to the present embodiment will be described with reference to FIG. FIG. 21 is a flowchart illustrating an example of a flow of a method for controlling the imaging range of the camera using the electronic blackboard according to the present embodiment.

  When instructed to execute the whiteboard application and the video conference application, the control unit 121 first activates the touch panel 115 (step S2101). Next, the controller 121 rotates the motor 132 to move the Web camera 130 to the center of the upper end of the display 3 in the left-right direction of the display 3 (step S2102). Thereafter, the control unit 121 waits until a touch operation on the display 3 is detected by the touch panel 115 (step S2103).

  When the touch operation on the display 3 is detected by the touch panel 115 (step S2103: Yes), the control unit 121 obtains the coordinates of the detection position where the touch operation is detected on the display 3 from the coordinate detection unit 22. Obtain (step S2104). Next, the control unit 121 determines whether or not the acquired coordinates are on the left side when viewed from the front of the display 3 on the display 3 (step S2105). When the acquired coordinates are on the left side when viewed from the front of the display 3 on the display 3 (step S2105: Yes), the control unit 121 rotates the motor 132 to display the web camera 130 on the display 3 3 is moved to the right side as viewed from the front (step S2106).

  On the other hand, when the acquired coordinates are on the right side of the display 3 when viewed from the front of the display 3 (step S2105: No), the control unit 121 rotates the motor 132 to rotate the web camera 130. When viewed from the front of the display 3, it is moved to the left side (step S2107). Next, the control unit 121 determines whether or not the touch operation is detected by the touch panel 115 and contact with the display 3 with the electronic pen 4 or the like is released (step S2108). When the touch operation is detected by the touch panel 115 (step S2108: No), the control unit 121 returns to step S2104, and the coordinate of the detection position where the touch operation is newly detected on the display 3 from the coordinate detection unit 22. To get.

  On the other hand, when the touch operation is no longer detected by the touch panel 115 (step S2108: Yes), the control unit 121 is instructed to end the whiteboard application and the video conference application, and the touch operation detection processing by the touch panel 115 is ended. Whether or not (step S2109). If the touch operation detection process using the touch panel 115 has not ended (step S2109: NO), the control unit 121 returns to step S2103 and waits for a new touch operation to be detected by the touch panel 115. On the other hand, when the detection process of the touch operation by the touch panel 115 is finished (step S2109: Yes), the control unit 121 finishes the process for controlling the position of the Web camera 130.

  Thus, according to the electronic blackboard 2 according to the present embodiment, even when writing in the vicinity of the electronic blackboard 2, it is possible to prevent the writer from occupying most of the imaging range h of the Web camera 130. Therefore, other attendees who are attending a meeting or the like can also be imaged.

  Note that the program executed by the electronic blackboard 2 of the present embodiment is provided by being incorporated in advance in a ROM or the like. The program executed on the electronic blackboard 2 of the present embodiment is a file in an installable format or an executable format, and is a computer such as a CD-ROM, a flexible disk (FD), a CD-R, a DVD (Digital Versatile Disk). The information may be provided by being recorded on a recording medium that can be read by the user.

  Furthermore, the program executed on the electronic blackboard 2 of the present embodiment may be configured to be stored by being stored on a computer connected to a network such as the Internet and downloaded via the network. Further, the program executed on the electronic blackboard 2 of the present embodiment may be configured to be provided or distributed via a network such as the Internet.

2 Electronic blackboard 3 Display 101 CPU
102 ROM
103 RAM
104 SSD
105 Network I / F
111 Capture device 112 GPU
113 Display Controller 114 Sensor Controller 115 Touch Panel 116 Electronic Pen Controller 120 Bus Line 121 Control Unit 127 Infrared LED
128 Camera 129 Touch Panel Driver 130 Web Camera 132 Motor c Camera

JP 2009-194900 A

Claims (4)

A display screen;
A detection unit for detecting a touch operation on the display screen;
A camera capable of capturing an imaging range in front of the display screen and capable of moving the imaging range in the left-right direction of the display screen;
A transmission unit that transmits a captured image obtained by imaging the front of the display screen by the camera to an external device;
A control unit configured to move the imaging range in a direction opposite to a detection position where the touch operation is detected by the detection unit in the left-right direction of the display screen;
A display device comprising:   The display device according to claim 1, wherein the camera moves the imaging range by moving in a horizontal direction of the display screen. Detect touch operations on the display screen,
A captured image obtained by imaging the front of the display screen by a camera that is provided so as to be able to capture the imaging range in front of the display screen and is movable in the left-right direction of the display screen. Send
Moving the imaging range in a direction opposite to the detection position where the touch operation is detected in the left-right direction of the display screen;
A camera position control method. Computer
A detection unit for detecting a touch operation on the display screen;
A captured image obtained by imaging the front of the display screen by a camera that is provided so as to be able to capture the imaging range in front of the display screen and is movable in the left-right direction of the display screen. A transmission unit for transmission;
A control unit configured to move the imaging range in a direction opposite to a detection position where the touch operation is detected by the detection unit in the left-right direction of the display screen;
Program to function as.

Images