JP6911595B2 - Communication terminals, communication systems, output methods, and programs - Google Patents

Description

The present invention relates to communication terminals, communication systems, output methods, and programs.

In recent years, a communication system for making a call or a conference via a communication network such as the Internet or a dedicated line has become widespread in response to a request for reducing the cost and time of moving a party. In such a communication system, one or more content data such as video data of conference participants, image data such as conference materials, or stroke data based on manual operation is transmitted between communication terminals according to the function of the communication terminal. can do.

Patent Document 1 discloses a method of transmitting image data by a session established between an electronic blackboard and a telephone terminal.

However, when terminating the communication for transmitting and receiving images related to a plurality of content data, the user must perform an operation for terminating each communication from the screen for each operation of the plurality of communications. Usability issues arise.

The communication terminal of the invention according to claim 1 is a first image relating to stroke data transmitted or received in a first communication with another communication terminal, and a second image with another communication terminal. A display instruction image that receives a transmission / reception means for transmitting or receiving a second image related to video data transmitted or received by communication, and an instruction for displaying at least one of the first image and the second image. Display of at least one image of the first image and the second image in response to the instruction, and a stop instruction image for receiving the instruction to stop the first communication and the second communication. It has a means, a stop means for stopping both the first communication and the second communication when the stop instruction is received, and a management means for managing the output of the stroke data. When the management means determines that the stroke data is stored when the stop instruction is received, the display means gives priority to the first image over the second image. Display it.

According to the present invention, there is an effect that usability at the time of terminating communication for transmitting / receiving images related to a plurality of content data is improved.

It is an overall block diagram of the communication system which concerns on one Embodiment. It is a hardware block diagram of the electronic blackboard which concerns on one Embodiment. It is a hardware block diagram of the management system which concerns on one Embodiment. It is a hardware block diagram of the PC as a video conferencing terminal which concerns on one Embodiment. It is a software block diagram of the electronic blackboard or PC which concerns on one Embodiment. It is a functional block diagram of an electronic blackboard, a PC, and a management system which form a part of the communication system which concerns on one Embodiment. It is a functional block diagram which showed the drawing control part which concerns on one Embodiment. It is a sequence diagram which shows the process of establishing communication between electronic blackboards. It is a sequence diagram which shows the process of establishing the communication between an electronic blackboard and a PC. It is a flow chart which shows the process of generating an image. It is a conceptual diagram which shows the superimposition image. It is a conceptual diagram which shows the superimposition image. It is a sequence diagram which shows the process which controls the display order of an image. It is a sequence diagram which shows the process which controls the display order of an image. This is a display example. This is a display example. It is a sequence diagram which shows the processing of each functional block of the electronic blackboard 2c at the time of ending a meeting. This is a display example. It is a flow chart which shows the processing of an electronic blackboard.

Hereinafter, the first embodiment of the present invention will be described with reference to the drawings.

<< Overall configuration of communication system >>
First, the communication system 1 will be described with reference to FIG. FIG. 1 is an overall configuration diagram of a communication system according to an embodiment.

Communication system 1 includes a plurality of electronic blackboards 2a, 2b, 2c, PC8, and a management system 50.

The electronic blackboards 2a, 2b, and 2c are communicably connected via the communication network 9a. Further, the electronic blackboards 2a, 2b, and 2c are provided with displays 3a, 3b, and 3c, respectively.

The electronic blackboards 2a, 2b, 2c are drawn by an event generated by the electronic pens 4a, 4b, 4c (the pen tip of the electronic pens 4a, 4b, 4c on the displays 3a, 3b, 3c, or the touch of the pen tail). The image can be displayed on the displays 3a, 3b, 3c. The electronic blackboards 2a, 2b, 2c are based on not only the electronic pens 4a, 4b, 4c but also events (gestures such as enlargement, reduction, page turning, etc.) caused by the user's hands Ha, Hb, Hc, etc. It is also possible to change the image displayed on the displays 3a, 3b, 3c. Hereinafter, when any of the electronic blackboards 2a, 2b, and 2c is indicated, it is referred to as the electronic blackboard 2. When any display among the displays 3a, 3b, and 3c is shown, it is referred to as the display 3. Further, when any electronic pen among the electronic pens 4a, 4b, and 4c is indicated, it is indicated as the electronic pen 4. Further, when an arbitrary hand among the user's hands Ha, Hb, and Hc is indicated, it is indicated as a hand H.

Further, USB (Universal Serial Bus) memories 5a, 5b, 5c can be connected to the electronic blackboards 2a, 2b, 2c. The electronic blackboards 2a, 2b, 2c can read electronic files such as PDF (Portable Document Format) from the USB memories 5a, 5b, 5c, and can record the electronic files in the USB memories 5a, 5b, 5c. In addition, the electronic blackboard 2a is connected to a notebook via a cable capable of communicating according to standards such as DisplayPort, DVI (Digital Visual Interface), HDMI (registered trademark) (High-Definition Multimedia Interface) and VGA (Video Graphics Array). PC6a, 6b, 6c are connected. When any USB memory among the USB memories 5a, 5b, and 5c is indicated, it is referred to as USB memory 5. When any notebook PC among the notebook PCs 6a, 6b, and 6c is indicated, it is indicated as the notebook PC 6.

As a result, the stroke data of the stroke drawn on the display 3 of the electronic blackboard 2 at one base is transmitted to the electronic blackboard 2 at the other base via the communication network 9a, so that the display 3 at the other base It is also displayed above. On the contrary, the stroke data of the stroke drawn on the display 3 of the electronic blackboard 2 at another base is transmitted to the electronic blackboard 2 of one base via the communication network 9a, so that the electronic blackboard 2 of one base Is displayed on the display 3. As described above, since the communication system 1 can perform remote sharing processing for sharing the same image at a remote location, it is very convenient to use it for a conference or the like at a remote location.

In addition to the electronic blackboard 2 described above, stroke data can be shared with the electronic blackboard 2 by connecting a communication terminal other than the electronic blackboard to the communication network 9a. Other communication terminals include PCs with stroke sharing functions, video conferencing terminals, tablets, smartphones, electronic signage (digital signage), telestrator used for sports and weather forecasts, or remote image (video) diagnostic equipment. Such as an image processing device, a personal digital assistant, a digital video camera, a digital camera, a computer such as a terminal capable of supplying an image frame such as a game machine.

The electronic blackboard 2c, the PC (Personal computer) 8, and the management system 50 are communicably connected via the communication network 9b.

The electronic blackboard 2c and the PC 8 communicate by transmitting and receiving video data and sound data for video conferencing as an example of content data.

The management system 50 centrally manages the communication status of the electronic blackboard 2c and the PC 8.

By connecting a communication terminal other than the electronic blackboard 2c and the PC 8 to the communication network 9b, video data can be shared between the other communication terminal and the electronic blackboard 2c and the PC 8. Other communication terminals include video conference terminals, tablets, smartphones, electronic signage (digital signage), telestaters, image processing devices, mobile information terminals, digital video cameras, digital cameras, game machines, etc. that have a video sharing function. A computer such as a terminal capable of supplying the image frame of the above is included.

The electronic blackboard 2c simultaneously activates the stroke data, the video data, and the sound data between the terminals by activating both the communication application for sharing the stroke data and the communication application for sharing the video data. Can be shared.

According to one embodiment, the communication network 9a is an intranet and the communication network 9b is the Internet. However, the communication networks 9a and 9b are arbitrarily selected from any communication network such as the Internet, LAN (Local Area Network), intranet, or mobile phone network according to the communication protocol used by each terminal. Further, the communication networks 9a and 9b may be the same communication network as in the case where both are the Internet.

Further, in one embodiment, the USB memory will be described as an example of the recording medium, but the present invention is not limited to this, and various recording media such as an SD card may be used as another example of the recording medium.

<Hardware configuration of electronic blackboard>
The hardware configuration of the electronic blackboard according to the embodiment will be described with reference to FIG. Note that FIG. 2 is a hardware configuration diagram of the electronic blackboard according to the embodiment.

As shown in FIG. 2, the electronic whiteboard 2 is a work of a CPU (Central Processing Unit) 201 that controls the operation of the entire electronic whiteboard 2, a ROM 202 that stores a program used to drive a CPU 201 such as an IPL, and a CPU 201. RAM 203 used as an area, SSD 204 for storing various data such as programs for electronic whiteboards, network I / F 205 for controlling communication with communication network 9, and external storage controller 206 for controlling communication with USB memory 5. It has. Further, the electronic blackboard 2 displays video information as a still image or a moving image on the display of the camera 207 that captures a subject under the control of the CPU 201, the image pickup element I / F208 that controls the drive of the camera 207, and the notebook PC 6. It includes a capture device 211, a GPU (Graphics Processing Unit) 212 that specializes in graphics, and a display controller 213 that controls and manages a screen display to output an image output from the GPU to the display 3. Further, the electronic blackboard 2 includes a sensor controller 214 that controls the processing of the contact sensor 215, and a contact sensor 215 that detects that the electronic pen 4 or the user's hand H or the like touches the display 3. In one embodiment, the contact sensor 215 inputs and detects coordinates by an infrared blocking method. In the method of inputting the coordinates and detecting the coordinates, two light receiving and emitting devices installed at both upper ends of the display 3 radiate a plurality of infrared rays in parallel with the display 3 and are provided around the display 3. This is a method of receiving light that is reflected by a reflecting member and returns on the same optical path as the light path emitted by the light receiving element. The contact sensor 215 outputs an infrared ID (Identification) emitted by two light emitting / receiving devices blocked by the object to the sensor controller 214, and the sensor controller 214 identifies a coordinate position which is a contact position of the object. All the IDs shown below are examples of identification information.

Further, the contact sensor 215 is not limited to the infrared blocking method, but is a touch panel of the capacitance type that identifies the contact position by detecting a change in capacitance, and the contact position is determined by a voltage change of two opposing resistance films. Various detection means such as a touch panel of the resistance film type to be specified and a touch panel of the electromagnetic induction type which detects the electromagnetic induction generated by the contact object coming into contact with the display unit and specifies the contact position may be used. Further, the contact sensor 215 may use a camera to determine the contact and specify the coordinate position.

Further, the electronic blackboard 2 includes an electronic pen controller 216. The electronic pen controller 216 determines whether or not there is a touch of the pen tip or a touch of the pen tip on the display 3 by communicating with the electronic pen 4. The electronic pen controller 216 may determine whether or not there is a touch not only on the pen tip and pen tip of the electronic pen 4 but also on the portion gripped by the user of the electronic pen 4 and other electronic pen portions.

Further, the electronic whiteboard 2 has a sound input / output I / F 224 that processes sound signal input / output between the external microphone 222 and the external speaker 223 under the control of the CPU 101, and an operation button 225 that accepts user operations. And a power switch 226 for switching the power ON / OFF of the electronic blackboard 2. Either the microphone 222 or the speaker 223 may be built in the electronic blackboard 2.

Further, the electronic blackboard 2 is provided with a bus line 230 such as an address bus or a data bus for electrically connecting the above-mentioned components to each other.

The program for the electronic blackboard may be recorded on a computer-readable recording medium such as a CD-ROM or a server on the network for distribution or download.

<Hardware configuration of management system>
The hardware configuration of the management system according to the embodiment will be described with reference to FIG. Note that FIG. 3 is a hardware configuration diagram of the management system according to the embodiment.

The management system 50 includes a CPU 501 that controls the operation of the entire management system 50, a ROM 502 that stores a transmission management program, a RAM 503 that is used as a work area of the CPU 501, and an HD (Hard Disk) 504 that stores various data. The HDD 505 controls reading and writing of various data to the HD 504 according to the control of the CPU 501, and the media I / F 507 controls reading and writing (storage) of data to a recording medium 506 such as a flash memory. Further, the management system 50 includes a display I / F 508 for displaying various information such as a cursor, a menu, a window, characters, and an image on the display 516, and a network I / F for transmitting data using the communication network 9. F509, a keyboard 511 equipped with a plurality of keys for inputting characters, numerical values, various instructions, etc., and a mouse 512 for selecting and executing various instructions, selecting a processing target, moving a pointing device such as a mouse cursor, and the like. And have. Further, the management system 50 is located between a CD-ROM drive 514 that controls reading and writing of data to a CD-ROM (Compact Disc Read Only Memory) 513 as an example of a removable recording medium, and an external device. It includes an external device I / F 515 for transmitting and receiving information, and a bus line 510 such as an address bus or a data bus for electrically connecting each of the above components to each other.

Further, the transmission management program may be a file in an installable format or an executable format, and may be recorded and distributed on a computer-readable recording medium such as the recording medium 506 or the CD-ROM 513. .. The transmission management program may be stored in the HD504.

<PC hardware configuration>
FIG. 4 is a hardware configuration diagram of the PC 8 as a video conferencing terminal according to the embodiment. The PC8 includes CPU801, ROM802, RAM803, flash memory 804, SSD805, media I / F807, power switch 809, bus line 810, network I / F811, camera 812, image pickup element I / F813, microphone 814, speaker 815, and sound input. It includes an output I / F 816, a display I / F 817, an external device connection I / F 818, a keyboard 821, and a mouse 822. Of these, the CPU 801 controls the operation of the entire PC 8. The ROM (Read Only Memory) 802 stores a program used for driving the CPU 801 such as an IPL (Initial Program Loader). The RAM (Random Access Memory) 803 is used as a work area of the CPU 801. The flash memory 804 stores various data such as a communication program, display data, and sound data. The SSD (Solid State Drive) 805 controls reading or writing of various data to the flash memory 804 according to the control of the CPU 801. An HDD may be used instead of the SSD. The media I / F807 controls reading or writing (storage) of data to a recording medium 806 such as a flash memory.

Further, the network I / F (Interface) 811 is an interface for performing data communication using the communication network 9b. The camera 812 is a kind of built-in imaging means that images a subject and obtains display data under the control of the CPU 801. The image sensor I / F 813 is a circuit that controls the drive of the camera 812. The microphone 814 is a kind of built-in sound collecting means for inputting voice. The sound input / output I / F 816 is a circuit that processes sound signal input / output between the microphone 814 and the speaker 815 under the control of the CPU 801. The display I / F 817 is a circuit that transmits display data to the external display 820 under the control of the CPU 801. The external device connection I / F818 is an interface for connecting various external devices. The keyboard 821 includes a plurality of keys for inputting characters, numerical values, various instructions, and the like. The mouse 822 selects and executes various instructions, selects a processing target, moves a pointing device such as a mouse cursor, and the like.

Further, the bus line 810 is an address bus, a data bus, or the like for electrically connecting each component shown in FIG. 4 such as the CPU 801.

The display 820 is a kind of display means composed of a liquid crystal or an organic EL for displaying an image of a subject, an operation icon, and the like. Further, the display 820 is connected to the display I / F 817 by the cable 820c. The cable 820c may be a cable for analog RGB (VGA) signals, a cable for component video, HDMI (registered trademark) (High-Definition Multimedia Interface), or DVI (Digital Video). It may be a cable for an Interactive) signal.

The camera 812 includes a lens and a solid-state image sensor that converts light into electric charges to digitize an image (video) of the subject. As the solid-state image sensor, CMOS (Complementary Metal Oxide Semiconductor) or CCD (Charge Coupled) Device) etc. are used. External devices such as an external camera, an external microphone, and an external speaker can be connected to the external device connection I / F818 by a USB (Universal Serial Bus) cable or the like. When an external camera is connected, the external camera is driven in preference to the built-in camera 812 according to the control of the CPU 801. Similarly, when an external microphone is connected or an external speaker is connected, the external microphone and the external speaker are given priority over the built-in microphone 814 and the built-in speaker 815 according to the control of the CPU 801. The external speaker is driven.

Further, the recording medium 806 has a structure that can be attached to and detached from the PC 8. Further, as long as it is a non-volatile memory that reads or writes data under the control of the CPU 801, not only the flash memory 804 but also an EEPROM (Electrically Erasable and Programmable ROM) or the like may be used.

<Software configuration>
FIG. 5A is a software configuration diagram of an electronic blackboard according to an embodiment. As shown in FIG. 5A, the OS (Operating System) 2020, the communication application A11, the communication application B11, and the I / F application C11 operate on the work area 2010 of the RAM 203. Hereinafter, the application will be referred to as an "application".

OS2020 is basic software that provides basic functions and manages the entire electronic blackboard. The communication applications A11 and B11 are applications for communicating with other terminals, and each has a different communication protocol. The communication application A11 is a client application that causes the electronic blackboard 2 to execute a communication control function for transmitting stroke data with another terminal and an image processing function for outputting stroke data as an image. The communication application B11 inputs / outputs video data such as conference video, a communication control function for transmitting sound data to and from other terminals, an image processing function for inputting / outputting video data, and sound data. It is a client application that causes the electronic blackboard 2 to execute a sound processing function for performing data processing. The I / F application C11 communicates with the communication applications A11 and B11 running in the same electronic blackboard 2 and controls the generation of UI images by the communication applications A11 and B11.

The OS2020 and the communication application A11 are installed on the electronic blackboards 2a, 2b, and 2c before shipment from the factory. Further, the communication application B11 and the I / F application C11 can be arbitrarily installed on the electronic blackboard 2 after being shipped from the factory. Hereinafter, it is assumed that the communication application B11 and the I / F application C11 are installed on the electronic blackboard 2c, and the communication application B11 and the I / F application C11 are not installed on the electronic blackboards 2a and 2b.

FIG. 5B is a software configuration diagram of a PC according to an embodiment. As shown in FIG. 5B, the OS 8020 and the communication application B11 operate on the work area 8010 of the RAM 803.

OS8020 is basic software that provides basic functions and manages the entire PC8. The communication application B11 causes the PC 8 to execute a communication control function for transmitting video data such as a conference video with another communication terminal and an image processing function for outputting the video data as an image.

Further, the OS 8020 and the communication application B11 are installed on the PC 8 before or after the factory shipment.

<< Functional configuration of the embodiment >>
Next, the functional configuration of the communication system according to the embodiment will be described. FIG. 6 is a functional block diagram of an electronic blackboard, a PC, and a management system 50 that form a part of the communication system according to the embodiment. In FIG. 6, the electronic blackboards 2a, 2b, and 2c are connected so that data can be communicated via the communication network 9a. Further, in FIG. 6, the electronic blackboard 2c, the PC8, and the management system 50 are connected so as to be able to perform data communication via the communication network 9b.

The electronic blackboard 2 realizes each functional configuration shown in FIG. 6 by the hardware configuration and program shown in FIG. 2. Further, the electronic blackboard 2 has a storage unit 2000 constructed by SSD 204. The PC 8 realizes each functional configuration shown in FIG. 6 by the hardware configuration and program shown in FIG. Further, the PC 8 has a storage unit 8000 constructed by the SSD 805.

The electronic blackboard 2 can be a "sponsoring device" that first starts the remote sharing process, and can also be a "participating device" that later participates in the remote sharing process that has already been started. Further, the electronic blackboard 2 is roughly divided into both a client unit 20 and a server unit 90. The client unit 20 and the server unit 90 are functions realized by activating the communication application A11 in one housing of the electronic blackboard 2. Of the client unit 20 and the server unit 90, the electronic blackboard 2 may be provided with the client unit 20, and the server unit 90 may be a device different from the electronic blackboard 2 such as a distribution control device.

When the electronic blackboard 2 serves as a host device, the client unit 20 and the server unit 90 are realized (executed) in the electronic blackboard 2. When the electronic blackboard 2 serves as a participating device, the client unit 20 is realized (executed) but the server unit 90 is not realized (executed) in the electronic blackboard 2. That is, when the electronic blackboard 2a is the host device and the electronic blackboards 2b and 2c are the participating devices, the client unit 20 of the electronic blackboard 2a uses another electronic device via the server unit 90 realized in the same electronic blackboard 2a. Communicates with the client unit 20 of the blackboards 2b and 2c. On the other hand, the client unit 20 of the electronic blackboards 2b and 2c communicates with the client unit of the other electronic blackboards 2a, 2b or 2c via the server unit 90 realized in the other electronic blackboard 2a.

[Functional Configuration of Client Unit 20] When the communication application A11 is activated on the electronic blackboard 2, the client unit 20 starts operating. The client unit 20 has a communication control unit 21 and a drawing control unit 22.

The communication control unit 21 is realized by a command from the CPU 201 or a network I / F 205, and controls communication performed with another electronic blackboard 2 and communication performed with the server unit 90 of the same electronic blackboard 2.

The drawing control unit 22 is realized by a command from the CPU 201, and is based on stroke data generated by an operation on the display 3 or data acquired from the USB memory 5, the notebook PC 6, the communication control unit 21, the storage unit 2000, and the like. Perform image processing. Further, the drawing control unit 22 controls to generate an image layer based on the processed data and output a superimposed image in which these layers are superimposed.

FIG. 7 is a functional block diagram showing a drawing control unit according to an embodiment. The drawing control unit 22 includes a stroke processing unit 22A, a superimposing unit 22Y, and a page processing unit 22Z.

The stroke processing unit 22A is realized by a command from the CPU 201, and when an event of stroke drawing occurs by the operation of the electronic pen 4 or the hand H on the display 3, the stroke processing unit 22A performs a process of generating stroke data according to the operation.

The superimposition unit 22Y is realized by a command from the CPU 201, generates each image layer based on each data, and performs a process of superimposing them.

The page processing unit 22Z collects data for each display such as stroke data into one page data and stores it in the page data storage unit 220 which is a part of the storage unit 2000.

The page data storage unit 220 is constructed as a part of the storage unit 2000 and stores page data as shown in Table 1. Table 1 is a conceptual diagram showing page data. The page data is data for one page (stroke array data (each stroke data) and media data) displayed on the display 3.

The page data includes a page data ID for identifying an arbitrary page, a start time indicating the time when the display of this page is started, and an end time indicating the time when the contents of the page are no longer rewritten by strokes or gestures. , The stroke array data ID for identifying the stroke array data generated by the stroke by the electronic pen 4 or the user's hand H, and the media data ID for identifying the media data are stored in association with each other. The stroke array data is data for displaying the stroke image on the display 3. The media data is data for displaying another image superimposed on the stroke image on the display 3.

Further, the stroke array data shows detailed information as shown in Table 2. Table 2 is a conceptual diagram showing stroke array data. As shown in Table 2, one stroke array data is represented by a plurality of stroke data. Then, one stroke data includes a stroke data ID for identifying the stroke data, a start time indicating the writing start time of one stroke, an end time indicating the writing end time of one stroke, and a stroke color. The width of the stroke and the coordinate array data ID for identifying the array of passing points of the stroke are shown. For example, when the user draws the alphabet "S" with the electronic pen 4, it is written in one stroke, so that one stroke data ID indicates the one-letter alphabet [S]. However, when the user draws the alphabet "T" with the electronic pen 4, since it is written in two strokes, the alphabet "T" of one character is indicated by two stroke data IDs.

Further, this coordinate array data shows detailed information as shown in Table 3. Table 3 is a conceptual diagram showing coordinate array data. The coordinate array data includes one point (X coordinate value, Y coordinate value) on the display 3, the time (ms) of the difference from the start time of the stroke when passing through this one point, and the electronic pen at this one point. Each information of the pen pressure of 4 is shown. That is, the set of one point shown in Table 3 is shown by one coordinate array data shown in Table 2. For example, when the user draws the alphabet "S" with the electronic pen 4, it is written in one stroke, but since the user passes through a plurality of passing points by the time the drawing of the "S" is completed, the coordinate array data has these multiple passing points. Information is shown.

Further, the media data among the page data shown in Table 1 shows detailed information as shown in Table 4. Table 4 is a conceptual diagram showing media data. As shown in FIG. 9, the media data is displayed on the display 3 according to the media data ID in the page data shown in Table 1, the data type of the media data, the recording time when the media data is stored, and the media data. The position (X coordinate value, Y coordinate value) of the displayed image, the size (width, height) of the image, and the data indicating the content of the media data are shown in association with each other. Of these, the position of the image displayed on the display 3 by the media data is displayed by the media data when the coordinates of the upper left corner of the display 3 are (X coordinate value, Y coordinate value) = (0,0). It shows the position of the upper left corner of the image to be displayed.

[Functional configuration of server unit 90]
The server unit 90 has a communication control unit 91.

The communication control unit 91 is realized by a command from the CPU 201, and is realized by a communication control unit 21 in the client unit 20 in the same electronic blackboard 2 and a communication control unit in the client unit 20 in another electronic blackboard 2 via the communication network 9a. Controls communication with 21.

[Functional configuration of client unit 80]
When the communication application B11 is activated on the electronic blackboard 2 and the PC8, the client unit 80 starts operating. The client unit 80 includes a communication control unit 81, a display control unit 82, and an input reception unit 83.

The communication control unit 81 is realized by commands from CPUs 201 and 801 or networks I / F 205 and 811 and controls communication with other terminals and the like.

The display control unit 82 is realized by commands from the CPUs 201 and 801 and performs image processing based on the data acquired from the storage units 2000 and 8000 and controls to output an image.

The input receiving unit 83 is realized by a command from the CPUs 201 and 801 and receives an operation input by the user.

[Functional configuration of management system 50]
The management system 50 realizes each functional configuration shown in FIG. 6 by the hardware configuration and program shown in FIG. Further, the management system 50 has a storage unit 5000 constructed from the HD 504.

The communication control unit 51 of the management system 50 is realized by an instruction from the CPU 501, and controls communication with the electronic blackboard 2 or the PC 8 via the communication network 9b.

[Functional configuration of electronic blackboard 2c]
The electronic blackboard 2c further includes an application I / F unit 23, a display order control unit 24, and a mode switching unit 25.

When the I / F application C11 is activated on the electronic blackboard 2c, the application I / F unit 23 starts operating. The application I / F unit 23 is realized by a command from the CPU 201, and controls the client units 20 and 80 to output a predetermined UI image.

When OS2020 is started on the electronic blackboard 2c, the display order control unit 24 starts operation. The display order control unit 24 controls the display order of each image by controlling one of the images created by the client units 20 and 80 to be active and the other to be inactive, which is realized by a command from the CPU 201. The process of actively displaying and controlling an image is a process of displaying the image with priority over other images, and is a process of displaying the image in the foreground. The process of inactively controlling the display of an image is a process of giving priority to displaying another image over the image, and is a process of displaying the image on a surface other than the foreground.

When OS2020 is started on the electronic blackboard 2c, the mode switching unit 25 starts operation. The mode switching unit 25 is realized by a command from the CPU 201, and switches the operation mode in the OS 2020. Switching the operation mode includes switching between the normal mode and the sleep mode. When the normal mode is switched to the sleep mode, the mode switching unit 25 sends a request to shift to the standby state to each running application.

<< Processing of the embodiment >>
Subsequently, the process according to one embodiment will be described.

(Communication establishment process between electronic blackboards)
A process of establishing communication by the communication application A11 between the electronic blackboards 2a, 2b, and 2c will be described with reference to FIG. FIG. 8 is a sequence diagram showing a process of establishing communication between electronic blackboards. When the user turns on the power switch 226 of the electronic blackboard 2a, the communication application A11 is activated to start the processing of the client unit 20 of the electronic blackboard 2a (step S1). Then, when the request for hosting the conference is received by the operation on the display 3a, the communication control unit 21 of the client unit 20 instructs the communication control unit 91 of the server unit 90 of the same electronic blackboard 2a to start the processing of the server unit 90. Is output. As a result, in the electronic blackboard 2a, not only the client unit 20 but also the server unit 90 starts various processes (step S2).

Next, the communication control unit 91 of the electronic blackboard 2a generates connection information for establishing a connection with the electronic blackboard 2a, and displays the generated connection information on the display 3a (step S3). This connection information includes the IP address of the host device and the passcode generated for this remote sharing process. This passcode is stored in the storage unit 2000. Then, the connection information is transmitted to the users of the electronic blackboards 2b and 2c by the user of the electronic blackboard 2a by telephone or e-mail.

Next, in the electronic blackboards 2b and 2c, when the user turns on the power switch 226 of the electronic blackboards 2b and 2c, the communication application A11 is activated and the processing of the client unit 20 of the electronic blackboards 2b and 2c is started (step). S4, S5). When the input of the connection information is received by the operation on the displays 3b and 3c on the electronic whiteboards 2b and 2c, the communication control unit 21 in the client unit 20 of the electronic whiteboards 2b and 2c communicates based on the IP address of the connection information. A passcode is transmitted to the communication control unit 91 in the server unit 90 of the electronic blackboard 2a via the network 9a to request participation (steps S6 and S7). The communication control unit 91 of the electronic blackboard 2a receives a participation request (including a passcode) from the electronic blackboards 2b and 2c.

Next, the communication control unit 91 authenticates the passcodes received from the electronic blackboards 2b and 2c using the passcodes managed by the storage unit 2000 (step S8). Then, the communication control unit 91 notifies the client unit 20 of each of the electronic blackboards 2b and 2c of the authentication result (steps S9 and S10). If it is determined by the authentication in step S8 that each of the electronic blackboards 2b and 2c is a legitimate electronic blackboard, the electronic blackboard 2a, which is the host device, and the electronic blackboards 2b, 2c, which are the participating devices, are remotely shared. Communication of processing is established (steps S11, S12). In this embodiment, a case where the content data transmitted between the electronic blackboards 2a, 2b, and 2c includes stroke data will be described.

When a user of the electronic blackboards 2a, 2b, 2c draws a stroke image using the electronic pen 4c, the stroke processing unit 22A of the electronic blackboards 2a, 2b, 2c is a contact portion between the electronic pen 4 and the displays 3a, 3b, 3c. Generates stroke data for drawn strokes based on the coordinate position of. The communication control unit 21 of the electronic blackboards 2a, 2b, 2c transmits the generated stroke data to the communication control unit 91 of the electronic blackboard 2a, which is the host device.

When the communication control unit 91 of the electronic blackboard 2a receives the stroke data sent from the electronic blackboards 2a, 2b, 2c, the communication control unit 91 transmits the stroke data to the client units 20 of the electronic blackboards 2a, 2b, 2c. The transmitted stroke data is received by the communication control unit 21 of the electronic blackboards 2a, 2b, and 2c. The page processing unit 22Z of the electronic blackboards 2a, 2b, 2c adds and stores newly received stroke data in the page data storage unit 220 without overwriting the stroke data already stored (step S13). , 14, 15). As a result, in the page data storage unit 220 of the electronic blackboards 2a, 2b, 2c, the stroke generated by any of the electronic blackboards 2a, 2b, 2c after the communication between the electronic blackboards 2a, 2b, and 2c is established. Data will be accumulated.

(Communication establishment process between electronic blackboard and PC)
A process of establishing communication by the communication application B11 between the electronic blackboard 2c and the PC 8 will be described with reference to FIG. FIG. 9 is a sequence diagram showing a process of establishing communication between the electronic blackboard and the PC.

When the communication application B11 is activated in response to the input of the activation request by the user on the electronic blackboard 2c, the client unit 80 starts various processes (step S21).

The communication control unit 81 of the client unit 80 transmits a login request to the management system 50 (step S22). This login request includes a user ID and a passcode. As a result, the communication control unit 51 of the management system 50 receives the login request.

The management system 50 requests a login based on whether the set of the user ID and the passcode included in the login request matches the set of the user ID and the passcode stored in advance in the storage unit 5000. The original electronic blackboard 2c is authenticated (step S23). The communication control unit 51 of the management system 50 transmits the authentication result to the electronic blackboard 2c of the login request source (step S24). As a result, the communication control unit 81 of the electronic blackboard 2c receives the authentication result. Hereinafter, the case where it is determined that the login request source is legitimate according to step S23 will be continued.

When the activation request of the communication application B11 is received in the PC8 based on the operation of the keyboard 821 or the mouse 822, the communication application B11 is activated and the processing of the client unit 80 of the PC8 can be started (step S31).

Subsequently, the communication control unit 81 of the PC 8 transmits a login request to the management system 50 in the same manner as the processing between the electronic blackboard 2c and the management system 50 described above (step S32). As a result, the management system 50 receives the login request.

The management system 50 authenticates the login requesting source PC8 (step S33), and transmits the authentication result to the PC8 (step S34). Hereinafter, the case where it is determined that the login request source is legitimate according to step S33 will be continued.

In response to the operation by the user of the electronic blackboard 2c, the input receiving unit 83 of the electronic blackboard 2c accepts the input of the call start request with the PC 8 (step S41). In response to the reception of the input of the call start request, the communication control unit 81 of the electronic blackboard 2c transmits the call start request to the management system 50 (step S42). The call start request includes the user ID used by the electronic blackboard 2c to log in to the management system and the user ID of the destination PC8. As a result, the management system 50 receives the communication start request.

Next, the management system 50 transmits the above-mentioned call start request to the destination PC 8 (step S43). As a result, the destination PC 8 receives the call start request.

The communication control unit 81 of the PC 8 transmits information indicating whether or not the response is possible to the management system 50 in response to the operation input by the user (step S44). Hereinafter, the case where the PC 8 transmits the information indicating the response to the management system 50 will be described.

The communication control unit 51 of the management system 50 receives information indicating that it responds. The communication control unit 51 of the management system 50 transmits information indicating the above response to the electronic blackboard 2c of the start request source (step S45). As a result, the communication control unit 81 of the electronic blackboard 2c receives the information indicating that it responds.

Subsequently, the management system 50 controls to establish communication between the electronic blackboard 2c and the PC 8 for transmitting content data via the communication network 9b (step S46). As a method of establishing communication, for example, the management system 50 authorizes the electronic blackboard 2c and the PC 8 to access the access destination of the content data. The access destination may be in the management system 50 or a device other than the management system 50. When the communication between the electronic whiteboard 2c and the PC 8 is established, the electronic whiteboard 2c and the PC 8 access the access destination, and the video data corresponding to the video captured by the cameras 207 and 812 on the own terminal side, and the video data corresponding to the video captured by the cameras 207 and 812 on the own terminal side, and Sound data corresponding to the sound collected by the microphones 222 and 814 is transmitted, and video data and sound data of the other party are acquired from the access destination (step S47). The video data and sound data are streaming data and include image data or sound data of a plurality of still images at different time points of imaging or sound collection.

When the communication control unit 81 of the electronic blackboard 2c receives the image data and sound data at a new time point, the image data and sound data stored in the storage unit 2000 are overwritten and updated by the image data and sound data at the new time point. (Steps S48 and S49). As a result, the latest image data and sound data are stored in the storage unit 2000.

(Image Generation by Communication Application A11) Next, a process of generating an image based on stroke data transmitted between the electronic blackboards 2a, 2b, and 2c will be described with reference to FIG. 10A. FIG. 10A is a flow chart showing a process of generating an image by the communication application A11. Hereinafter, the process of generating an image by the communication application A11 of the electronic blackboard 2c will be described.

The page data storage unit 220 stores image data (media data) of a background image acquired from a USB memory 5 or the like as an example. The page processing unit 22Z of the electronic blackboard 2c acquires the image data of the background image stored in the page data storage unit 220 (step S51).

The page data storage unit 220 stores image data (media data) of the object image in advance. The page processing unit 22Z of the electronic blackboard 2c acquires the image data of the object image stored in the page data storage unit 220 (step S52). The object image is an image showing a predetermined object such as a circle, a quadrangle, or a triangle.

The page processing unit 22Z of the electronic blackboard 2c acquires the stroke data (see step S15) stored in the page data storage unit 220 (step S53).

The page processing unit 22Z of the electronic blackboard 2c generates image data of a stroke image based on the stroke data acquired in step S53 (step S54). For example, when the stroke data indicating the character "S" is acquired in step S53, the page processing unit 22Z generates the image data of the image of "S".

Image data of a UI image (user interface) is stored in the page data storage unit 220 of the electronic blackboard 2c. The image data of the UI image is, for example, an icon or a button. The page processing unit 22Z of the electronic blackboard 2c acquires the image data of the UI image stored in the page data storage unit 220 (step S55). The image data of the UI image includes an icon indicating the communication status by the client unit 80, an icon I1 for accepting an instruction to end the conference, an icon I2 for accepting an instruction to move to the screen of the video conference, and an electronic blackboard. The image data of the icon I3 for accepting the instruction to move to the screen is included. The icon I2 may be a UI image for displaying the screens of the video conference side by side on the screen of the electronic blackboard when the screen of the electronic blackboard is displayed. The icon I3 may be a UI image for displaying the screens of the electronic blackboard side by side on the screen of the video conference when the screen of the video conference is displayed. Among the image data of the UI image stored in the page data storage unit 220, there are a plurality of icons indicating the communication status. Of these, the image data acquired by the page processing unit 22Z is controlled by the application I / F unit 23. This process will be described later. Further, the page data storage unit 220 manages the display positions of the icons I1, I2, and I3 so as to be changeable according to a drag operation by the user. In this way, when the area where the stroke is written or the image area to be confirmed overlaps with these icons I1, I2, I3, the display position of the icons I1, I2, I3 can be moved to improve operability and visibility. Can be secured.

The superimposing portion 22Y of the electronic blackboard 2c is a superimposing image in which a background image, an object image, a stroke image, and a UI image are superposed in order using each image data acquired or generated in steps S51, S52, S54, and S55. Image data is generated (step S56).

The superimposition unit 22Y outputs the image data of the superimposition image generated in step S56 to the display order control unit 24 (step S57). FIG. 11A is a conceptual diagram showing each of the superimposed images. FIG. 11B is a conceptual diagram showing a superimposed image. As shown in FIGS. 11A and 11B, the superimposed image 140 output in step S57 includes a UI image 140a, a stroke image 140b, an object image 140c, and a background image 140d.

While the communication application A11 is running, the client unit 20 of the electronic blackboard 2c repeats the processes of steps S51, S52, S53, S54, S55, S56, and S57. As a result, the superimposed image reflecting the latest background image, object image, stroke image, and UI image is repeatedly output to the display order control unit 24.

If the image data of the background image or the stroke data stored in the page data storage unit 220 is not updated in the repeated processing, the client unit 20 skips the processing in step S51 or steps S53 and S54. You may. Further, in the repeated processing, the client unit 20 may skip the processing of steps S52 and S55 if there is no change in the image data of the object image to be acquired or the image data of the UI image. In this case, the previously acquired or generated image data is used to generate the superimposed image.

(Image generation by communication application B11)
Subsequently, a process of generating an image based on the video data transmitted between the electronic blackboard 2c and the PC 8 will be described with reference to FIG. 10B. FIG. 10B is a flow chart showing a process of generating an image by the communication application B11. Hereinafter, the process of generating an image by the communication application B11 of the electronic blackboard 2c will be described.

The display control unit 82 of the electronic blackboard 2c acquires the image data (see step S48) of the latest image stored in the storage unit 2000 (step S61). The image data of the latest image is the image data of the latest image for constructing the video data received in step S47, and is overwritten and saved in step S48.

The image data of the UI image is stored in the storage unit 2000 of the electronic blackboard 2c. UI images are, for example, icons and buttons. Further, the UI image may be a message corresponding to an event such as "accepting a request for adding a destination list". The display control unit 82 of the electronic blackboard 2c acquires the image data of the UI image stored in the page data storage unit 220 (step S62). The image data of the UI image includes an icon indicating the communication status by the client unit 20, an icon I1 for ending the conference, an icon I2 for shifting to the screen of the video conference, and an icon I2 for shifting to the screen of the electronic blackboard. The image data of the icon I3 is included. Among the image data of the UI image stored in the page data storage unit 220, there are a plurality of icons indicating the communication status. Of these, the image data acquired by the display control unit 82 is controlled by the application I / F unit 23. This process will be described later.

The display control unit 82 of the electronic blackboard 2c uses the image data acquired in steps S61 and S62 to generate image data of the latest image and the superimposed image in which the UI image is superimposed in order (step S63).

The display control unit 82 of the electronic blackboard 2c outputs the image data of the superimposed image generated in step S63 to the display order control unit 24 (step S64). FIG. 12A is a conceptual diagram showing each of the superimposed images. FIG. 12B is a conceptual diagram showing a superimposed image. As shown in FIGS. 12A and 12B, the superimposed image 150 output in step S64 includes the UI image 150a and the latest image 150b for constructing the image.

While the communication application B11 is running, the client unit 80 of the electronic blackboard 2c repeats the processes of steps S61, S62, S63, and S64. As a result, the latest image for constructing the image and the superimposed image reflecting the UI image are repeatedly output to the display order control unit 24.

The client unit 80 may skip the process of step S62 if there is no change in the image data of the UI image to be acquired in the repeated process. In this case, the previously acquired image data is used to generate the superimposed image.

(Display order control)
Subsequently, with reference to FIG. 13, a process in which the display order control unit 24 of the electronic blackboard 2c controls the display order of the images output by the client unit 20 and the images output by the client unit 80 will be described. FIG. 13 is a sequence diagram showing a process of controlling the display order of images. Note that FIG. 13 shows a sequence of each part in the electronic blackboard 2c.

At the timing when the second application of the communication applications A11 and B11 is started, the display order control unit 24 actively displays the image data (see step S57) output from the client unit 20 as the default process. The image signal is output to the display 3c so that the image data (see step S64) output from the client unit 80 is displayed inactive (step S71). The display order is controlled, for example, by superimposing a display area such as a window corresponding to the communication application A11 on the display area such as a window corresponding to the communication application B11 by the OS 2020. When the image output by the client unit 20 is a full-screen image, the display of the display 3c is occupied by the image output by the client unit 20.

The page data storage unit 220 stores image data of an icon having a predetermined brightness and image data of an icon having a half brightness as a UI image indicating a state of the communication application B11. The icon having a predetermined brightness indicates a state in which a predetermined event has occurred in the communication application B11. The semi-bright icon indicates a state in which a predetermined event has not occurred in the communication application B11. At the timing when the second application of the communication applications A11 and B11 is started, as a default process, the application I / F unit 23 is a UI indicating a half-bright icon among the UI images indicating the state of the communication application B11. The image information is transmitted to the client unit 20 (step S72).

When the communication control unit 21 of the client unit 20 receives the UI image information indicating the image data of the half-brightness icon, the page processing unit 22Z acquires the image data of the half-brightness icon in step S55. As a result, the superimposed image output to the display order control unit 24 in step S57 includes the half-luminance icon 141 as shown in FIG. 11B. As a result, even when the image output by the client unit 20 is actively controlled, the user visually recognizes the half-brightness icon 141 in a state where a predetermined event has not occurred in the communication application B11. I can understand that there is.

The application I / F unit 23 detects a predetermined event that triggers a state change of the communication application B11. Examples of the predetermined event include an event that triggers the communication application B11 to change to a state of accepting some input from the user. Examples of such an event include reception of a request on the PC8 side, such as reception of a call start request by the PC8 and reception of a request for adding a user of the electronic blackboard 2c to the destination list on the PC8 side. The case where the event is the reception of an additional request to the destination list will be described below.

The communication control unit 81 of the electronic blackboard 2c receives an additional request to the destination list transmitted by the PC8 side as an event (step S81).

The application I / F unit 23 of the electronic blackboard 2c detects the reception of an additional request to the destination list as a predetermined event (step S82). There is no particular limitation on the method in which the application I / F unit 23 detects a predetermined event. For example, the application I / F unit 23 may transmit information for confirming whether or not a predetermined event has been received to the communication control unit 81 at predetermined intervals such as every 30 seconds. In this case, the application I / F unit 23 can detect the event based on the response from the communication control unit 81 to this information. Further, when the communication control unit 81 receives a predetermined event, the communication control unit 81 may transmit information indicating the occurrence of the event to the application I / F unit 23. In this case, the application I / F unit 23 detects the event based on the received information.

The application I / F unit 23 of the electronic blackboard 2c manages the corresponding UI image for each predetermined event. Hereinafter, a case where the application I / F unit 23 manages information indicating an icon having a predetermined brightness as a UI image corresponding to the reception of an addition request to the destination list will be described. The application I / F unit 23 transmits change request information indicating a predetermined brightness icon corresponding to this event to the client unit 20 in response to the detection of the reception of the addition request to the destination list (step S83). The change request information is an instruction to change the UI image to the image shown in the change request information. In this way, when the application I / F unit 23 detects the event of one communication application A11, B11, the application I / F unit 23 transmits the change request information to the other communication application B11, A11.

The communication control unit 21 of the electronic blackboard 2c receives the change request information transmitted by the application I / F unit 23. Based on the received change request information, the page processing unit 22Z of the electronic blackboard 2c sets the image data of the icon having a predetermined brightness among the image data of the UI image stored in the page data storage unit 220 in step S55. get. As a result, in step S57, an image of the icon having a predetermined brightness is output as a UI image. FIG. 11C is a conceptual diagram showing a superimposed image changed based on the reception of change request information. As shown in FIG. 11C, the superimposed image 140 output in step S57 after receiving the change request information includes an icon 142 having a predetermined brightness. As a result, even when the image output by the client unit 20 is actively controlled, the user visually recognizes the icon 142 having a predetermined brightness, so that a predetermined event is generated in the communication application B11. You can see that you have transitioned to.

When the user of the electronic blackboard 2c touches the position where the icon 142 is displayed on the display 3c, the drawing control unit 22 receives an input of a request for changing the display order on the display 3c (step S84).

In response to the acceptance of the input of the request for changing the display order, the communication control unit 21 of the electronic blackboard 2c transmits the display order change request information for changing the display order on the display 3c to the display order control unit 24. (Step S85).

The display order control unit 24 of the electronic blackboard 2c receives the display order change request information. In response to the reception of the display order change request information, the display order control unit 24 deactivates the image output from the client unit 20 which is the source of the display order change request information, and outputs the image from the other client unit 80. The image is actively controlled (step S86). As a result, the image by the communication application B11 as shown in FIG. 12B is output from the display 3c. The display order control unit 24 may control the image output from the client unit 20 to be displayed in a small size and the image output from the client unit 80 to be displayed in a large size at the same time.

When the user of the electronic blackboard 2c touches the position where the button 153 of the display 3c is displayed, the input receiving unit 83 accepts the input of the permission response to the addition request to the destination list on the PC8 side (step S87).

In response to the acceptance of the input of the permission response, the communication control unit 81 of the electronic blackboard 2c transmits the permission response to the PC8 that is the source of the request for addition to the destination list (step S88).

(End of Meeting) Next, the process for ending the meeting between the electronic blackboards 2a, 2b, 2c, and PC8 will be described. FIG. 17 is a sequence diagram showing processing of each functional block of the electronic blackboard 2c at the end of the conference.

First, using (A) of FIG. 17, a process of ending the conference when the image output by the client unit 20 is active will be described. When the user of the electronic blackboard 2c touches the icon I1 of the superimposed image 140 with the electronic pen 4c or the like, the client unit 20 receives the request for the end of the conference (step S101).

When the request for the end of the meeting is received, the communication control unit 21 of the client unit 20 transmits a request for switching to the sleep mode to the mode switching unit 25 of the same electronic blackboard 2c (step S102).

When the mode switching unit 25 receives the request to switch to the sleep mode, the mode switching unit 25 transmits a request to shift to the standby state to each application running on the OS 2020 of the electronic blackboard 2c. As a result, the mode switching unit 25 transmits a request to shift to the standby state to the client unit 20 and the client unit 80 (steps S103 and S104).

When the client unit 80 receives the request to shift to the standby state, the client unit 80 stops the operation of the communication control unit 81 and the display control unit 82, which are preset as functions for not operating in the standby state. As a result, the communication control unit 81 disconnects the communication with the PC 8 (step S105).

When the client unit 20 receives the request to shift to the standby state, the client unit 20 stops the operation of the communication control unit 21 and the drawing control unit 22, which are preset as functions for not operating in the standby state. As a result, the communication control unit 21 disconnects the communication with the electronic blackboard 2a, which is the host device (step S106).

Subsequently, the process of ending the conference when the image output by the client unit 80 is active will be described with reference to FIG. 17B. When the user of the electronic blackboard 2c touches the icon I1 of the superimposed image 150 with the electronic pen 4c or the like, the client unit 80 receives the request for the end of the conference (step S111).

When the request for the end of the meeting is received, the client unit 80 determines whether the stroke data is stored in the page data storage unit 220 of the storage unit 2000 (step S112). In this process, the client unit 80 determines YES if the stroke data can be acquired from the page data storage unit 220 of the storage unit 2000, and determines NO if the stroke data cannot be acquired.

When it is determined in step S112 that the stroke data is stored (YES), the communication control unit 81 of the electronic blackboard 2c activates the output image by the client unit 20 to the display order control unit 24 of the same electronic blackboard 2c. Send display order change request information to change the display order to.

The display order control unit 24 of the electronic blackboard 2c receives the display order change request information. In response to the reception of the display order change request information, the display order control unit 24 deactivates the image output from the client unit 80, which is the source of the display order change request information, and outputs the image from the other client unit 20. The image is actively controlled (step S113). As a result, the user can confirm the stroke image, save the image as necessary, and then end the conference. The process of ending the conference after switching the screen is the same as in steps S101 to S105.

If it is determined in step S112 that the stroke data is not stored (NO), the communication control unit 81 of the client unit 80 transmits a request to switch to the sleep mode to the mode switching unit 25 of the same electronic blackboard 2c (step S122). ).

The process of receiving the request for switching to the sleep mode by the mode switching unit 25 is the same as in steps S103 to S106, and thus the description thereof will be omitted (steps S123 to S126).

<<< Second Embodiment >>>
Subsequently, with reference to FIG. 14, the second embodiment will be described with respect to the differences from the first embodiment. FIG. 14 is a sequence diagram showing a process of controlling the display order of images. Note that FIG. 14 shows a sequence of each part in the electronic blackboard 2c.

The storage unit 2000 stores image data of an icon having a predetermined brightness and image data of a half-brightness icon as a UI image indicating a state of the communication application A11. The icon having a predetermined brightness indicates a state in which the communication application A11 has received the stroke data. The half-bright icon indicates a state in which the communication application A11 has not received stroke data.

At the timing when the display order is changed in step S86, the application I / F unit 23 transmits the UI image information indicating the half-brightness icon among the UI images indicating the state of the communication application A11 to the client unit 80 (step S90). ..

When the communication control unit 81 of the client unit 80 receives the UI image information indicating the image data of the half-brightness icon, the display control unit 82 acquires the image data of the half-brightness icon from the storage unit 2000 in step S62. As a result, the superimposed image output to the display order control unit 24 in step S64 includes the half-luminance icon 151 as shown in FIG. 12 (B). As a result, even when the image output by the client unit 80 is actively controlled, the user visually recognizes the half-brightness icon 151 to indicate that the communication application A11 is not receiving stroke data. I can grasp it.

The application I / F unit 23 detects a predetermined event that triggers a state change of the communication application A11. As a predetermined event, reception of stroke data by the communication application A11 can be mentioned.

The communication control unit 21 of the electronic blackboard 2c receives the stroke data transmitted by the communication control unit 91 of the electronic blackboard 2a as an event (step S91).

The application I / F unit 23 of the electronic blackboard 2c detects the reception of stroke data by the communication control unit 21 of the electronic blackboard 2c as a predetermined event (step S92).

The application I / F unit 23 of the electronic blackboard 2c manages information indicating a corresponding UI image for each predetermined event. Hereinafter, a case where the application I / F unit 23 manages information indicating an icon having a predetermined brightness as a UI image corresponding to an event of receiving stroke data will be described. The application I / F unit 23 transmits change request information indicating a predetermined brightness icon corresponding to this event to the client unit 80 in response to the detection of the event of receiving the stroke data (step S93).

The communication control unit 81 of the electronic blackboard 2c receives the change request information transmitted by the application I / F unit 23. In step S62, the display control unit 82 of the electronic blackboard 2c acquires the image data of the icon having a predetermined brightness among the image data of the UI image stored in the storage unit 2000 based on the received change request information. do. As a result, in step S64, an image of the icon having a predetermined brightness is output as a UI image. FIG. 12C is a conceptual diagram showing a superimposed image changed based on the reception of change request information. As shown in FIG. 12C, the superimposed image 150 output in step S64 includes an icon 152 having a predetermined brightness. As a result, even when the image output by the client unit 80 is actively controlled, the user can grasp that the communication application A11 has received the stroke data by visually recognizing the icon 152 having a predetermined brightness.

When the user of the electronic blackboard 2c touches the position where the icon 152 of the display 3c is displayed, the input receiving unit 83 receives the input of the request for changing the display order on the display 3c (step S94).

In response to the acceptance of the input of the request for changing the display order, the communication control unit 81 of the electronic blackboard 2c transmits the display order change request information for changing the display order on the display 3c to the display order control unit 24. (Step S95).

The display order control unit 24 of the electronic blackboard 2c receives the display order change request information. In response to the reception of the display order change request information, the display order control unit 24 deactivates the image output from the client unit 80, which is the source of the display order change request information, and outputs the image from the other client unit 20. The image is actively controlled (step S96). As a result, when the image output from the client unit 20 becomes active, the user of the electronic blackboard 2c can confirm the newly added stroke. The display order control unit 24 may control the image output from the client unit 20 to be displayed in a large size and the image output from the client unit 80 to be displayed in a small size at the same time.

<<< Third Embodiment >>>
Subsequently, the third embodiment will be described with respect to the differences from the first and second embodiments.

In the first embodiment, the case where the communication applications activated on the electronic blackboard 2c are the communication applications A11 and B11 has been described. In the third embodiment, a case where there are three or more communication applications activated on the electronic blackboard 2c will be described.

In the third embodiment, the application I / F unit 23 transmits the identification information of the application in which the event has occurred when the change request information is transmitted in steps S83 and S93. Subsequently, the communication control units 21 and 81 transmit the identification information of the application received in steps S83 and S93 when transmitting the display order change request information to the display order control unit 24. The display order control unit 24 actively controls the image output by the application identified by the identification information of the received application, and inactively controls the image output by another application. By the above processing, even if there are three or more communication applications, the display order control unit 24 can determine which communication application is used to activate the image.

<<< Fourth Embodiment >>>
Subsequently, the fourth embodiment will be described with respect to the differences from the first to third embodiments. In each of the above embodiments, the case where the event is a request from a communication partner or reception of stroke data has been described. However, the event may be various information that triggers the transition of the presence of the communication applications A11 and B11.

The application I / F unit 23 may detect, for example, the reception of the authentication result in steps S10 and 24 as an event (see steps S82 and S92). Since the communication applications A11 and B11 become an online presence triggered by the above event, the application I / F unit 23 changes the UI image to an icon corresponding to the online presence in response to the detection of the above event. The change request information for this purpose is transmitted to the client units 20 and 80.

The event detected by the application I / F unit 23 in steps S82 and S92 is not limited to the event received by the client units 20 and 80. For example, the application I / F unit 23 may detect the transmission of the response by the client unit 80 in step S88. The presence of the communication application B11 transitions during a call triggered by the above event. In response to the detection of the above event, the application I / F unit 23 transmits the change request information for changing the UI image to the icon corresponding to the presence during the call to the client unit 20.

<<< Fifth Embodiment >>>
Subsequently, the fifth embodiment will be described with respect to the differences from the first to fourth embodiments. In the fifth embodiment, the display management DB 2001 is constructed in the storage unit 2000 of the electronic blackboard 2c by the display management table. Table 5 is a conceptual diagram showing a display management table.

In the display management table, event information indicating a predetermined event detected by the application I / F unit 23, format information indicating in what output format the notification corresponding to the event is displayed, and display contents are displayed. It is associated and managed. The event information and the format information may be IDs indicating the event or format, respectively. The display content may be, for example, text indicating the display content, a badge, or information such as a file name for identifying a dialog.

When the application I / F unit 23 of the electronic blackboard 2c detects an event in the client unit 80, in step S83, the application I / F unit 23 transmits change request information including event information indicating the detected event to the client unit 20.

The communication control unit 21 of the client unit 20 of the electronic blackboard 2c receives the change request information transmitted by the application I / F unit 23. The drawing control unit 22 of the electronic blackboard 2c searches the display management table using the event information included in the received change request information as a search key, and acquires the corresponding format information and the display contents. Then, the drawing control unit 22 generates an image in which the acquired display content is displayed in the format of the acquired format information. The superimposing portion 22Y of the electronic blackboard 2c superimposes the generated image on another layer in step S56. Then, in step S57, the superimposition unit 22Y outputs the superimposition image including the display contents to the display order control unit 24. The display order control unit 24 outputs image data including the display contents to the display 3c.

FIG. 15A is a display example of the display when the event of “communication disconnection” is detected in step S83. In the display management table, the format information "ticker" and the display content "disconnected to the network" are associated with "communication disconnection". Therefore, the drawing control unit 22 generates an image of the ticker T including the above display contents. The drawing control unit 22 may generate an image of the ticker T for a predetermined time such as 10 seconds. As a result, it is possible to prevent the visibility of the display from being lowered due to the ticker T being displayed for a long time.

FIG. 15B is a display example of the display when the event of “screen sharing” is detected in step S83. In the display management table, the format information "dialog" and the display content "dialog D1" are associated with "screen sharing". The drawing control unit 22 generates an image of the dialog D1 specified by these pieces of information. With this dialog, the electronic blackboard 2c can effectively prompt the user to switch screens. When communication cannot be achieved without switching to the other party's shared screen, as in this "screen sharing" event, it is possible to increase the degree to which switching is prompted by a dialog instead of the ticker.

FIG. 15C is a display example of the display when the “in-meeting” event (presence) is detected in step S83. In the display management table, the format information "badge" and the display content "badge BD1" are associated with "during a meeting". The drawing control unit 22 generates an image of the badge BD1 specified by this information. The electronic blackboard 2c can make the user know that the video conference is in progress by the badge BD1. In the electronic blackboard 2c, the position and area of the badge BD1 are preset so as not to interfere with the display of other interfaces. As a result, the electronic blackboard 2c can continuously display the badge BD1 during the meeting without interfering with other operations by the user.

In the above, an example in which the event generated in the client unit 80 is detected by the application I / F23 and the information corresponding to the event is output in the client unit 20 has been described. The event generated in the client unit 20 is detected by the application I / F23, and the process of outputting the information corresponding to the event in the client unit 80 is also executed in the same manner as the above process. However, the destination of the information indicating the event by the application I / F unit 23 is the client unit 80, and the processing by the drawing control unit 22 is replaced with the processing by the display control unit 82.

<<< Sixth Embodiment >>>
Subsequently, the sixth embodiment will be described with respect to the differences from the first to fifth embodiments.

The page data storage unit 220 of the electronic blackboard 2c stores the function setting screen and the video conference setting screen. The page processing unit 22Z of the electronic blackboard 2c acquires the function setting screen stored in the page data storage unit 220 at a predetermined timing such as at the time of activation or at the time of a request by the user. In step S57, the superimposition unit 22Y outputs the superimposition image including the acquired function setting screen to the display order control unit 24. The display order control unit 24 outputs the image data including the function setting screen to the display 3c. FIG. 16A is a diagram showing an example of a function setting screen. The function setting screen includes a button B1 for shifting to the video conference setting screen.

When the user touches the button B1 output to the display 3c with the electronic pen 4c or the like, the page processing unit 22Z of the electronic blackboard 2c acquires the video conference setting screen stored in the page data storage unit 220. do. In step S57, the superimposition unit 22Y outputs the superimposition image including the acquired video conference setting screen to the display order control unit 24. The display order control unit 24 outputs the image data including the video conference setting screen to the display 3c. FIG. 16B is a diagram showing an example of a video conference setting screen.

When the user checks the check box to use the video conferencing function in the input field B2 of the video conferencing setting screen with the electronic pen 4c or the like and inputs the user ID and the passcode, the client unit 20 Stores the input user ID and passcode in the storage unit 2000. When the electronic blackboard 2 is activated after this setting, the client unit 80 automatically requests login to the management system 50 based on the user ID and passcode stored in the storage unit 2000 without accepting the operation by the user.

Furthermore. When the user performs an operation of selecting a video conference as the priority start screen in the input field B3 of the video conference setting screen, the client unit 20 stores information indicating the video conference screen as the default priority start screen. Store in part 2000. As a result, when the electronic blackboard 2 is activated, the display order control unit 24 outputs an image output from the client unit 20 in step S71 based on the information indicating the video conference screen stored in the storage unit 2000. The data is deactivated, the image data output from the client unit 80 is activated, and the image data is output to the display 3c.

The video conference setting screen of FIG. 16B includes a button B4 for setting a region and a language. When the user touches the button B4 by operating the electronic pen 4c or the like, the electronic blackboard 2c has the client unit 20. Accepts the setting of the area where the electronic blackboard 2c is used. After the setting, the page processing unit 22Z of the electronic blackboard 2c may generate an image according to the set area. For example, when Japan is set as the region, the page processing unit 22Z may output the conference setting screen without including the video conference setting button B1. Alternatively, when the United States is set as the region, the page processing unit 22Z may output a pre-managed English conference setting screen in response to a request from the conference setting screen.

By connecting the PC to the communication network 9a or 9b of FIG. 1 and entering the desired IP address of the electronic blackboard 2 into the Web browser to access the PC, the video conference setting screen of the electronic blackboard 2 is displayed on the PC. The above items and values may be settable.

<<< Seventh Embodiment >>>
Subsequently, the seventh embodiment will be described with respect to the differences from the first to sixth embodiments.

In the page data storage unit 220 of the electronic whiteboard 2c, the icon I4 for starting the video conference application B11 and the output screen by the video conference application B11 are maximized as the image data of the UI image (user interface). It includes an icon I5 for displaying in the foreground and an icon I6 for minimizing the output screen by the video conference application B11.

When the video conferencing application B11 is not activated, the page processing unit 22Z of the electronic blackboard 2c acquires the image data of the icon I4 in step S55. When the video conferencing application B11 is running but the output by the application B11 is inactive, the page processing unit 22Z acquires the image data of the icon I5 in step S55. When the video conferencing application B11 is running and the output by the application B11 is active, the page processing unit 22Z acquires the image data of the icon I6 in step S55. (A) to (C) of FIG. 18 are display examples of the display 3c when the image data of the icons I4, I5, or I6 is acquired in step S55.

FIG. 19A is a flow chart showing the processing of the electronic blackboard 2c when the operation of selecting the icon I4 is executed. By this operation, the client unit 20 of the electronic blackboard 2c receives a request for activating the video conferencing application B11 (hereinafter referred to as an activation request) (step S131).

The communication control unit 21 of the electronic blackboard 2c transmits the received activation request to the application I / F unit 23 of the same electronic blackboard 2c (step S132). The application I / F unit 23 transmits the received activation request to the OS 2020 of the same electronic blackboard 2c (step S133).

The OS 2020 starts the application B11 in response to the received start request (step S134). The display order control unit 24 of the OS 2020 controls the image output by the application B11 to be actively displayed and the image output by the application A11 to be inactively displayed (step S135).

FIG. 19B is a flow chart showing the processing of the electronic blackboard 2c when the operation of selecting the icon I5 is executed. By this operation, the client unit 20 of the electronic blackboard 2c receives a request for maximizing the image output by the video conferencing application B11 and displaying it in the foreground (hereinafter referred to as a maximization request) (step S141). ).

The communication control unit 21 of the electronic blackboard 2c transmits the accepted maximization request to the application I / F unit 23 of the same electronic blackboard 2c (step S142). The application I / F unit 23 transmits the received maximization request to the display order control unit 24 of the OS 2020 of the same electronic blackboard 2c (step S143).

The display order control unit 24 of the OS 2020 deactivates the image output by the application A11 in response to the received maximization request, maximizes the image output by the application B11, and then actively displays the image. (Step S144).

FIG. 19C is a flow chart showing the processing of the electronic blackboard 2c when the operation of selecting the icon I6 is executed. By this operation, the client unit 20 of the electronic blackboard 2c receives a request for minimizing the image output by the video conferencing application B11 (hereinafter referred to as a minimization request) (step S151).

The communication control unit 21 of the electronic blackboard 2c transmits the accepted minimization request to the application I / F unit 23 of the same electronic blackboard 2c (step S152). The application I / F unit 23 transmits the received minimization request to the display order control unit 24 of the OS 2020 of the same electronic blackboard 2c (step S153).

The display order control unit 24 of the OS 2020 minimizes and deactivates the image output by the application B11 in response to the received minimization request so that the image output by the application A11 is actively displayed. (Step S154)

In the seventh embodiment, the case where the image output by the application A11 is larger than the image output by the application B11 and any of the icons I4, I5, and I6 is always displayed has been described. For example, if the image output by the application A11 is smaller than the image output by the application B11, the icons I4, I5, and I6 may be displayed on the application B11. In this case, the processing by the client unit 20 and the processing for the client unit 20 are executed in the same manner as the processing in steps S131 to S154, except that the processing by the client unit 80 and the processing for the client unit 80 are replaced with the processing for the client unit 80.

<< Main effects of the embodiment >>
According to the output method of the above embodiment, the communication control unit 21 (an example of transmission / reception means) of the electronic blackboard 2c is stroke data transmitted / received in the first communication with the electronic blackboard 2a (an example of another communication terminal). A stroke image (an example of a first image) relating to (an example of content data) is transmitted / received (an example of transmission / reception processing). The communication control unit 81 (an example of a transmission / reception unit) of the electronic blackboard 2c is a video (an example of content data) related to video data (an example of content data) transmitted / received in a second communication with a PC 8 (an example of another communication terminal). Send and receive (an example of the image of 2) (an example of transmission / reception processing). The drawing control unit 22 and the display control unit 82 of the electronic whiteboard 2c, and the display 3c (an example of a display means) are icons I2 and I3 (an example of a display instruction image) that receive an instruction to display at least one of a stroke image and an image. ), At least one image of the trolley image and the video corresponding to the instruction, and the icon I1 (an example of the stop instruction image) for receiving the instruction to stop the first communication and the second communication are displayed (an example of the display process). .. As a result, the user can stop both communications by operating via the icon I1, so that the usability of the electronic blackboard 2 is improved.

When the operation input via the icon I1 of the superimposed image 140 or the icon I1 of the superimposed image 150 is received, the mode switching unit 25 (an example of the stopping means) of the electronic blackboard 2c shifts the OS 2020 to the sleep mode. Then, both the first communication and the second communication are stopped (an example of the stop process). As a result, the user can stop both communications by operating via the icon I1.

On the electronic blackboard 2c, the application A11 (an example of the first application) for establishing the first communication and the application B11 (the second application) for establishing the second communication are started. The mode switching unit 25 (an example of the stopping means) of the electronic blackboard 2c shifts the OS 2020 to the sleep mode, so that the first communication and the second communication can be performed without stopping the activation of the application A11 and the application B11. Stop both. Thereby, when the conference is desired to be restarted, the electronic blackboard 2c can promptly reestablish the first communication and the second communication.

The content data transmitted / received in the first communication is stroke data, and the content data transmitted / received in the second communication is video data. The page data storage unit 220 constructed in the storage unit 2000 (an example of management means) of the electronic blackboard 2c manages stroke data related to strokes output by the display control unit 82. When the stroke data is managed in the storage unit 2000 when the operation of the instruction to stop the communication via the icon I1 of the superimposed image 150 is received, the display order control unit 24 actively displays the superimposed image 140. Let me. As a result, the electronic blackboard 2c allows the user to check the stroke image before the communication is stopped.

The application I / F unit 23 of the electronic blackboard 2c transmits information indicating an event generated by communication by the communication application B11 to the communication application A11. By the processing of the communication application A11, the display 3c outputs a notification (an example of information) corresponding to the information indicating this event. As a result, the electronic blackboard 2c can make the user decide to stop the communication by this notification.

The display management table constructed in the storage unit 2000 (an example of the management means) of the electronic blackboard 2c manages the information indicating the output format in association with the information indicating the event. When the communication application A11 receives the information indicating the event, the communication application A11 outputs a notification in an output format indicated by the information managed in the display management table associated with the information indicating the event. As a result, the electronic blackboard 2c can output the notification in the optimum format according to the detected event.

<< Supplement to the embodiment >>
The communication applications A11, B11, and C11 may be recorded and distributed on a computer-readable recording medium by a file in an installable format or an executable format. Further, as another example of the above-mentioned recording medium, a CD-R (Compact Disc Recordable), a DVD (Digital Versatile Disk), a Blu-ray disc, and the like can be mentioned. Further, the above-mentioned recording medium or HD (Hard Disk) in which these programs are stored can be provided as a program product in Japan or abroad.

Further, the electronic blackboard 2, the PC 8, the relay device 30, and the management system 50 in the above embodiment may be constructed by a single device, or each part (function or means) may be divided and arbitrarily assigned. It may be constructed by the device of.

Each function of the embodiment described above can be realized by one or more processing circuits. Here, the "processing circuit" in the present specification is a processor programmed to execute each function by software such as a processor including an electronic circuit, or an ASIC designed to execute each function described above. It shall include devices such as (Application Specific Integrated Circuit) and conventional circuit modules.

1 Communication system 2 Electronic blackboard 3 Display 4 Electronic pen 5 USB memory 6 Notebook PC
8 PC
9a, 9b Communication network 20 Client unit 21 Communication control unit 22 Drawing control unit 22A Stroke processing unit 22Y Superimposition unit 22Z Page processing unit 23 Application I / F unit 24 Display order control unit 25 Mode switching unit 220 Page data storage unit 30 Relay device 50 Management system 80 Client unit 81 Communication control unit 82 Display control unit 83 Input reception unit 90 Server unit 91 Communication control unit 2000 Storage unit 8000 Storage unit

Japanese Unexamined Patent Publication No. 2015-89943

Claims (10)

The first image relating to the stroke data transmitted or received in the first communication with the other communication terminal, and the video data transmitted or received in the second communication with the other communication terminal. A transmission / reception means for transmitting or receiving a second image,
A display instruction image that accepts an instruction to display at least one of the first image and the second image, and at least one of the first image and the second image in response to the instruction. In addition, a display means for displaying a stop instruction image for receiving an instruction to stop the first communication and the second communication, and
When the stop instruction is accepted, the stop means for stopping both the first communication and the second communication, and the stop means.
It has a management means for managing the output of the stroke data, and has
When the management means determines that the stroke data is stored when the stop instruction is received, the display means gives priority to the first image over the second image. Communication terminal to display. The stop means does not stop the activation of the first application for establishing the first communication and the second application for establishing the second communication, and the first communication and the stop means. The communication terminal according to claim 1 , wherein both of the second communications are stopped. When the display means receives information indicating an event that occurs in the second communication from the other communication terminal while displaying the first image, the event that occurs in the second communication. The communication terminal according to claim 1 or 2 , which displays information indicating the above. The management means manages an output format of information corresponding to the information indicating the event in association with the information indicating the event.
The communication terminal according to claim 3 , wherein the display means displays the corresponding information in an output format managed by the management means in association with the received information indicating the event. When the display means is displaying the second image and the stroke data is received from the other communication terminal, the display means displays an icon having a predetermined brightness.
When the second image is displayed and the stroke data is not received from the other communication terminal, a half-brightness icon is displayed.
The communication terminal according to any one of claims 1 to 4. The communication terminal according to any one of claims 1 to 5.
With the other communication terminal that communicates with the communication terminal by the first communication,
Communication system including. The communication terminal according to any one of claims 1 to 5.
With the other communication terminal that communicates with the communication terminal by the second communication,
Communication system including. The communication system according to claim 7 , further comprising a management system for managing the second communication. For communication terminals
The first image relating to the stroke data transmitted or received in the first communication with the other communication terminal, and the video data transmitted or received in the second communication with the other communication terminal. Transmission / reception processing to send or receive the second image,
A display instruction image that accepts an instruction to display at least one of the first image and the second image, and at least one of the first image and the second image in response to the instruction. In addition, a display process for displaying a stop instruction image for receiving an instruction to stop the first communication and the second communication, and
When the stop instruction is accepted, the stop process for stopping both the first communication and the second communication, and
It is an output method for executing the management process for managing the output of the stroke data.
When it is determined that the stroke data is stored in the management process when the stop instruction is received, the display process gives priority to the first image over the second image. Output method to display. For communication terminals
The first image relating to the stroke data transmitted or received in the first communication with the other communication terminal, and the video data transmitted or received in the second communication with the other communication terminal. Transmission / reception processing to send or receive the second image,
A display instruction image that accepts an instruction to display at least one of the first image and the second image, and at least one of the first image and the second image in response to the instruction. In addition, a display process for displaying a stop instruction image for receiving an instruction to stop the first communication and the second communication, and
When the stop instruction is accepted, the stop process for stopping both the first communication and the second communication, and
A program that executes a management process that manages the output of the stroke data.
When it is determined that the stroke data is stored in the management process when the stop instruction is received, the display process gives priority to the first image over the second image. Program to display.

Images