JP2017153062A - Communication terminal, communication system, output method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a communication terminal capable of enabling a user to grasp a status change by an event generated by communication with the other communication application in a case where a display with one communication application is given priority over a display with the other communication application.SOLUTION: A display order control unit of an electronic blackboard performs control to output a display with a communication application A11 in preference to a display with a communication application B11. The application I/F unit of the electronic blackboard performs control to cause the communication application A11 to output an icon indicating a state corresponding to an event in accordance with the event generated by communication with the communication application B11.SELECTED DRAWING: Figure 5

Description

  The present invention relates to a communication terminal, a communication system, an output method, and a program.

  2. Description of the Related Art In recent years, communication systems that perform calls, conferences, and the like via communication networks such as the Internet and dedicated lines have become widespread along with requests for reducing expenses and time required for movement of parties. In such a communication system, various content data such as video data of conference participants, image data such as conference materials, or stroke data based on handwriting operations are transmitted between the communication terminals according to the function of the communication terminal. be able to.

  Patent Document 1 discloses a method of transmitting image data through a session established between an electronic blackboard and a call terminal.

  When a plurality of communication applications are activated, the communication terminal may prioritize display by one communication application over display by the other communication application. In this case, even if an event for changing the communication state occurs in communication by the other communication application, the communication terminal gives priority to the display by the one communication application, so that the user can The problem that cannot be grasped.

  In the communication terminal according to the first aspect of the present invention, after the first application for communication and the second application for communication are activated, the display by the first application is changed to the display by the second application. Output means for causing the first application to output state information indicating a state corresponding to the event in response to an event generated by communication by the second application.

  As described above, according to the present invention, when priority is given to the display by one communication application over the display by the other communication application, it is based on the event generated by the communication by the other communication application. There is an effect that the user can grasp the state change.

FIG. 1 is an overall configuration diagram of a communication system according to an embodiment. FIG. 2 is a hardware configuration diagram of the electronic blackboard according to the embodiment. FIG. 3 is a hardware configuration diagram of the management system according to the embodiment. FIG. 4 is a hardware configuration diagram of a PC according to an embodiment. FIG. 5 is a software configuration diagram of an electronic blackboard or PC according to an embodiment. FIG. 6 is a functional block diagram of an electronic blackboard, a PC, and a management system that constitute a part of the communication system according to the embodiment. FIG. 7 is a functional block diagram illustrating a drawing control unit according to an embodiment. FIG. 8 is a sequence diagram showing processing for establishing communication between electronic blackboards. FIG. 9 is a sequence diagram showing processing for establishing communication between the electronic blackboard and the PC. FIG. 10 is a sequence diagram illustrating processing for generating an image. FIG. 11 is a conceptual diagram showing a superimposed image. FIG. 12 is a conceptual diagram showing a superimposed image. FIG. 13 is a sequence diagram illustrating processing for controlling the display order of images. FIG. 14 is a sequence diagram illustrating processing for controlling the display order of images. FIG. 15 is a display example of the display. FIG. 16 is a display example of the display. FIG. 17 is a display example of the display. FIG. 18 is a flowchart showing processing of the electronic blackboard.

  Hereinafter, a first embodiment of the present invention will be described with reference to the drawings.

<< Overall configuration of communication system >>
First, the communication system 1 will be described with reference to FIG. FIG. 1 is an overall configuration diagram of a communication system according to an embodiment.

  The communication system 1 includes a plurality of electronic blackboards 2a, 2b, 2c, a PC 8, and a management system 50.

  The electronic blackboards 2a, 2b, and 2c are communicably connected via a communication network 9a. The electronic blackboards 2a, 2b, and 2c are provided with displays 3a, 3b, and 3c, respectively.

  The electronic blackboards 2a, 2b, and 2c are drawn by an event generated by the electronic pens 4a, 4b, and 4c (touch of the pen tip of the electronic pens 4a, 4b, and 4c, or the pen butt to the displays 3a, 3b, and 3c). The displayed images can be displayed on the displays 3a, 3b, 3c. The electronic blackboards 2a, 2b, and 2c are based on not only the electronic pens 4a, 4b, and 4c but also events (gestures such as enlargement, reduction, and page turning) generated by the user's hands Ha, Hb, and Hc. Images displayed on the displays 3a, 3b, and 3c can be changed. Hereinafter, when an arbitrary electronic blackboard is shown among the electronic blackboards 2a, 2b, and 2c, it is expressed as an electronic blackboard 2. Further, when an arbitrary display among the displays 3a, 3b, and 3c is shown, it is expressed as a display 3. Moreover, when showing arbitrary electronic pens among electronic pen 4a, 4b, 4c, it shows as electronic pen 4. FIG. In addition, when an arbitrary hand among the user's hands Ha, Hb, and Hc is indicated, it is indicated as a hand H.

  In addition, USB (Universal Serial Bus) memories 5a, 5b, and 5c can be connected to the electronic blackboards 2a, 2b, and 2c. The electronic blackboards 2a, 2b, and 2c can read electronic files such as PDF (Portable Document Format) from the USB memories 5a, 5b, and 5c, and record the electronic files in the USB memories 5a, 5b, and 5c. The electronic blackboard 2a is connected to a notebook via a cable capable of communication according to standards such as DisplayPort, DVI (Digital Visual Interface), HDMI (registered trademark) (High-Definition Multimedia Interface), and VGA (Video Graphics Array). PC6a, 6b, 6c is connected. In addition, when showing arbitrary USB memory among USB memory 5a, 5b, 5c, it shows as USB memory 5. FIG. When an arbitrary notebook PC among the notebook PCs 6a, 6b, and 6c is indicated, it is indicated as a notebook PC 6.

  Thereby, the stroke data of the stroke drawn on the display 3 of the electronic blackboard 2 at one base is transmitted to the electronic blackboard 2 at the other base via the communication network 9a, so that the display 3 of the other base is displayed. Also displayed above. Conversely, the stroke data of the stroke drawn on the display 3 of the electronic blackboard 2 at another base is transmitted to the electronic blackboard 2 at the single base via the communication network 9a, so that the electronic blackboard 2 at the single base is displayed. Displayed on the display 3. As described above, the communication system 1 can perform remote sharing processing for sharing the same image at a remote location, and thus is very convenient when used for a conference or the like at a remote location.

  In addition to the electronic blackboard 2, stroke data can be shared with the electronic blackboard 2 by connecting a communication terminal other than the electronic blackboard to the communication network 9 a. Other communication terminals include a PC having a stroke sharing function, a video conference terminal, a tablet, a smartphone, an electronic signboard (digital signage), a telestrator used for sports and weather forecasts, or a remote image (video) diagnostic device And a computer such as a terminal capable of supplying an image frame, such as an image processing device such as a portable information terminal, a digital video camera, a digital camera, and a game machine.

  The electronic blackboard 2c, the PC (Personal computer) 8, and the management system 50 are communicably connected via a communication network 9b.

  The electronic blackboard 2c and the PC 8 perform communication by transmitting and receiving video data and sound data for video conference as an example of content data.

  The management system 50 centrally manages the communication status of the electronic blackboard 2c and the PC 8.

  In addition, video data can also be shared between other communication terminals, the electronic blackboard 2c, and PC8 by connecting communication terminals other than the electronic blackboard 2c and PC8 to the communication network 9b. Other communication terminals include a video conference terminal having a video sharing function, a tablet, a smartphone, a digital signage (digital signage), a telestor, or an image processing device, a portable information terminal, a digital video camera, a digital camera, a game machine, etc. A computer such as a terminal capable of supplying a plurality of image frames is included.

  The electronic blackboard 2c activates both a communication application for sharing stroke data and a communication application for sharing video data, so that stroke data, video data, and sound data can be simultaneously transmitted between terminals. Can be shared.

  According to one embodiment, the communication network 9a is an intranet and the communication network 9b is the Internet. However, the communication networks 9a and 9b are arbitrarily selected from an arbitrary communication network such as the Internet, a LAN (Local Area Network), an intranet, or a mobile phone network according to a communication protocol used by each terminal. Further, the communication networks 9a and 9b may be the same communication network as in the case where both are the Internet.

  In one embodiment, a USB memory will be described as an example of a recording medium. However, the present invention is not limited to this, and various recording media such as an SD card may be used as another example of the recording medium.

<Hardware configuration of electronic blackboard>
A hardware configuration of the electronic blackboard according to the embodiment will be described with reference to FIG. FIG. 2 is a hardware configuration diagram of the electronic blackboard according to the embodiment.

  As shown in FIG. 2, the electronic blackboard 2 includes a CPU (Central Processing Unit) 201 that controls the entire operation of the electronic blackboard 2, a ROM 202 that stores programs used to drive the CPU 201 such as an IPL, and a work of the CPU 201. RAM 203 used as an area, SSD 204 for storing various data such as a program for an electronic blackboard, network I / F 205 for controlling communication with the communication network 9, and external storage controller 206 for controlling communication with the USB memory 5 It has. Furthermore, the electronic blackboard 2 displays video information as a still image or a moving image on the display of the camera 207 that captures the subject according to the control of the CPU 201, the image sensor I / F 208 that controls the driving of the camera 207, and the notebook PC 6. And a display controller 213 that controls and manages screen display in order to output an output image from the GPU to the display 3. Furthermore, the electronic blackboard 2 includes a sensor controller 214 that controls processing of the contact sensor 215 and a contact sensor 215 that detects that the electronic pen 4 or the user's hand H is touching the display 3. In one embodiment, the contact sensor 215 performs coordinate input and coordinate detection by an infrared ray blocking method. In this coordinate input and coordinate detection method, two light emitting / receiving devices installed at both upper ends of the display 3 emit a plurality of infrared rays in parallel with the display 3 and are provided around the display 3. This is a method of receiving light that is reflected by the reflecting member and returns on the same optical path as the light path emitted by the light receiving element. The contact sensor 215 outputs infrared ID (Identification) emitted by the two light receiving and emitting devices blocked by the object to the sensor controller 214, and the sensor controller 214 identifies the coordinate position that is the contact position of the object. All IDs shown below are examples of identification information.

  Further, the contact sensor 215 is not limited to the infrared ray blocking method, but is a capacitance type touch panel that identifies a contact position by detecting a change in capacitance, and a contact position by a voltage change of two opposing resistance films. Various detection means such as a resistance film type touch panel to be identified and an electromagnetic induction type touch panel to identify a contact position by detecting electromagnetic induction caused by contact of a contact object with the display unit may be used. Moreover, the contact sensor 215 may determine a contact and specify a coordinate position using a camera.

  The electronic blackboard 2 includes an electronic pen controller 216. The electronic pen controller 216 communicates with the electronic pen 4 to determine whether the display 3 is touched with a pen tip or a pen butt. It should be noted that the electronic pen controller 216 may determine whether there is a touch not only on the pen tip and the pen butt of the electronic pen 4 but also on the part of the electronic pen 4 that is gripped by the user and other electronic pen parts.

  Furthermore, the electronic blackboard 2 controls the sound input / output I / F 224 for processing input / output of sound signals between the external microphone 222 and the external speaker 223 according to the control of the CPU 101, and the operation button 225 for receiving the user's operation. And a power switch 226 for switching on / off the power supply of the electronic blackboard 2. Note that either the microphone 222 or the speaker 223 may be built in the electronic blackboard 2.

  Further, the electronic blackboard 2 includes a bus line 230 such as an address bus or a data bus for electrically connecting the above components.

  The program for the electronic blackboard may be recorded on a computer-readable recording medium such as a CD-ROM or a server on a network for distribution or download.

<Hardware configuration of management system>
A hardware configuration of the management system according to the embodiment will be described with reference to FIG. FIG. 3 is a hardware configuration diagram of the management system according to the embodiment.

  The management system 50 includes a CPU 501 that controls the overall operation of the management system 50, a ROM 502 that stores a transmission management program, a RAM 503 that is used as a work area for the CPU 501, and an HD (Hard Disk) 504 that stores various data. , An HDD 505 that controls reading and writing of various data to and from the HD 504 according to the control of the CPU 501, and a media I / F 507 that controls reading and writing (storage) of data to a recording medium 506 such as a flash memory. Furthermore, the management system 50 includes a display I / F 508 for displaying various information such as a cursor, menu, window, character, and image on the display 516, and a network I / F for transmitting data using the communication network 9. F509, a keyboard 511 having a plurality of keys for inputting characters, numerical values, various instructions, and the like, and a mouse 512 for selecting and executing various instructions, selecting a processing target, moving a pointing device such as a mouse cursor, and the like And. Further, the management system 50 includes a CD-ROM drive 514 that controls reading and writing of data with respect to a CD-ROM (Compact Disc Read Only Memory) 513 as an example of a removable recording medium, and an external device. An external device I / F 515 for transmitting and receiving information, and a bus line 510 such as an address bus and a data bus for electrically connecting the respective components to each other are provided.

  Further, the transmission management program may be an installable or executable file, recorded on a computer-readable recording medium such as the recording medium 506 or CD-ROM 513, and distributed. . The transmission management program may be stored in the HD 504.

<PC hardware configuration>
FIG. 4 is a hardware configuration diagram of the PC 8 as a video conference terminal according to an embodiment. The PC 8 includes a CPU 801, ROM 802, RAM 803, flash memory 804, SSD 805, media I / F 807, power switch 809, bus line 810, network I / F 811, camera 812, image sensor I / F 813, microphone 814, speaker 815, sound input An output I / F 816, a display I / F 817, an external device connection I / F 818, a keyboard 821, and a mouse 822 are provided. Of these, the CPU 801 controls the overall operation of the PC 8. A ROM (Read Only Memory) 802 stores a program used to drive the CPU 801 such as an IPL (Initial Program Loader). A RAM (Random Access Memory) 803 is used as a work area of the CPU 801. The flash memory 804 stores various data such as a communication program, display data, and sound data. An SSD (Solid State Drive) 805 controls reading or writing of various data with respect to the flash memory 804 according to the control of the CPU 801. An HDD may be used instead of the SSD. A media I / F 807 controls reading or writing (storage) of data with respect to a recording medium 806 such as a flash memory.

  A network I / F (Interface) 811 is an interface for performing data communication using the communication network 9b. The camera 812 is a kind of built-in imaging unit that captures an image of a subject under the control of the CPU 801 and obtains display data. The image sensor I / F 813 is a circuit that controls driving of the camera 812. The microphone 814 is a kind of built-in sound collecting means for inputting sound. The sound input / output I / F 816 is a circuit that processes input / output of a sound signal between the microphone 814 and the speaker 815 under the control of the CPU 801. A display I / F 817 is a circuit that transmits display data to an external display 820 under the control of the CPU 801. The external device connection I / F 818 is an interface for connecting various external devices. The keyboard 821 includes a plurality of keys for inputting characters, numerical values, various instructions, and the like. The mouse 822 performs selection and execution of various instructions, selection of a processing target, movement of a pointing device such as a mouse cursor, and the like.

  The bus line 810 is an address bus, a data bus, or the like for electrically connecting each component shown in FIG. 4 such as the CPU 801.

  The display 820 is a kind of display means configured by a liquid crystal or an organic EL that displays an image of a subject, an operation icon, and the like. The display 820 is connected to the display I / F 817 by a cable 820c. This cable 820c may be an analog RGB (VGA) signal cable, a component video cable, HDMI (High-Definition Multimedia Interface) or DVI (Digital Video). Interactive) signal cable may be used.

  The camera 812 includes a lens and a solid-state image sensor that converts light into electric charges and digitizes a subject image (video). As the solid-state image sensor, a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge Coupled) is used. Device) or the like is used. External devices such as an external camera, an external microphone, and an external speaker can be connected to the external device connection I / F 818 by a USB (Universal Serial Bus) cable or the like. When an external camera is connected, the external camera is driven in preference to the built-in camera 812 according to the control of the CPU 801. Similarly, when an external microphone is connected or when an external speaker is connected, each of the external microphone and the internal speaker 815 is given priority over the internal microphone 814 or the internal speaker 815 according to the control of the CPU 801. An external speaker is driven.

  The recording medium 806 is detachable from the PC 8. Further, as long as it is a non-volatile memory that reads or writes data according to the control of the CPU 801, not only the flash memory 804 but also an EEPROM (Electrically Erasable and Programmable ROM) or the like may be used.

<Software configuration>
FIG. 5A is a software configuration diagram of the electronic blackboard according to the embodiment. As shown in FIG. 5A, an OS (Operating System) 2020, a communication application A11, a communication application B11, and an I / F application C11 operate on a work area 2010 in the RAM 203. Hereinafter, the application is expressed as “application”.

  The OS 2020 is basic software that provides basic functions and manages the entire electronic blackboard. The communication applications A11 and B11 are applications for communicating with other terminals, and have different communication protocols. The communication application A11 is a client application that causes the electronic blackboard 2 to execute a communication control function for transmitting stroke data to other terminals and an image processing function for outputting the stroke data as an image. The communication application B11 is a communication control function for transmitting video data such as conference video and sound data to and from other terminals, an image processing function for inputting / outputting video data, and sound data input / output This is a client application that causes the electronic blackboard 2 to execute a sound processing function for The I / F application C11 communicates with the communication applications A11 and B11 activated in the same electronic blackboard 2, and controls the generation of UI images by the communication applications A11 and B11.

  The OS 2020 and the communication application A11 are installed in the electronic blackboards 2a, 2b, and 2c before shipment from the factory. Further, the communication application B11 and the I / F application C11 can be arbitrarily installed on the electronic blackboard 2 after factory shipment. Hereinafter, the description is continued assuming that the communication application B11 and the I / F application C11 are installed on the electronic blackboard 2c, and the communication application B11 and the I / F application C11 are not installed on the electronic blackboards 2a and 2b.

  FIG. 5B is a software configuration diagram of a PC according to an embodiment. As shown in FIG. 5B, the OS 8020 and the communication application B11 operate on the work area 8010 of the RAM 803.

  The OS 8020 is basic software that provides basic functions and manages the entire PC 8. The communication application B11 causes the PC 8 to execute a communication control function for transmitting video data such as a conference video to / from another communication terminal and an image processing function for outputting the video data as an image.

  Further, the OS 8020 and the communication application B11 are installed in the PC 8 before factory shipment or after factory shipment.

<< Functional Configuration of Embodiment >>
Next, a functional configuration of the communication system according to the embodiment will be described. FIG. 6 is a functional block diagram of the electronic blackboard, the PC, and the management system 50 that constitute a part of the communication system according to the embodiment. In FIG. 6, the electronic blackboards 2a, 2b, and 2c are connected so that data communication can be performed via the communication network 9a. In FIG. 6, the electronic blackboard 2c, the PC 8, and the management system 50 are connected so that data communication can be performed via the communication network 9b.

  The electronic blackboard 2 implements each functional configuration shown in FIG. 6 by the hardware configuration and program shown in FIG. The electronic blackboard 2 also has a storage unit 2000 constructed by the SSD 204. The PC 8 implements each functional configuration shown in FIG. 6 by the hardware configuration and program shown in FIG. Further, the PC 8 has a storage unit 8000 constructed by the SSD 805.

  The electronic blackboard 2 can be a “host device” that starts the remote sharing process first, and can also be a “participating device” that participates in the remote sharing process that has already started. The electronic blackboard 2 is roughly composed of both the client unit 20 and the server unit 90. The client unit 20 and the server unit 90 are functions realized when the communication application A11 is activated in one housing of the electronic blackboard 2. Of the client unit 20 and the server unit 90, the electronic blackboard 2 may include the client unit 20, and the server unit 90 may be a device different from the electronic blackboard 2 such as a distribution control device.

  When the electronic blackboard 2 is a hosting device, the electronic blackboard 2 implements (executes) the client unit 20 and the server unit 90. When the electronic blackboard 2 is a participating device, the client unit 20 is realized (executed) in the electronic blackboard 2, but the server unit 90 is not realized (executed). That is, when the electronic blackboard 2a is a host device and the electronic blackboards 2b and 2c are participating devices, the client unit 20 of the electronic blackboard 2a passes through another server unit 90 implemented in the same electronic blackboard 2a. It communicates with the client part 20 of the blackboards 2b and 2c. On the other hand, the client unit 20 of the electronic blackboard 2b, 2c communicates with the client unit of another electronic blackboard 2a, 2b, or 2c via the server unit 90 realized in the other electronic blackboard 2a.

[Functional configuration of client unit 20]
In the electronic blackboard 2, when the communication application A11 is activated, the client unit 20 starts to operate. The client unit 20 includes a communication control unit 21 and a drawing control unit 22.

  The communication control unit 21 is realized by an instruction from the CPU 201 or a network I / F 205, and controls communication with other electronic blackboards 2, communication with the server unit 90 of the same electronic blackboard 2, and the like.

  The drawing control unit 22 is realized by a command from the CPU 201 and is based on stroke data generated by an operation on the display 3 or data acquired from the USB memory 5, the notebook PC 6, the communication control unit 21, the storage unit 2000, or the like. Perform image processing. In addition, the drawing control unit 22 performs control to generate an image layer based on the processed data and output a superimposed image in which these layers are superimposed.

  FIG. 7 is a functional block diagram illustrating a drawing control unit according to an embodiment. The drawing control unit 22 includes a stroke processing unit 22A, a superimposing unit 22Y, and a page processing unit 22Z.

  The stroke processing unit 22A is realized by a command from the CPU 201. When a stroke drawing event occurs by an operation of the electronic pen 4 or the hand H on the display 3, the stroke processing unit 22A performs a process of generating stroke data corresponding to the operation.

  The superimposing unit 22Y is realized by an instruction from the CPU 201, generates each image layer based on each data, and performs a process of superimposing them.

  The page processing unit 22Z collects each display data such as stroke data into one page data and stores it in the page data storage unit 220 which is a part of the storage unit 2000.

  The page data storage unit 220 is constructed in part of the storage unit 2000 and stores page data as shown in Table 1. Table 1 is a conceptual diagram showing page data. The page data is data for one page displayed on the display 3 (stroke arrangement data (each stroke data) and media data).

  The page data includes a page data ID for identifying an arbitrary page, a start time indicating the time when the display of this page is started, and an end time indicating the time when the content of the page is no longer rewritten by a stroke, a gesture, or the like. The stroke arrangement data ID for identifying the stroke arrangement data generated by the stroke of the electronic pen 4 or the user's hand H and the media data ID for identifying the media data are stored in association with each other. The stroke arrangement data is data for displaying a stroke image on the display 3. The media data is data for displaying another image superimposed on the stroke image on the display 3.

  The stroke arrangement data shows detailed information as shown in Table 2. Table 2 is a conceptual diagram showing stroke arrangement data. As shown in Table 2, one stroke arrangement data is represented by a plurality of stroke data. One stroke data includes a stroke data ID for identifying the stroke data, a start time indicating the start time of writing one stroke, an end time indicating the end time of writing one stroke, the color of the stroke, A stroke width and coordinate array data ID for identifying an array of stroke passing points are shown. For example, when the user draws the alphabet “S” with the electronic pen 4, since the stroke is written with a single stroke, one stroke data ID is indicated and one letter of the alphabet [S] is indicated. However, when the user draws the alphabet “T” with the electronic pen 4, since it is a two-stroke writing, the one letter “T” is indicated by two stroke data IDs.

  Further, the coordinate array data shows detailed information as shown in Table 3. Table 3 is a conceptual diagram showing coordinate array data. The coordinate array data includes one point on the display 3 (X coordinate value, Y coordinate value), the time difference (ms) from the start time of the stroke when passing through this one point, and the electronic pen at this one point. 4 shows each piece of writing pressure information. That is, a set of one point shown in Table 3 is shown by one coordinate array data shown in Table 2. For example, when the user draws the alphabet “S” with the electronic pen 4, it is drawn with a single stroke, but passes through a plurality of passing points until the drawing of “S” is completed. Information.

  Further, the media data of the page data shown in Table 1 shows detailed information as shown in Table 4. Table 4 is a conceptual diagram showing media data. As shown in FIG. 9, the media data is displayed on the display 3 according to the media data ID in the page data shown in Table 1, the data type of the media data, the recording time when the media data is stored, and the media data. Data indicating the position (X coordinate value, Y coordinate value) of the displayed image, the size (width, height) of the image, and the contents of the media data are shown in association with each other. Among these, the position of the image displayed on the display 3 by the media data is displayed by the media data when the coordinates of the upper left corner of the display 3 are (X coordinate value, Y coordinate value) = (0, 0). The position of the upper left corner of the image to be displayed is shown.

[Functional configuration of server unit 90]
The server unit 90 has a communication control unit 91.

  The communication control unit 91 is realized by a command from the CPU 201, and the communication control unit 21 in the client unit 20 in the same electronic blackboard 2 and the communication control unit in the client unit 20 in another electronic blackboard 2 via the communication network 9a. The communication with 21 is controlled.

[Functional configuration of client unit 80]
When the communication application B11 is activated on the electronic blackboard 2 and the PC 8, the client unit 80 starts to operate. The client unit 80 includes a communication control unit 81, a display control unit 82, and an input reception unit 83.
The communication control unit 81 is realized by a command from the CPU 201 or 801 or the network I / F 205 or 811 and controls communication with other terminals.

The display control unit 82 is realized by a command from the CPUs 201 and 801, performs image processing based on data acquired from the storage units 2000 and 8000, and performs control to output an image.
The input receiving unit 83 is realized by a command from the CPUs 201 and 801, and receives an operation input by a user.

[Functional configuration of management system 50]
The management system 50 implements each functional configuration shown in FIG. 6 by the hardware configuration and program shown in FIG. The management system 50 also has a storage unit 5000 constructed from the HD 504.

  The communication control unit 51 of the management system 50 is realized by a command from the CPU 501 and controls communication with the electronic blackboard 2 or the PC 8 via the communication network 9b.

[Functional configuration of electronic blackboard 2c]
The electronic blackboard 2c further includes an application I / F unit 23 and a display order control unit 24.
When the I / F application C11 is activated on the electronic blackboard 2c, the application I / F unit 23 starts to operate. The application I / F unit 23 is realized by a command from the CPU 201 and performs control for causing the client units 20 and 80 to output a predetermined UI image.

  The display order control unit 24 starts to operate when the OS 2020 is activated in the electronic blackboard 2c. The display order control unit 24 is realized by a command from the CPU 201 and controls one of the images created by the client units 20 and 80 to be active and the other to be inactive, thereby controlling the display order of the images. Note that the process of actively controlling display of an image is a process of displaying the image with priority over other images, and is a process of displaying the image in the foreground. The process for inactive display control of an image is a process for displaying other images with priority over the image, and a process for displaying the image on a surface other than the foreground.

<< Process of Embodiment >>
Subsequently, processing according to an embodiment will be described.

(Communication establishment process between electronic blackboards)
Processing for establishing communication by the communication application A11 between the electronic blackboards 2a, 2b, and 2c will be described with reference to FIG. FIG. 8 is a sequence diagram showing processing for establishing communication between electronic blackboards. When the user turns on the power switch 226 of the electronic blackboard 2a, the processing of the client unit 20 of the electronic blackboard 2a starts by starting the communication application A11 (step S1). Then, when a request for sponsoring the meeting is accepted by an operation on the display 3a, an instruction to start the processing of the server unit 90 from the communication control unit 21 of the client unit 20 to the communication control unit 91 in the server unit 90 of the same electronic blackboard 2a Is output. Thereby, in the electronic blackboard 2a, not only the client unit 20 but also the server unit 90 starts various processes (step S2).

  Next, the communication control unit 91 of the electronic blackboard 2a generates connection information for establishing a connection with the electronic blackboard 2a, and displays the generated connection information on the display 3a (step S3). This connection information includes the IP address of the host device and the passcode generated for the current remote sharing process. This passcode is stored in the storage unit 2000. Then, the connection information is transmitted to the users of the electronic blackboards 2b and 2c by telephone or electronic mail by the user of the electronic blackboard 2a.

  Next, in the electronic blackboards 2b and 2c, when the user turns on the power switch 226 of the electronic blackboards 2b and 2c, the communication application A11 is activated to start the processing of the client unit 20 of the electronic blackboards 2b and 2c (step) S4, S5). In the electronic blackboards 2b and 2c, when input of connection information is accepted by an operation on the displays 3b and 3c, the communication control unit 21 in the client unit 20 of each electronic blackboard 2b and 2c performs communication based on the IP address of the connection information. A pass code is transmitted to the communication control unit 91 in the server unit 90 of the electronic blackboard 2a via the network 9a to make a participation request (steps S6 and S7). The communication control unit 91 of the electronic blackboard 2a receives a participation request (including a pass code) from each of the electronic blackboards 2b and 2c.

  Next, the communication control unit 91 authenticates the passcode received from each of the electronic blackboards 2b and 2c using the passcode managed in the storage unit 2000 (step S8). And the communication control part 91 notifies an authentication result to the client part 20 of each electronic blackboard 2b, 2c (step S9, S10). If it is determined by the authentication in step S8 that each of the electronic blackboards 2b and 2c is a valid electronic blackboard, remote sharing between the electronic blackboard 2a that is the hosting device and the electronic blackboards 2b and 2c that are the participating devices Processing communication is established (steps S11 and S12). In the present embodiment, a case where content data transmitted between the electronic blackboards 2a, 2b, and 2c includes stroke data will be described.

  When the user of the electronic blackboard 2a, 2b, 2c draws a stroke image using the electronic pen 4c, the stroke processing unit 22A of the electronic blackboard 2a, 2b, 2c is a contact portion between the electronic pen 4 and the displays 3a, 3b, 3c. Based on the coordinate position, stroke data of the drawn stroke is generated. The communication control unit 21 of the electronic blackboard 2a, 2b, 2c transmits the generated stroke data to the communication control unit 91 of the electronic blackboard 2a that is the hosting device.

  When the communication control unit 91 of the electronic blackboard 2a receives the stroke data sent from the electronic blackboard 2a, 2b, 2c, it transmits this stroke data to the client unit 20 of the electronic blackboard 2a, 2b, 2c. The transmitted stroke data is received by the communication control unit 21 of the electronic blackboards 2a, 2b, 2c. The page processing unit 22Z of the electronic blackboard 2a, 2b, 2c adds and stores the newly received stroke data in the page data storage unit 220 without overwriting the already stored stroke data (step S13). , 14, 15). As a result, the page data storage unit 220 of the electronic blackboards 2a, 2b, and 2c has a stroke generated by any of the electronic blackboards 2a, 2b, and 2c after the communication between the electronic blackboards 2a, 2b, and 2c is established. Data will be accumulated.

(Communication establishment process between electronic blackboard and PC)
Processing for establishing communication between the electronic blackboard 2c and the PC 8 by the communication application B11 will be described with reference to FIG. FIG. 9 is a sequence diagram showing processing for establishing communication between the electronic blackboard and the PC.

  In the electronic blackboard 2c, the communication application B11 is activated in response to the input of the activation request by the user, and the client unit 80 starts various processes (step S21).

  The communication control unit 81 of the client unit 80 transmits a login request to the management system 50 (step S22). This login request includes a user ID and a passcode. Thereby, the communication control unit 51 of the management system 50 receives the login request.

  The management system 50 determines the login request based on whether the set of user ID and passcode included in the login request matches the set of user ID and passcode stored in the storage unit 5000 in advance. The original electronic blackboard 2c is authenticated (step S23). The communication control unit 51 of the management system 50 transmits the authentication result to the electronic whiteboard 2c that is the login request source (step S24). Thereby, the communication control part 81 of the electronic blackboard 2c receives an authentication result. Hereinafter, the description will be continued for the case where it is determined in step S23 that the login request source is valid.

  When the PC 8 receives a request for activation of the communication application B11 based on the operation of the keyboard 821 or the mouse 822, the communication application B11 is activated and the processing of the client unit 80 of the PC 8 can be started (step S31).

  Subsequently, the communication control unit 81 of the PC 8 transmits a login request to the management system 50 in the same manner as the processing between the electronic blackboard 2c and the management system 50 (step S32). Thereby, the management system 50 receives the login request.

  The management system 50 authenticates the login request source PC 8 (step S33), and transmits an authentication result to the PC 8 (step S34). Hereinafter, the description will be continued for the case where it is determined in step S33 that the login request source is valid.

  In response to an operation by the user of the electronic blackboard 2c, the input receiving unit 83 of the electronic blackboard 2c receives an input of a request for starting a call with the PC 8 (step S41). In response to receiving the input of the call start request, the communication control unit 81 of the electronic blackboard 2c transmits a call start request to the management system 50 (step S42). This call start request includes the user ID used when the electronic blackboard 2c logs in to the management system and the user ID of the destination PC 8. As a result, the management system 50 receives a communication start request.

  Next, the management system 50 transmits the call start request to the destination PC 8 (step S43). As a result, the destination PC 8 receives the call start request.

  The communication control unit 81 of the PC 8 transmits information indicating whether or not a response is possible to the management system 50 in response to an operation input by the user (step S44). Hereinafter, the case where the PC 8 transmits information indicating that a response is made to the management system 50 will be described.

  The communication control unit 51 of the management system 50 receives information indicating a response. The communication control unit 51 of the management system 50 transmits information indicating that the response is made to the electronic whiteboard 2c that is the start request source (step S45). Thereby, the communication control part 81 of the electronic blackboard 2c receives the information which shows that it responds.

  Subsequently, the management system 50 performs control for establishing communication for transmitting content data between the electronic blackboard 2c and the PC 8 via the communication network 9b (step S46). As a method for establishing communication, for example, the management system 50 authorizes the electronic blackboard 2c and the PC 8 to access the access destination for the content data. The access destination may be within the management system 50 or a device other than the management system 50. When the communication between the electronic blackboard 2c and the PC 8 is established, the electronic blackboard 2c and the PC 8 access the access destination, and the video data corresponding to the video captured by the cameras 207 and 812 on the own terminal side, and The sound data corresponding to the sound collected by the microphones 222 and 814 is transmitted, and the video data and sound data of the other party are acquired from the access destination (step S47). The video data and the sound data are streaming data, and include image data or sound data of a plurality of still images at different points in time when they are picked up or collected.

  When the communication control unit 81 of the electronic blackboard 2c receives image data and sound data at a new time, it overwrites and updates the image data and sound data stored in the storage unit 2000 with the image data and sound data at the new time. (Steps S48, S49). As a result, the latest image data and sound data are stored in the storage unit 2000.

(Image generation by communication application A11)
Next, a process for generating an image based on stroke data transmitted between the electronic blackboards 2a, 2b, and 2c will be described with reference to FIG. FIG. 10A is a flowchart showing processing for generating an image by the communication application A11. Hereinafter, a process in which the communication application A11 of the electronic blackboard 2c generates an image will be described.

  As an example, the page data storage unit 220 stores image data (media data) of a background image acquired from the USB memory 5 or the like. The page processing unit 22Z of the electronic blackboard 2c acquires the image data of the background image stored in the page data storage unit 220 (Step S51).

  The page data storage unit 220 stores image data (media data) of the object image in advance. The page processing unit 22Z of the electronic blackboard 2c acquires the image data of the object image stored in the page data storage unit 220 (Step S52). The object image is an image showing a predetermined object such as a circle, a rectangle, or a triangle.

  The page processing unit 22Z of the electronic blackboard 2c acquires the stroke data (see step S15) accumulated in the page data storage unit 220 (step S53).

  The page processing unit 22Z of the electronic blackboard 2c generates image data of a stroke image based on the stroke data acquired in step S53 (step S54). For example, when the stroke data indicating the character “S” is acquired in step S53, the page processing unit 22Z generates image data of the image “S”.

  The page data storage unit 220 of the electronic blackboard 2c stores UI image (user interface) image data. The image data of the UI image is, for example, an icon or a button. The page processing unit 22Z of the electronic blackboard 2c acquires the image data of the UI image stored in the page data storage unit 220 (Step S55). Note that among the image data of UI images stored in the page data storage unit 220, there are a plurality of icons indicating the communication status. Among these, the image data acquired by the page processing unit 22Z is controlled by the application I / F unit 23. This process will be described later.

  The superimposing unit 22Y of the electronic blackboard 2c uses the image data acquired or generated in steps S51, S52, S54, and S55 to generate a superimposed image in which a background image, an object image, a stroke image, and a UI image are sequentially superimposed. Image data is generated (step S56).

  The superimposing unit 22Y outputs the image data of the superimposed image generated in step S56 to the display order control unit 24 (step S57). FIG. 11A is a conceptual diagram showing each image to be superimposed. FIG. 11B is a conceptual diagram showing a superimposed image. As shown in FIGS. 11A and 11B, the superimposed image 140 output in step S57 includes a UI image 140a, a stroke image 140b, an object image 140c, and a background image 140d.

  While the communication application A11 is activated, the client unit 20 of the electronic blackboard 2c repeats the processes of steps S51, S52, S53, S54, S55, S56, and S57. As a result, a superimposed image reflecting the latest background image, object image, stroke image, and UI image is repeatedly output to the display order control unit 24.

  In the repeated processing, the client unit 20 skips the processing of step S51 or steps S53, S54 if there is no update to the image data or the stroke data of the background image stored in the page data storage unit 220. May be. Further, in the repeated processing, the client unit 20 may skip the processing of steps S52 and S55 if there is no change in the image data of the object image to be acquired or the image data of the UI image. In this case, previously generated or generated image data is used to generate the superimposed image.

(Image generation by communication app B11)
Next, a process for generating an image based on video data transmitted between the electronic blackboard 2c and the PC 8 will be described with reference to FIG. FIG. 10B is a flowchart showing processing for generating an image by the communication application B11. Hereinafter, a process in which the communication application B11 of the electronic blackboard 2c generates an image will be described.

  The display control unit 82 of the electronic blackboard 2c acquires the latest image data (see step S48) stored in the storage unit 2000 (step S61). The image data of the latest image is the image data of the latest image that constructs the video data received in step S47, and is overwritten and saved in step S48.

  The storage unit 2000 of the electronic blackboard 2c stores UI image image data. The UI image is, for example, an icon or a button. The UI image may be a message corresponding to an event such as “destination list addition request is being accepted”. The display control unit 82 of the electronic blackboard 2c acquires the image data of the UI image stored in the page data storage unit 220 (Step S62). Among the image data of the UI image stored in the page data storage unit 220, there are a plurality of icons indicating the communication status. Among these, the image data acquired by the display control unit 82 is controlled by the application I / F unit 23. This process will be described later.

  The display control unit 82 of the electronic blackboard 2c uses the image data acquired in steps S61 and S62 to generate image data of a superimposed image in which the latest image and UI image are sequentially superimposed (step S63).

  The display control unit 82 of the electronic blackboard 2c outputs the image data of the superimposed image generated in step S63 to the display order control unit 24 (step S64). FIG. 12A is a conceptual diagram showing each superimposed image. FIG. 12B is a conceptual diagram showing a superimposed image. As shown in FIGS. 12A and 12B, the superimposed image 150 output in step S64 includes a UI image 150a and a latest image 150b for constructing a video.

  While the communication application B11 is activated, the client unit 80 of the electronic blackboard 2c repeats the processes of steps S61, S62, S63, and S64. As a result, the latest image for constructing the video and the superimposed image reflecting the UI image are repeatedly output to the display order control unit 24.

  Note that, in the repeated processing, the client unit 80 may skip the process of step S62 if there is no change in the image data of the UI image to be acquired. In this case, previously acquired image data is used to generate the superimposed image.

(Display order control)
Next, a process in which the display order control unit 24 of the electronic blackboard 2c controls the display order of the images output by the client unit 20 and the images output by the client unit 80 will be described with reference to FIG. FIG. 13 is a sequence diagram illustrating processing for controlling the display order of images. FIG. 13 shows a sequence by each part in the electronic blackboard 2c.

  At the timing when the second app of the communication apps A11 and B11 is activated, the display order control unit 24 actively displays the image data (see step S57) output from the client unit 20 as a default process, An image signal is output to the display 3c so that the image data output from the client unit 80 (see step S64) is displayed inactive (step S71). The control of the display order is executed by, for example, processing of superimposing a display area such as a window corresponding to the communication application A11 on a display area such as a window corresponding to the communication application B11 by the OS 2020. When the image output by the client unit 20 is a full-screen image, the display on the display 3 c is occupied by the image output by the client unit 20.

  The page data storage unit 220 stores icon data of a predetermined luminance and image data of a half-luminance icon as a UI image indicating the state of the communication application B11. The icon with a predetermined luminance indicates a state in which a predetermined event occurs in the communication application B11. The half-bright icon indicates a state in which a predetermined event has not occurred in the communication application B11. At the timing when the second app of the communication apps A11 and B11 is activated, as a default process, the app I / F unit 23 displays a UI indicating a half-luminance icon in the UI image indicating the state of the communication app B11. The image information is transmitted to the client unit 20 (step S72).

  When the communication control unit 21 of the client unit 20 receives the UI image information indicating the image data of the half-bright icon, the page processing unit 22Z acquires the image data of the half-bright icon in step S55. Thereby, the superimposed image output to the display order control unit 24 in step S57 includes the half-brightness icon 141 as shown in FIG. Thereby, even when the image output by the client unit 20 is actively controlled, the user can visually recognize the half-bright icon 141 in a state where a predetermined event has not occurred in the communication application B11. I can understand.

  The application I / F unit 23 detects a predetermined event that triggers the state change of the communication application B11. Examples of the predetermined event include an event that triggers the communication application B11 to change to a state in which some input from the user is received. Examples of such an event include reception of a request on the PC 8 side such as reception of a call start request by the PC 8 or reception of a request to add a user of the electronic blackboard 2c to the destination list on the PC 8 side. Hereinafter, a case where the event is reception of an addition request to the destination list will be described.

  The communication control unit 81 of the electronic blackboard 2c receives an addition request to the destination list transmitted from the PC 8 side as an event (step S81).

  The application I / F unit 23 of the electronic blackboard 2c detects reception of an addition request to the destination list as a predetermined event (step S82). There is no particular limitation on the method by which the application I / F unit 23 detects a predetermined event. For example, the application I / F unit 23 may transmit to the communication control unit 81 information for confirming whether a predetermined event has been received at a predetermined interval such as every 30 seconds. In this case, the application I / F unit 23 can detect an event based on a response from the communication control unit 81 to this information. In addition, when receiving a predetermined event, the communication control unit 81 may transmit information indicating the occurrence of the event to the application I / F unit 23. In this case, the application I / F unit 23 detects an event based on the received information.

  The application I / F unit 23 of the electronic blackboard 2c manages a corresponding UI image for each predetermined event. Hereinafter, a case where the application I / F unit 23 manages information indicating an icon having a predetermined luminance as a UI image corresponding to reception of an addition request to the destination list will be described. The application I / F unit 23 transmits change request information indicating an icon of a predetermined luminance corresponding to this event to the client unit 20 in response to detection of reception of the request for adding to the destination list (step S83). The change request information is a command for changing the UI image to the image indicated in the change request information. Thus, when the application I / F unit 23 detects an event of one communication application A11, B11, the application I / F unit 23 transmits change request information to the other communication application B11, A11.

  The communication control unit 21 of the electronic blackboard 2c receives the change request information transmitted by the application I / F unit 23. Based on the received change request information, the page processing unit 22Z of the electronic blackboard 2c outputs image data of an icon having a predetermined luminance among UI image image data stored in the page data storage unit 220 in step S55. get. Thereby, in step S57, an icon image having a predetermined brightness is output as a UI image. FIG. 11C is a conceptual diagram showing a superimposed image changed based on reception of change request information. As shown in FIG. 11C, after the change request information is received, the superimposed image 140 output in step S57 includes an icon 142 having a predetermined luminance. Thus, even when an image output by the client unit 20 is actively controlled, a state in which a predetermined event occurs in the communication application B11 by the user visually recognizing the icon 142 having a predetermined luminance. It can be grasped that it has changed to.

  When the user of the electronic blackboard 2c touches the position where the icon 142 is displayed on the display 3c, the drawing control unit 22 receives an input of a request for changing the display order on the display 3c (step S84).

  In response to accepting an input of a request for changing the display order, the communication control unit 21 of the electronic blackboard 2c transmits display order change request information for changing the display order on the display 3c to the display order control unit 24. (Step S85).

  The display order control unit 24 of the electronic blackboard 2c receives the display order change request information. In response to the reception of the display order change request information, the display order control unit 24 deactivates the image output from the client unit 20 that is the transmission source of the display order change request information and outputs it from the other client unit 80. The image is actively controlled (step S86). Thereby, an image by the communication application B11 as shown in FIG. 12B is output from the display 3c.

  When the user of the electronic blackboard 2c touches the position where the button 153 of the display 3c is displayed, the input receiving unit 83 receives an input of a permission response to the request for adding to the destination list on the PC 8 side (step S87).

  In response to accepting the input of the permission response, the communication control unit 81 of the electronic blackboard 2c transmits a permission response to the PC 8 that requested the addition to the destination list (step S88).

<<< Second Embodiment >>>
Subsequently, with respect to the second embodiment, differences from the first embodiment will be described with reference to FIG. FIG. 14 is a sequence diagram illustrating processing for controlling the display order of images. FIG. 14 shows a sequence by each part in the electronic blackboard 2c.

  The storage unit 2000 stores image data of an icon with a predetermined luminance and image data of an icon with a half luminance as a UI image indicating a state related to the communication application A11. The icon with a predetermined luminance indicates a state in which the communication application A11 has received stroke data. The half-bright icon indicates a state in which the communication application A11 has not received stroke data.

  At the timing when the display order is switched in step S86, the application I / F unit 23 transmits UI image information indicating a half-bright icon among the UI images indicating the state of the communication application A11 to the client unit 80 (step S90). .

  When the communication control unit 81 of the client unit 80 receives the UI image information indicating the image data of the half-luminance icon, the display control unit 82 acquires the image data of the half-luminance icon from the storage unit 2000 in step S62. Thereby, the superimposed image output to the display order control unit 24 in step S64 includes the half-brightness icon 151 as shown in FIG. As a result, even when the image output by the client unit 80 is actively controlled, the user visually recognizes the half-bright icon 151 to confirm that the communication application A11 is not receiving stroke data. I can grasp.

  The application I / F unit 23 detects a predetermined event that triggers a state change of the communication application A11. An example of the predetermined event is reception of stroke data by the communication application A11.

  The communication control unit 21 of the electronic blackboard 2c receives the stroke data transmitted by the communication control unit 91 of the electronic blackboard 2a as an event (step S91).

  The application I / F unit 23 of the electronic blackboard 2c detects the reception of stroke data by the communication control unit 21 of the electronic blackboard 2c as a predetermined event (step S92).

  The application I / F unit 23 of the electronic blackboard 2c manages information indicating a corresponding UI image for each predetermined event. Hereinafter, a case where the application I / F unit 23 manages information indicating an icon having a predetermined luminance as a UI image corresponding to an event of receiving stroke data will be described. In response to detection of an event of receiving stroke data, the application I / F unit 23 transmits change request information indicating an icon having a predetermined luminance corresponding to the event to the client unit 80 (step S93).

  The communication control unit 81 of the electronic blackboard 2c receives the change request information transmitted by the application I / F unit 23. In step S62, the display control unit 82 of the electronic blackboard 2c acquires image data of an icon having a predetermined luminance from the UI image image data stored in the storage unit 2000 based on the received change request information. To do. Thereby, in step S64, an icon image having a predetermined luminance is output as a UI image. FIG. 12C is a conceptual diagram showing a superimposed image that is changed based on reception of change request information. As shown in FIG. 12C, the superimposed image 150 output in step S64 includes an icon 152 having a predetermined luminance. Thus, even when the image output by the client unit 80 is actively controlled, the user can grasp that the communication application A11 has received the stroke data by visually recognizing the icon 152 having a predetermined luminance.

  When the user of the electronic blackboard 2c touches the position where the icon 152 of the display 3c is displayed, the input receiving unit 83 receives an input of a request for changing the display order on the display 3c (step S94).

  In response to accepting an input of a request for changing the display order, the communication control unit 81 of the electronic blackboard 2c transmits display order change request information for changing the display order on the display 3c to the display order control unit 24. (Step S95).

  The display order control unit 24 of the electronic blackboard 2c receives the display order change request information. In response to the reception of the display order change request information, the display order control unit 24 deactivates the image output from the client unit 80 that is the transmission source of the display order change request information, and outputs it from the other client unit 20. The image is actively controlled (step S96). Thus, when the image output from the client unit 20 becomes active, the user of the electronic blackboard 2c can check the newly added stroke.

<<< Third Embodiment >>>
Next, the third embodiment will be described with respect to differences from the first and second embodiments.

  In the first embodiment, a case has been described in which there are two communication apps, the communication apps A11 and B11, activated on the electronic blackboard 2c. In the third embodiment, a case will be described in which there are three or more communication applications activated on the electronic blackboard 2c.

  In the third embodiment, the application I / F unit 23 transmits the identification information of the application in which the event has occurred when transmitting the change request information in steps S83 and S93. Subsequently, when transmitting the display order change request information to the display order control unit 24, the communication control units 21 and 81 transmit the application identification information received in steps S83 and S93. The display order control unit 24 controls the image output by the application identified by the received application identification information to be active and controls the image output by the other application to be inactive. With the above processing, even when there are three or more communication applications, the display order control unit 24 can determine which communication application activates the image.

<<< Fourth Embodiment >>>
Subsequently, the difference between the fourth embodiment and the first to third embodiments will be described. In each of the above embodiments, the case has been described where the event is a request from a communication partner or reception of stroke data. However, the event may be various kinds of information that trigger the presence transition of the communication applications A11 and B11.

  For example, the application I / F unit 23 may detect the reception of the authentication result in steps S10 and 24 as an event (see steps S82 and S92). Since the communication applications A11 and B11 have an online presence triggered by the event, the application I / F unit 23 changes the UI image to an icon corresponding to the online presence in response to the detection of the event. Change request information is transmitted to the client units 20 and 80.

  The event detected by the application I / F unit 23 in steps S82 and S92 is not limited to the event received by the client units 20 and 80. For example, the application I / F unit 23 may detect transmission of a response by the client unit 80 in step S88. The presence of the communication application B11 transitions during a call triggered by the above event. In response to the detection of the event, the application I / F unit 23 transmits change request information for changing the UI image to an icon corresponding to presence during a call to the client unit 20.

<<< Fifth Embodiment >>>
Next, a difference of the fifth embodiment from the first to fourth embodiments will be described. In the fifth embodiment, a display management DB 2001 is constructed by a display management table in the storage unit 2000 of the electronic blackboard 2c. Table 5 is a conceptual diagram showing a display management table.

  The display management table includes event information indicating a predetermined event detected by the application I / F unit 23, format information indicating in what output format a notification corresponding to the event is displayed, and display contents. It is associated and managed. The event information and the format information may be an ID indicating an event or a format, respectively. The display content may be information such as text indicating the display content, a badge, or a file name for specifying a dialog, for example.

  When detecting the event of the client unit 80, the application I / F unit 23 of the electronic blackboard 2c transmits change request information including event information indicating the detected event to the client unit 20 in step S83.

  The communication control unit 21 of the client unit 20 of the electronic blackboard 2c receives the change request information transmitted by the application I / F unit 23. The drawing control unit 22 of the electronic blackboard 2c searches the display management table using event information included in the received change request information as a search key, and acquires corresponding format information and display contents. Then, the drawing control unit 22 generates an image in which the acquired display content is displayed in the format of the acquired format information. In step S56, the superimposing unit 22Y of the electronic blackboard 2c superimposes the generated image on another layer. In step S57, the superimposing unit 22Y outputs the superimposed image including the display content to the display order control unit 24. The display order control unit 24 outputs image data including display contents to the display 3c.

  FIG. 15A shows a display example when the “communication disconnection” event is detected in step S83. In the display management table, “communication disconnection” is associated with the format information “ticker” and the display content “connection to the network is disconnected”. For this reason, the drawing control unit 22 generates an image of the ticker T including the display content. Note that the drawing control unit 22 may generate the image of the ticker T for a predetermined time such as 10 seconds, for example. Thereby, it can prevent that the visibility of a display falls because ticker T is displayed for a long time.

  FIG. 15B is a display example of the display when the “screen sharing” event is detected in step S83. In the display management table, “screen sharing” is associated with format information “dialog” and display content “dialog D1”. The drawing control unit 22 generates an image of the dialog D1 specified by these pieces of information. With this dialog, the electronic blackboard 2c can effectively prompt the user to switch the screen. If communication is not possible without switching to the other party's shared screen, as in this “screen sharing” event, it is possible to increase the degree of prompting switching by a dialog instead of a ticker.

  FIG. 15C is a display example when the “meeting” event (presence) is detected in step S83. In the display management table, “in meeting” is associated with format information “badge” and display content “badge BD1”. The drawing control unit 22 generates an image of the badge BD1 specified by these pieces of information. The electronic blackboard 2c can make the user know that the video conference is in progress by the badge BD1. In the electronic blackboard 2c, the position and area of the badge BD1 are set in advance so as not to disturb the display of other interfaces. Thereby, the electronic blackboard 2c can continuously display the badge BD1 during the meeting without hindering other operations by the user.

  In the above description, an example in which an event occurring in the client unit 80 is detected by the application I / F 23 and information corresponding to the event is output by the client unit 20 has been described. An event that has occurred in the client unit 20 is detected by the application I / F 23, and a process for outputting information corresponding to the event in the client unit 80 is also performed in the same manner as the above process. However, the transmission destination of the information indicating the event by the application I / F unit 23 is the client unit 80, and the processing by the drawing control unit 22 is replaced with the processing by the display control unit 82.

<<< Sixth Embodiment >>>
Next, the sixth embodiment will be described with respect to differences from the first to fifth embodiments.

  The page data storage unit 220 of the electronic blackboard 2c stores a function setting screen and a video conference setting screen. The page processing unit 22Z of the electronic blackboard 2c acquires the function setting screen stored in the page data storage unit 220 at a predetermined timing such as when activated or when requested by the user. In step S57, the superimposing unit 22Y outputs the superimposed image including the acquired function setting screen to the display order control unit 24. The display order control unit 24 outputs the image data including the function setting screen to the display 3c. FIG. 16A illustrates an example of a function setting screen. The function setting screen includes a button B1 for shifting to the video conference setting screen.

  When the user performs an operation of touching the button B1 output to the display 3c with the electronic pen 4c or the like, the page processing unit 22Z of the electronic blackboard 2c acquires the video conference setting screen stored in the page data storage unit 220. To do. In step S57, the superimposing unit 22Y outputs the superimposed image including the acquired video conference setting screen to the display order control unit 24. The display order control unit 24 outputs the image data including the video conference setting screen to the display 3c. FIG. 16B is a diagram illustrating an example of a video conference setting screen.

  When the user checks the check box for using the video conference function in the input field B2 of the video conference setting screen with the electronic pen 4c or the like and performs an operation of inputting the user ID and the passcode, the client unit 20 Stores the input user ID and passcode in the storage unit 2000. When the electronic blackboard 2 is activated after this setting, the client unit 80 automatically makes a login request to the management system 50 based on the user ID and the passcode stored in the storage unit 2000 without accepting an operation by the user.

  Furthermore. When the user performs an operation of selecting a video conference as the priority activation screen in the input field B3 of the video conference setting screen, the client unit 20 stores information indicating the video conference screen as the default priority activation screen. Stored in the unit 2000. Thus, when the electronic blackboard 2 is activated, the display order control unit 24 outputs the image output from the client unit 20 in step S71 based on the information indicating the video conference screen stored in the storage unit 2000. The data is deactivated and the image data output from the client unit 80 is activated and output to the display 3c.

The video conference setting screen shown in FIG. 16B includes a region / language setting button B4. When the user touches the button B4 by the operation of the electronic pen 4c or the like, the client unit 20 in the electronic blackboard 2c. The setting of the area where the electronic blackboard 2c is used is accepted. After the setting, the page processing unit 22Z of the electronic blackboard 2c may generate an image according to the set area. For example, when Japan is set as the region, the page processing unit 22Z may output the conference setting screen without including the video conference setting button B1. Alternatively, when the United States is set as the region, the page processing unit 22Z may output a pre-managed English conference setting screen in response to a request for the conference setting screen.
In addition, by connecting the PC to the communication network 9a or 9b in FIG. 1 and accessing the web browser by inputting the desired IP address of the electronic blackboard 2, the video conference setting screen of the electronic blackboard 2 is displayed on the PC. The above items and values may be settable.

<<< Seventh Embodiment >>>
Next, the seventh embodiment will be described with respect to differences from the first to sixth embodiments.

  The page data storage unit 220 of the electronic blackboard 2c maximizes an output screen by the icon I4 for starting the video conference application B11 and the video conference application B11 as UI image (user interface) image data. An icon I5 for displaying in the foreground and an icon I6 for minimizing the output screen by the video conference application B11 are included.

  When the video conference application B11 is not activated, the page processing unit 22Z of the electronic blackboard 2c acquires the image data of the icon I4 in step S55. If the video conference application B11 is activated but the output from the application B11 is inactive, the page processing unit 22Z acquires the image data of the icon I5 in step S55. When the video conference application B11 is activated and the output from the application B11 is active, the page processing unit 22Z acquires the image data of the icon I6 in step S55. 17A to 17C are display examples of the display 3c when the image data of the icon I4, I5, or I6 is acquired in step S55.

  FIG. 18A is a flowchart showing processing of the electronic blackboard 2c when an operation for selecting the icon I4 is executed. By this operation, the client unit 20 of the electronic blackboard 2c receives a request for starting the video conference application B11 (hereinafter, referred to as a start request) (step S131).

  The communication control unit 21 of the electronic blackboard 2c transmits the accepted activation request to the application I / F unit 23 of the same electronic blackboard 2c (step S132). The application I / F unit 23 transmits the received activation request to the OS 2020 of the same electronic blackboard 2c (step S133).

  In response to the received activation request, the OS 2020 activates the application B11 (step S134). The display order control unit 24 of the OS 2020 controls the image output by the application B11 to be active and the image output from the application A11 to be displayed inactive (step S135).

  FIG. 18B is a flowchart showing processing of the electronic blackboard 2c when an operation for selecting the icon I5 is executed. By this operation, the client unit 20 of the electronic blackboard 2c receives a request (hereinafter referred to as a maximization request) for maximizing and displaying the image output by the video conference application B11 in the foreground (step S141). ).

  The communication control unit 21 of the electronic blackboard 2c transmits the accepted maximization request to the application I / F unit 23 of the same electronic blackboard 2c (step S142). The application I / F unit 23 transmits the received maximization request to the display order control unit 24 of the OS 2020 of the same electronic blackboard 2c (step S143).

  In response to the received maximization request, the display order control unit 24 of the OS 2020 deactivates the image output by the application A11 and maximizes the image output by the application B11 so as to be actively displayed. (Step S144).

  FIG. 18C is a flowchart showing processing of the electronic blackboard 2c when an operation for selecting the icon I6 is executed. With this operation, the client unit 20 of the electronic blackboard 2c receives a request for minimizing an image output by the video conference application B11 (hereinafter, referred to as a minimization request) (step S151).

  The communication control unit 21 of the electronic blackboard 2c transmits the accepted minimization request to the application I / F unit 23 of the same electronic blackboard 2c (step S152). The application I / F unit 23 transmits the received minimization request to the display order control unit 24 of the OS 2020 of the same electronic blackboard 2c (step S153).

  In response to the received minimization request, the display order control unit 24 of the OS 2020 minimizes the image output by the application B11 and deactivates the image so that the image output by the application A11 is actively displayed. (Step S154)

  In the seventh embodiment, the case where the image output by the application A11 is larger than the image output by the application B11 and any one of the icons I4, I5, and I6 is always displayed has been described. For example, when the image output by the application A11 is smaller than the image output by the application B11, the icons I4, I5, and I6 may be displayed on the application B11. In this case, the processing by the client unit 20 and the processing for the client unit 20 are executed in the same manner as the processing of steps S131 to S154 except that the processing by the client unit 80 and the processing by the client unit 80 are replaced.

<< Main effects of embodiment >>
According to the output method of the above embodiment, the display order control unit 24 (an example of output means) of the electronic blackboard 2c (an example of a communication terminal) includes a communication application A11 (an example of a first application for communication), and a communication application. After B11 (an example of a second application for communication) is activated, control is performed to preferentially output the display by the communication application A11 over the display by the communication application B11. The application I / F unit 23 (an example of an output unit) of the electronic blackboard 2c displays an icon (an example of status information) indicating a state corresponding to the event in response to an event that occurs in communication by the communication application B11. Control to output to. Thereby, even when the display by the communication application A11 is prioritized, the state of the communication application B11 is displayed. Therefore, it is possible to prevent the action corresponding to the event generated by the communication by the communication application B11 from being delayed.

  The drawing control unit 22 (an example of an accepting unit) of the electronic blackboard 2c accepts selection of an icon output by the communication application A11. The display order control unit 24 of the electronic blackboard 2c performs control to output the display by the communication application B11 with priority over the display by the communication application A11 in response to the reception of the selection. Thereby, priority can be given to the display by communication application B11 by operation on the display area of communication application A11.

  The application I / F unit 23 of the electronic blackboard 2c performs control for causing the communication application B11 to output an icon indicating a state corresponding to the event according to an event that occurs in communication by the communication application A11. Thereby, even when the display by the communication application B11 is prioritized, it is possible to prevent the action corresponding to the event generated by the communication by the communication application A11 from being delayed.

  The input reception unit 83 (an example of a reception unit) of the electronic blackboard 2c receives a selection of an icon output by the communication application B11. The display order control unit 24 of the electronic blackboard 2c performs control to output the display by the communication application A11 with priority over the display by the communication application B11 in response to the reception of the selection. Thereby, priority can be given to the display by communication application A11 by operation on the display area of communication application B11.

  The event is, for example, a request by the PC 8 that is the communication destination of the communication application B11. Thereby, even when the display by the communication application A11 is prioritized, it is possible to prevent the response to the request from the communication destination PC 8 of the communication application B11 from being delayed.

  The event is information that triggers presence transition. Thereby, even when the display by the communication application A11 is prioritized, when the presence of the communication application B11 transitions, the user can quickly take an action corresponding to the presence.

  The application I / F unit 23 (an example of a transmission unit) of the electronic blackboard 2c transmits information indicating an event that occurs in communication by the communication application B11 to the communication application A11. When receiving the information indicating the event, the communication application A11 outputs a notification (an example of information) corresponding to the information indicating the event. Thereby, the electronic blackboard 2c can prompt the user to switch the screen by this notification.

  A display management table constructed in the storage unit 2000 (an example of a management unit) of the electronic blackboard 2c manages information indicating a format in association with information indicating an event. When receiving the information indicating the event, the communication application A11 outputs a notification in a format indicated by the information managed in the display management table in association with the information indicating the event. Thereby, the electronic blackboard 2c can output a notification in an optimum format according to the detected event.

<< Supplement of Embodiment >>
The communication applications A11, B11, and C11 may be recorded and distributed on a computer-readable recording medium by using an installable or executable file. Other examples of the recording medium include CD-R (Compact Disc Recordable), DVD (Digital Versatile Disk), and Blu-ray Disc. Further, the recording medium or an HD (Hard Disk) in which these programs are stored can be provided domestically or abroad as a program product.

  Further, the electronic blackboard 2, the PC 8, the relay device 30, and the management system 50 in the above embodiment may be constructed by a single device, or a plurality of units arbitrarily assigned by dividing each unit (function or means). It may be constructed by the apparatus.

  Each function of the embodiment described above can be realized by one or a plurality of processing circuits. Here, the “processing circuit” in this specification refers to a processor programmed to execute each function by software, such as a processor including an electronic circuit, or an ASIC designed to execute each function described above. It includes devices such as (Application Specific Integrated Circuit) and conventional circuit modules.

1 communication system 2 electronic blackboard 3 display 4 electronic pen 5 USB memory 6 notebook PC
8 PC
9a, 9b Communication network 20 Client unit 21 Communication control unit 22 Drawing control unit 22A Stroke processing unit 22Y Superimposition unit 22Z Page processing unit 23 Application I / F unit 24 Display order control unit 220 Page data storage unit 30 Relay device 50 Management system 80 Client unit 81 Communication control unit 82 Display control unit 83 Input reception unit 90 Server unit 91 Communication control unit 2000 Storage unit 8000 Storage unit

Japanese Patent Laying-Open No. 2015-88943

Claims (15)

After the first application for communication and the second application for communication are activated, the display by the first application is output with priority over the display by the second application,
A communication terminal comprising: output means for causing the first application to output state information indicating a state corresponding to the event in response to an event that occurs in communication by the second application. Receiving means for receiving selection of the state information output by the first application;
The output means outputs the display by the second application with priority over the display by the first application in response to the selection of the state information output by the first application. Item 2. The communication terminal according to Item 1.   The communication terminal according to claim 2, wherein the output unit causes the second application to output state information indicating a state corresponding to the event in response to an event that occurs in communication by the first application. The accepting means accepts selection of the state information output by the second application;
The output means outputs the display by the first application with priority over the display by the second application in response to reception of selection of the state information output by the second application. Item 4. The communication terminal according to Item 3.   The communication terminal according to any one of claims 1 to 4, wherein the event is a request by a communication destination of the first application.   The communication terminal according to any one of claims 1 to 4, wherein the event is information that triggers presence transition. Transmission means for transmitting information indicating an event occurring in communication by the second application to the first application;
The communication terminal according to any one of claims 1 to 6, wherein when the first application receives information indicating the event, the first application outputs information corresponding to the information indicating the event. In association with the information indicating the event, management means for managing the information indicating the format,
8. The first application receives the information indicating the event and outputs the corresponding information in a format associated with the information indicating the event and indicated by the information managed by the management unit. The communication terminal described. The first application outputs a first interface for prioritizing display by the second application;
9. The operation according to claim 1, wherein, when an operation through the first interface is accepted, the output means outputs the display by the second application with priority over the display by the first application. The communication terminal according to any one of the above. The first application outputs a second interface for prioritizing display by the first application;
10. The device according to claim 1, wherein, when an operation through the second interface is accepted, the output unit outputs the display by the first application with priority over the display by the second application. The communication terminal according to any one of the above. The communication terminal according to any one of claims 1 to 10,
A communication terminal of a communication destination of the first application;
A communication system comprising: The communication terminal according to any one of claims 1 to 10,
A communication terminal of a communication destination of the second application;
A communication system comprising:   The communication system according to claim 12, further comprising a management system that manages communication by the second application. The communication terminal
A process in which the display by the first application is preferentially output with respect to the display by the second application after the first application for communication and the second application for communication are activated;
A process of causing the first application to output state information indicating a state corresponding to the event in response to an event that occurs in communication by the second application;
Output method to execute. In the communication terminal,
A process in which the display by the first application is preferentially output with respect to the display by the second application after the first application for communication and the second application for communication are activated;
A process of causing the first application to output state information indicating a state corresponding to the event in response to an event that occurs in communication of the second application;
A program that executes

Images