JP2021039616A - Information processing system, information processing apparatus, image display method, and program - Google Patents

Abstract

To provide an information processing system, an information processing apparatus, an image display method, and a program that can change an apparatus that performs capturing processing according to a situation.SOLUTION: An information processing system has: an information processing apparatus (content management server 6) that transmits data for displaying a web page to web browsers of a plurality of communication terminals (personal terminals 2a, (b, c), permanent terminal 4); a plurality of communication terminals that receive capturing operations for the web pages displayed on the web browsers; an apparatus determination unit 71 that determines an apparatus that captures a screen displayed on the web page according to the situation of the information processing system; and capturing units 63, 23a (b, c), 43 that capture at least part of the screen in the apparatus determined by the apparatus determination unit. The communication terminals have display control units 24a (b, c), 44 that display the captured images captured by the capturing units.SELECTED DRAWING: Figure 8

Description

The present invention relates to an information processing system, an information processing device, an image display method, and a program.

A conference system is known in which communication terminals communicating via a network share a screen to hold a conference. In the conference system, the captured image captured (captured) by the user can be saved. The user can also record a memo or the like in the captured image.

The user can refer to the captured image or memo later (see, for example, Patent Document 1). Patent Document 1 describes a system in which a communication terminal can display a personal memo of the meeting, a video of the meeting site, and a board at the time of the meeting when the user specifies the meeting.

However, the conventional technique has a problem that the device for capturing the screen is limited. For example, as a device capable of capturing, there is an information processing device corresponding to a communication terminal and a server, but conventionally, it has been decided that either the communication terminal or the information processing device captures the information. However, which device is appropriate to capture may differ depending on the conditions such as the amount of network communication and the processing load of the server. Further, depending on the situation, it may be preferable that a device other than the communication terminal or the information processing device captures the information.

In view of the above problems, it is an object of the present invention to provide an information processing system or the like capable of changing an apparatus for performing import processing according to a situation.

In view of the above problems, the present invention includes an information processing device that transmits data for displaying a web page on the web browsers of a plurality of communication terminals, and a plurality of communication terminals that accept an import operation for the web page displayed by the web browser. In the information processing system having the above, in the device determination unit that determines the device that captures the screen displayed on the web page according to the situation of the information processing system, and the device determined by the device determination unit, the screen It has an intake part that captures at least a part,
The communication terminal is characterized by having a display control unit that displays an captured image captured by the capture unit.

It is possible to provide an information processing system or the like that can change the device that performs the import processing according to the situation.

It is a figure explaining an example of the apparatus which can capture. It is an overall overview diagram of an information processing system during a meeting. It is a figure of an example explaining the outline of the personal portal in an information processing system. It is a hardware block diagram of an example of the computer which concerns on this embodiment. It is a hardware block diagram of an example of the smartphone which concerns on this embodiment. It is a hardware block diagram of an example of the projector which concerns on this embodiment. It is a hardware block diagram of an example of IWB which concerns on this embodiment. It is a functional block diagram of each terminal and a server constituting an information processing system. It is an image diagram of an example of a personal board screen before the display of the projection screen and the first screen capture are performed. It is an image diagram of an example of a personal board screen on which a projection screen is displayed. It is an image diagram of an example of a personal board screen in which the first screen capture was performed. It is a block diagram of an example of a personal memo management DB. It is a block diagram of an example of a shared memo management DB. It is a block diagram of an example of a personal memo DB. It is a block diagram of an example of a shared memo DB. It is a block diagram of an example of interest degree management DB. It is an example flowchart which showed the processing of the information processing system which concerns on this embodiment. It is a sequence diagram of an example of the process from the meeting preparation to the display of the projection screen on the personal board screen. It is an image diagram of an example of a screen for selecting a target to be screen-transmitted to a shared screen. This is an example of the subnet mask setting screen of the content management server. This is an example of a flowchart for explaining a method of determining a device to be captured by the device determination unit of the content management server. It is a sequence diagram of an example of the process of capturing the captured image of the projection screen and making a memo in the captured image and the text memo area (No. 1). It is a sequence diagram of an example of the process of capturing the captured image of the projection screen and making a memo in the captured image and the text memo area (No. 2). It is a sequence diagram of an example of the process of capturing the captured image of the projection screen and making a memo in the captured image and the text memo area (No. 3). It is a figure which shows an example of the setting screen of the apparatus to capture. This is an example of a sequence diagram showing a procedure in which each personal terminal transmits a capture device to a content management server. It is a functional block diagram of each terminal and server constituting an information processing system (Example 3). It is a figure which shows typically the information registered in the session management DB. This is an example of a flowchart showing a procedure in which the device determination unit of the content management server determines the device to be captured. It is a figure which shows an example of the setting screen of the number of terminals which becomes a threshold value. This is an example of a flowchart showing a procedure in which the content management server determines a device to capture according to the number of captures. It is a figure which shows an example of the setting screen of the judgment standard which determines a device to capture. It is a figure which shows an example of the setting screen of the judgment standard which determines a device to capture. This is an example of a sequence diagram showing a procedure in which the content management server sends a script to a personal terminal according to the capture method.

Hereinafter, the information processing system and the image display method performed by the information processing system will be described as an example of the embodiment for carrying out the present invention.

In this embodiment, an example of an information processing system used for a conference and which can use a shared screen and a personal board will be described, but the present invention is not limited to this, and various information processing systems used for seminars, classes, etc. It can be applied to various information processing systems. Further, in the present embodiment, an example of a remote conference in which there are users who are remotely connected is described, but all the users may be in the same room, and the users are physically separated from each other. Does not have to be present in.

<Example of capturing device>
FIG. 1 is a diagram illustrating an example of a device capable of capturing. The personal terminal 2a and the personal terminal 2c communicate with each other via the content management server 6 while sharing the screen for a meeting or the like. The personal terminal 2a is connected to the second network, and the personal terminal 2c is connected to the telephone communication network 10. The content management server 6 is connected to the first network. The first network is mainly the Internet, and the second network is a local network (inside the firewall) such as an in-house network or LAN.

For example, a case where the personal terminal 2a transmits the projection screen to the content management server 6 will be described. In this case, the personal terminal 2a is a transmitting terminal. Since the content management server 6 transmits the projection screen to the permanent terminal 4 and the personal terminal 2c, any device that receives the projection screen can capture the screen. For example, when the user of the personal terminal 2c performs the capture operation, each of the four devices can capture the projection screen. The four devices are represented by A to D as shown below.
A: Permanent terminal 4
B: Personal terminal 2a (transmitting terminal)
C: Content management server D: Personal terminal 2c
The relay device 5 is arranged in the second network and relays the communication between the personal terminal 2a and the permanent terminal 4. The relay device 5 may not be provided. Since the relay device 5 is provided, the only device that can pass through the firewall needs to be the relay device 5, so that the workload of the administrator can be reduced. The relay device 5 may be referred to as a proxy server. When the relay device 5 is arranged, the number of devices that can be captured is five.

As shown in Table 1, there are advantages and disadvantages depending on the devices A to D that capture.

表１は、Ａ〜Ｄの装置がキャプチャする場合のメリットとデメリットを示す。例えば、Ｄのように個人端末２ｃがキャプチャ処理を行った場合、個人端末２ｃからコンテンツ管理サーバ６へキャプチャ画像をアップロードする必要があることから（コンテンツ管理サーバ６は全てのキャプチャ画像を管理する）、通信量が多くなるおそれがある。また、コンテンツ管理サーバ６が大量の画像を受信するため負荷がかかる。しかし、キャプチャしたキャプチャ画像を個人端末２ｃが表示するまでの時間が短いというメリットがある。

Table 1 shows the advantages and disadvantages when the devices A to D capture. For example, when the personal terminal 2c performs the capture process as in D, it is necessary to upload the captured image from the personal terminal 2c to the content management server 6 (the content management server 6 manages all the captured images). , There is a risk that the amount of communication will increase. In addition, the content management server 6 receives a large amount of images, which imposes a load. However, there is an advantage that the time until the captured image is displayed on the personal terminal 2c is short.

Further, in the case of C, the content management server 6 needs to transmit the captured image to the personal terminal 2c, and it may take time for the personal terminal 2c to display the captured image. However, there is no need for a communication fee for uploading by the personal terminal 2c, and there is no need for the content management server 6 to receive the image.

In this way, there are advantages and disadvantages to which device captures. Therefore, in the present embodiment, it is possible to determine the device to capture according to the situation. Details will be described later, but for example, when the personal terminal 2a connected to the second network requests capture, the content management server causes the personal terminal 2a to capture. When the personal terminal 2c connected to the telephone communication network 10 requests capture, the content management server 6 determines that it captures itself. Therefore, the information processing system of this embodiment can appropriately determine the device for capturing according to the situation.

<Terminology>
Capturing a screen means saving the image displayed on the screen as data. In addition, it is said to capture the capture. The captured image is called a captured image or a captured image. The operation for capturing is called a capture operation or a capture operation.

Determining the device to be captured also includes switching or changing the device to be captured as a result.

The situation is a judgment factor that determines the device to be captured. The status of the network to which the personal terminal 2 that accepted the capture operation is connected, the status of settings by the user, the status of communication speed such as network bandwidth, the status of the number of personal terminals 2 participating in the conference, the status of the number of captures, There are various situations such as.

<Overview of the information processing system during the meeting>
First, the overall outline of the information processing system 100 during the meeting will be described with reference to FIG. FIG. 2 is an overall schematic view of an example of the information processing system 100 during a meeting. Here, a state is shown in which users A, B, and C in the meeting room X of the company are holding a meeting using the information processing system. In the conference room X, the user A brings the personal terminal 2a, the user B brings the personal terminal 2b, and the user C brings the personal terminal 2c. In the following, when the generic name of the personal terminals 2a, 2b, and 2c is shown, it is described as "personal terminal 2". Further, an example in which user A is a presenter and users B and C are listeners will be described below.

The personal terminal 2 is a computer that can be used individually (dedicatedly) and viewed (referenced) by each user (an example of an information processing device within the scope of claims). The personal terminal 2 is, for example, a notebook PC (Personal Computer), a desktop PC, a mobile phone, a smartphone, a tablet terminal, a wearable PC, or the like. The personal terminal 2 is an example of a communication terminal.

The personal terminal 2 can communicate with the content management server 6 via a communication network 9 such as the Internet. The communication network 9 may be one or more LANs (Local Area Network) inside the firewall, or may include the Internet outside the firewall in addition to the LAN. In addition, VPN (Virtual Private Network) and wide area Ethernet (registered trademark) may be included. The communication network 9 may be constructed either by wire or wirelessly, and may be a combination of wired and wireless. When the communication network 9 is connected by a mobile phone network such as 3G, LTE (Long Term Evolution), or 4G, there is no need for a LAN.

The content management server 6 is a computer having a function as a web (Web) server (or HTTP server) that stores and manages content data to be transmitted to the personal terminal 2 (information processing within the scope of patent claims). An example of a device or a second information processing device). The content management server 6 includes a storage unit 6000, which will be described later.

A storage position (or storage area) for realizing personal boards dc1 to dc3 that can be accessed only from each personal terminal 2 is constructed in the storage unit 6000. Only personal terminals 2a, 2b, and 2c can access the personal boards dc1, dc2, and dc3, respectively. In the following, when the general term of the personal board dc1, dc2, dc3 is shown, it is referred to as "personal board dc". The content management server 6 may support cloud computing. Cloud computing refers to a usage pattern in which resources on a network are used without being aware of specific hardware resources. Further, the storage unit 6000 of the content management server 6 has a storage position (or storage area) for realizing the shared screen ss described later.

The personal board dc is a virtual space created in the storage position (or storage area) of the storage unit 6000 of the content management server 6. The personal board dc can be accessed by, for example, a web application having a function of viewing and editing contents by using a Canvas element and JavaScript (registered trademark) or the like.

A web application operates by coordinating a program in a script language (for example, JavaScript (registered trademark)) that operates on a web browser application (hereinafter referred to as a web browser) and a program on the web server side, and operates on the web browser. Indicates the software used in or its mechanism. The personal board dc has a finite or infinite area within the storage area of the storage unit 6000. For example, the personal board dc may be finite or infinite in both the vertical and horizontal directions, or may be finite or infinite in either the vertical and horizontal directions.

Further, the shared screen ss is a virtual space created in the storage position (or storage area) of the storage unit 6000 of the content management server 6. This shared screen ss has a function of holding the data of the content uploaded by streaming from the personal terminal 2a of the user A who is the presenter until the data of the next content is acquired. The shared screen ss is, for example, a computer screen such as an application screen, and is a target for capturing a captured image as described later.

Further, the personal board dc is an electronic space dedicated to each individual of the users participating in the conference. Each user's personal terminal 2 can access only the personal board dc dedicated to each user, and can view and edit (input, delete, copy, etc.) contents such as characters and images.

The content management server 6 manages information such as contents developed on the shared screen ss and the personal board dc in association with each virtual conference room (hereinafter, simply referred to as a room) which is an example of a virtual room. .. As a result, even if the content management server 6 manages a plurality of rooms, the content data is not communicated between the different rooms.

Since each personal terminal 2 can access the contents of the personal board dc and the shared screen ss of the room in which the user participates by the web application of the installed web browser, it is possible to hold a conference close to the actual conference room. ..

With such an information processing system, the presenter user A uses the captured image of the content uploaded to the shared screen ss as personal material as the personal material of the listeners B and C's personal board dc. Can be incorporated into.

The system configuration of FIG. 2 is based on the case where the personal terminal 2c is also present in the conference room X. As shown in FIG. 1, one or more personal terminals 2a to 2c may be connected to the telephone communication network 10. Further, the permanent terminal 4 may be arranged.

<Overview of personal portal in information processing system>
Next, the outline of the personal portal will be described with reference to FIG. FIG. 3 is an example diagram illustrating an outline of a personal portal in an information processing system. The content management server 6 generates data of the personal portal screens dp1, dp2, dp3 dedicated to each personal terminal 2 and displays them on each personal terminal 2. In the following, when the general term of the personal portal screen dp1, dp2, dp3 is shown, it is described as "personal portal screen dp".

The content management server 6 stores the shared memo cm, which is the content edited during the meeting on the shared screen ss of FIG. 3, and the personal memo dm1, dm2, dm3, which is the content edited on the personal board dc. Be managed. In the following, when the generic name of the personal memo dm1, dm2, dm3 is shown, it is described as "personal memo dm". By accessing the dedicated personal portal screen dp of each personal terminal 2, each user can display a list of conferences in which the user who operates the personal terminal 2 participates.

The user can display the shared memo cm, the personal memo dm, and the bibliographic information of the conference of each conference from the list of conferences displayed on the personal portal screen dp as described later. Therefore, the user can easily display the desired shared memo cm, personal memo dm, and bibliographic information of the conference when looking back on the contents of the conference. In addition, each user accesses the dedicated personal portal screen dp of each personal terminal 2, and from the list of meetings of the user who operates the personal terminal 2, the desired meeting can be referred to as a keyword (character) as described later. You can search. As the search target of the character search, bibliographic information of the conference, text data and handwritten characters included in the personal memo dm, evaluation of the conference by the user, and the like are used. The bibliographic information of the conference is included in the conference information.

<Hardware configuration>
<< Computer >>
The content management server 6 is realized by, for example, the computer 500 having the hardware configuration shown in FIG. Further, when the personal terminal 2 is a PC which is an example of an information processing terminal, it is realized by, for example, the computer 500 having the hardware configuration shown in FIG.

FIG. 4 is a hardware configuration diagram of an example of the computer 500 according to the present embodiment. As shown in FIG. 4, the computer 500 includes a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, an HD504, an HDD (Hard Disk Drive) controller 505, and a display 506. , External device connection I / F (Interface) 508, Network I / F 509, Data bus 510, Keyboard 511, Pointing device 512, DVD-RW (Digital Versatile Disk Rewritable) drive 514, Media I / F 516.

Of these, the CPU 501 controls the operation of the entire computer 500. The ROM 502 stores a program used to drive the CPU 501 such as an IPL. The RAM 503 is used as a work area of the CPU 501. The HD504 stores various data such as programs. The HDD controller 505 controls reading or writing of various data to the HD 504 according to the control of the CPU 501.

The display 506 displays various information such as a cursor, a menu, a window, a character, or an image. The external device connection I / F 508 is an interface for connecting various external devices. The external device in this case is, for example, a USB (Universal Serial Bus) memory, a printer, or the like. The network I / F 509 is an interface for performing data communication using the communication network 9. The data bus 510 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 501.

The keyboard 511 is a kind of input means including a plurality of keys for inputting characters, numerical values, various instructions, and the like. The pointing device 512 is a kind of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like. The DVD-RW drive 514 controls reading or writing of various data to the DVD-RW 513 as an example of the removable recording medium. In addition, it is not limited to DVD-RW, and may be DVD-R or the like. The media I / F 516 controls reading or writing (storage) of data to a recording medium 515 such as a flash memory.

<< Smartphone >>
The personal terminal 2, which is an example of the information processing terminal, may be realized by, for example, the smartphone 600 having the hardware configuration shown in FIG.

FIG. 5 is a hardware configuration diagram of an example of the smartphone 600 according to the present embodiment. As shown in FIG. 5, the smartphone 600 includes a CPU 601, ROM 602, RAM 603, EEPROM 604, CMOS sensor 605, image sensor I / F 606, acceleration / orientation sensor 607, media I / F 609, and GPS receiver 611. There is.

Of these, the CPU 601 controls the operation of the entire smartphone 600. The ROM 602 stores a program used to drive the CPU 601 such as the CPU 601 and the IPL. The RAM 603 is used as a work area of the CPU 601. The EEPROM 604 reads or writes various data such as a smartphone program under the control of the CPU 601.

The CMOS (Complementary Metal Oxide Semiconductor) sensor 605 is a kind of built-in photographing means for acquiring image data by photographing a subject (mainly a self-portrait) under the control of the CPU 601. Instead of the CMOS sensor 605, a photographing means such as a CCD (Charge Coupled Device) sensor may be used. The image sensor I / F 606 is a circuit that controls the drive of the CMOS sensor 605. The acceleration / orientation sensor 607 is a variety of sensors such as an electronic magnetic compass, a gyro compass, and an acceleration sensor that detect the geomagnetism.

The media I / F 609 controls reading or writing (storage) of data to a recording medium 608 such as a flash memory. The GPS receiving unit 611 receives GPS signals from GPS satellites.

Further, the smartphone 600 includes a long-distance communication circuit 612, a CMOS sensor 613, an image sensor I / F 614, a microphone 615, a speaker 616, a sound input / output I / F 617, a display 618, an external device connection I / F 618, and a short-range communication circuit 620. , The antenna 620a of the short-range communication circuit 620, and the touch panel 621 are provided.

Of these, the long-distance communication circuit 612 is a circuit that communicates with other devices via the communication network 9. The CMOS sensor 613 is a kind of built-in photographing means for photographing a subject and obtaining image data under the control of the CPU 601. The image sensor I / F 614 is a circuit that controls the drive of the CMOS sensor 613. The microphone 615 is a built-in circuit that converts sound into an electrical signal. The speaker 616 is a built-in circuit that converts an electric signal into physical vibration to produce sounds such as music and voice.

The sound input / output I / F 617 is a circuit that processes sound signal input / output between the microphone 615 and the speaker 616 under the control of the CPU 601. The display 618 is a kind of display means such as a liquid crystal or an organic EL (Electro Luminescence) for displaying an image of a subject, various icons, and the like.

The external device connection I / F619 is an interface for connecting various external devices. The short-range communication circuit 620 is a communication circuit such as NFC (Near Field Communication) or Bluetooth (registered trademark). The touch panel 621 is a kind of input means for operating the smartphone 600 by the user pressing the display 618.

In addition, the smartphone 600 is provided with a bus line 610. The bus line 610 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 601 shown in FIG.

<< Projector >>
The projector 700, which is an example of the permanent terminal 4, may be realized by, for example, the hardware configuration shown in FIG.

FIG. 6 is a hardware configuration diagram of an example of the projector 700 according to the present embodiment. As shown in FIG. 6, the projector 700 includes a CPU 701, a ROM 702, a RAM 703, a media I / F 707, an operation unit 708, a power switch 709, a bus line 710, a network I / F 711, an LED (Light Emitting Diode) drive circuit 714, and an LED. It includes a light source 715, a projection device 716, a projection lens 717, an external device connection I / F718, a fan drive circuit 719, and a cooling fan 720.

Of these, the CPU 701 controls the operation of the entire projector 700. The ROM 702 stores a program used to drive the CPU 701. The RAM 703 is used as a work area for the CPU 701. The media I / F 707 controls reading or writing (storage) of data to a recording medium 706 such as a flash memory.

The operation unit 708 is provided with various keys, buttons, LEDs, and the like, and is used for performing various operations other than turning on / off the power of the projector 700 by the user. For example, the operation unit 708 receives instruction operations such as a projected image size adjustment operation, a color tone adjustment operation, a focus adjustment operation, and a keystone adjustment operation, and outputs the received operation contents to the CPU 701.

The power switch 709 is a switch for switching ON / OFF of the power supply of the projector 700. The bus line 710 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 701 shown in FIG. The network I / F711 is an interface for data communication using a communication network 9 such as the Internet.

The LED drive circuit 714 controls the lighting and extinguishing of the LED light source 715 under the control of the CPU 701. When the LED light source 715 is turned on under the control of the LED drive circuit 714, the LED light source 715 irradiates the projection device 716 with the projected light. The projection device 716 transmits the modulated light obtained by modulating the projected light from the LED light source 715 by the spatial light modulation method based on the image data given via the external device connection I / F718 or the like through the projection lens 717. , Project as an image on the projection surface of the screen. As the projection device 716, for example, a liquid crystal panel or a DMD (Digital Micro mirror Device) or the like is used.

The LED drive circuit 714, the LED light source 715, the projection device 716, and the projection lens 717 function as a projection unit (projection means) for projecting a projected image on the projection surface based on the image data as a whole.

The external device connection I / F718 is directly connected to a PC (Personal Computer) and acquires control signals and image data from the PC. Further, the external device connection I / F718 is an interface for connecting various external devices (stick PC730, etc.). The fan drive circuit 719 is connected to the CPU 701 and the cooling fan 720, and drives / stops the cooling fan 720 based on the control signal from the CPU 701. The cooling fan 720 rotates to exhaust the air inside the projector 700 and cool the inside of the projector 700.

Further, when the power supply power is supplied, the CPU 701 is activated according to the control program stored in advance in the ROM 702, gives a control signal to the LED drive circuit 714 to light the LED light source 715, and causes the fan drive circuit 719 to light up. A control signal is given to rotate the cooling fan 720 at a predetermined rated rotation speed. Further, when the power supply power from the power supply circuit is started to be supplied to the projector 700, the projection device 716 is in a state where the image can be displayed, and power is further supplied from the power supply circuit to various other components. Further, when the power switch 709 of the projector 700 is turned off, a power OFF signal is sent from the power switch 709 to the CPU 701.

When the CPU 701 detects the power OFF signal, it gives a control signal to the LED drive circuit 714 to turn off the LED light source 715. After that, when a predetermined time elapses, the CPU 701 gives a control signal to the fan drive circuit 719 to stop the cooling fan 720, terminates its own control process, and finally gives an instruction to the power supply circuit to supply power. Stop the supply of.

<< IWB >>
The IWB 800, which is an example of the permanent terminal 4, may be realized by, for example, the hardware configuration shown in FIG. 7.

FIG. 7 is a hardware configuration diagram of an example of the IWB 800 according to the present embodiment. As shown in FIG. 7, the IWB 800 includes a CPU 801 and a ROM 802, a RAM 803, an SSD (Solid State Drive) 804, a network I / F805, and an external device connection I / F806.

Of these, the CPU 801 controls the operation of the entire IWB 800. The ROM 802 stores a program used to drive the CPU 801 such as the CPU 801 and the IPL. The RAM 803 is used as a work area of the CPU 801. The SSD 804 stores various data such as a program for IWB. The network I / F805 controls communication with the communication network 9. The external device connection I / F806 is an interface for connecting various external devices. The external device in this case is, for example, a USB memory 830, an external device (microphone 840, speaker 850, camera 860).

Further, the IWB 800 includes a capture device 811, a GPU 812, a display controller 813, a contact sensor 814, a sensor controller 815, an electronic pen controller 816, a short-range communication circuit 819, and an antenna 819a, a power switch 822, and a selection switch of the short-range communication circuit 819. It has a class 823.

Of these, the capture device 811 causes the display of the external PC 870 to display the video information as a still image or a moving image. The GPU (Graphics Processing Unit) 812 is a semiconductor chip that specializes in graphics. The display controller 813 controls and manages the screen display in order to output the output image from the GPU 812 to the display 880 or the like.

The contact sensor 814 detects that the electronic pen 890, the user's hand H, or the like has come into contact with the display 880. The sensor controller 815 controls the processing of the contact sensor 814. The contact sensor 814 inputs and detects the coordinates by the infrared blocking method. In this method of inputting coordinates and detecting coordinates, two light receiving and emitting devices installed at both upper ends of the display 880 radiate a plurality of infrared rays in parallel with the display 880 and are provided around the display 880. This is a method of receiving light that is reflected by a reflecting member and returns on the same optical path as the light path emitted by the light receiving element.

The contact sensor 814 outputs an infrared ID emitted by two light emitting / receiving devices blocked by the object to the sensor controller 815, and the sensor controller 815 specifies a coordinate position which is a contact position of the object. The electronic pen controller 816 determines whether or not there is a touch of the pen tip or a touch of the pen tail on the display 880 by communicating with the electronic pen 890. The short-range communication circuit 819 is a communication circuit such as NFC or Bluetooth (registered trademark). The power switch 822 is a switch for switching ON / OFF of the power supply of the IWB 800. The selection switches 823 are, for example, a group of switches for adjusting the brightness and color tone of the display of the display 880.

In addition, the IWB 800 is equipped with a bus line 810. The bus line 810 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 801 shown in FIG. 7.

The contact sensor 814 is not limited to the infrared blocking method, but is a capacitive touch panel that specifies the contact position by detecting a change in capacitance, and the contact position is specified by a voltage change between two opposing resistance films. Various detection means such as a resistance film type touch panel and an electromagnetic induction type touch panel that detects the electromagnetic induction generated when a contact object comes into contact with the display unit and specifies the contact position may be used. Further, the electronic pen controller 816 may determine whether or not there is a touch not only on the pen tip and pen end of the electronic pen 890, but also on the portion gripped by the user of the electronic pen 890 and other electronic pen portions. ..

<Functional configuration>
The functional configuration of each terminal and server constituting the information processing system will be described with reference to FIG. FIG. 8 is a functional block diagram of each terminal and server constituting the information processing system.

<< Functional configuration of personal terminal >>
First, the functional configuration of the personal terminal 2a will be described. As shown in FIG. 8, the personal terminal 2a includes a transmission / reception unit 21a, a reception unit 22a, an image processing unit 23a, a display control unit 24a, a determination unit 25a, a storage / reading processing unit 29a, and a communication management unit 30a. Have. Each of these parts is a function realized or a means for functioning by operating any of the components shown in FIG. 5 by an instruction from the CPU 601 according to a program developed from the EEPROM 604 to the RAM 603. is there. Further, the personal terminal 2a has a storage unit 2000a constructed by the RAM 603 and the EEPROM 604 shown in FIG.

The transmission / reception unit 21a, the reception unit 22a, the image processing unit 23a, the display control unit 24a, the judgment unit 25a, and the storage / reading processing unit 29a are realized by a web browser (web application) that displays a personal board screen described later. Will be done. The communication management unit 30a is realized by a dedicated communication application.

(Each function configuration)
Next, each functional configuration of the personal terminal 2a will be described in detail. The transmission / reception unit 21a transmits / receives various data (or information) to / from another terminal, device, server, or the like via the communication network 9. For example, the transmission / reception unit 21a receives content data described in HTML, CSS (Cascade Style Sheet), and JavaScript (registered trademark) from the content management server 6. Further, the transmission / reception unit 21a transmits the operation information input by the user to the content management server 6.

The reception unit 22a receives various inputs from the touch panel 621 by the user. The image processing unit 23a performs processing such as creating vector data (or stroke data) according to the drawing of the user, for example. Further, the image processing unit 23a has a function as a capture unit, and captures a captured image by, for example, photographing a shared screen ss.

The display control unit 24a causes the display 618 to display the personal board screen and the shared screen ss, which will be described later. The determination unit 25a makes various determinations. The storage / reading processing unit 29a is realized by the instruction from the CPU 601 and the EEPROM 604, and performs a process of storing various data in the storage unit 2000a and reading various data from the storage unit 2000a.

The communication management unit 30a is mainly realized by an instruction from the CPU 601 shown in FIG. 5, and inputs / outputs data to / from the transmission / reception unit 21a and the like. The communication management unit 30a further includes a transmission / reception unit 31a, an acquisition unit 33a, and a determination unit 35a.

Of these, the transmission / reception unit 31a transmits / receives various data (or information) to and from the content management server 6 via the communication network 9 independently of the transmission / reception unit 21a. The capturing unit 33a basically has the same function as the function as the capturing unit of the image processing unit 23a. For example, the capture unit 33a captures the captured image by performing a screen capture of the shared screen ss described later. The determination unit 35a makes various determinations, and determines, for example, whether or not the captured image is referenced by the user. Since the functional configurations of the personal terminals 2b and 2c are the same as the functional configurations of the personal terminal 2a, the description thereof will be omitted.

<< Functional configuration of permanent terminal >>
Subsequently, the functional configuration of the permanent terminal 4 will be described. As shown in FIG. 8, the permanent terminal 4 includes a transmission / reception unit 41, a reception unit 42, an image processing unit 43, a display control unit 44, a determination unit 45, a storage / reading processing unit 49, and a communication management unit 50. Have. Each of these parts has a function or a function realized by operating any of the components shown in FIG. 6 or the like by an instruction from the CPU 701 according to a program developed on the RAM 703 from the recording medium 706. It is a means to do.

Each part is a function realized by operating any of the components shown in FIG. 6 by a command from the CPU of the stick PC 730 according to a program developed on the RAM of the stick PC 730. , Or it may be a functioning means. Further, the permanent terminal 4 has a storage unit 4000 constructed by the RAM 703 shown in FIG. 6 and the like. A shared memo DB 4002 is constructed in the storage unit 4000 of the permanent terminal 4.

In the permanent terminal 4, the transmission / reception unit 41, the reception unit 42, the image processing unit 43, the display control unit 44, the judgment unit 45, the storage / reading processing unit 49, the communication management unit 50, and the storage unit 4000 are individuals. Since it is the same as the transmission / reception unit 21a, the reception unit 22a, the image processing unit 23a, the display control unit 24a, the judgment unit 25a, the storage / reading processing unit 29a, the communication management unit 30, and the storage unit 2000a in the terminal 2a, the description is omitted. To do. Further, the communication management unit 50 in the permanent terminal 4 has a transmission / reception unit 51, an acquisition unit 53, and a determination unit 55, which are the same as the transmission / reception unit 31a, the acquisition unit 33a, and the determination unit 35a, respectively. Since it is a function, the description thereof will be omitted.

The transmission / reception unit 41, the reception unit 42, the image processing unit 43, the display control unit 44, the determination unit 45, and the storage / reading processing unit 49 are realized by a web browser (web application) that displays the shared screen ss. .. The communication management unit 50 is realized by a dedicated communication application.

<< Functional configuration of content management server >>
Subsequently, the functional configuration of the content management server 6 will be described. As shown in FIG. 8, the content management server 6 includes a transmission / reception unit 61, a schedule interlocking unit 62, an image processing unit 63, a creation unit 64, a judgment unit 65, a web page creation unit 66, a search unit 67, and an authentication unit. It has 68, an apparatus determination unit 71, and a storage / reading processing unit 69. Each of these parts is a function realized by operating any of the components shown in FIG. 4 by an instruction from the CPU 501 according to a program developed on the RAM 503 from the HD 504, or a means for the function. is there. Further, the content management server 6 has a storage unit 6000 constructed by the RAM 503 and the HD 504 shown in FIG.

(Each function configuration)
Next, each functional configuration of the content management server 6 will be described in detail. The transmission / reception unit 61 transmits / receives various data (or information) to / from another terminal, device, server, or the like via the communication network 9. The schedule interlocking unit 62 acquires schedule information including bibliographic information of a meeting in which a user participates from a schedule management server 8 connected to a communication network 9 so as to be able to send and receive various data (or information). The schedule management server 8 stores schedule information (meeting (list) information) for each user (for each user ID).

The image processing unit 63 has a function as a capture unit, and captures the captured image by performing screen capture of the shared screen ss described later. The creation unit 64 creates a unique content ID, personal memo ID, shared memo ID, and the like. The determination unit 65 determines whether or not the content ID, personal memo ID, shared memo ID, and the like have been received by the transmission / reception unit 61.

The web page creation unit 66 creates web page data to be displayed on the web browsers of the personal terminal 2 and the permanent terminal 4. The search unit 67 receives a search request from the personal portal screen described later displayed on the web browsers of the personal terminal 2 and the permanent terminal 4, and performs a search according to the search request. In addition, the authentication unit 68 performs user authentication processing. The authentication unit 68 may be other than the content management server 6, and may use, for example, an authentication server connected to the communication network 9. The device determination unit 71 determines a device that captures the projection screen according to the situation.

The storage / reading processing unit 69 is executed by commands from the CPU 501, the HDD controller 505, the media I / F516, and the DVD-RW drive 514, and stores various data in the storage unit 6000, the DVD-RW513, and the recording media 515. It also performs processing such as reading various data from the storage unit 6000, the DVD-RW 513, and the recording medium 515.

Further, in the storage unit 6000 of the content management server 6, a personal memo DB 6001, a shared memo DB 6002, an interest level management DB 6003, a personal memo management DB 6004, and a shared memo management DB 6005 are constructed.

Note that these data may be stored in another server other than the content management server 6. In that case, the timing of data acquisition / transmission may be acquired from another server each time the personal terminal 2 requests data acquisition / transmission, or the user can refer to the meeting, the personal board, or the shared screen ss. During this period, the data may be stored in the content management server 6, deleted from the content management server 6 after the end of the conference or reference (or after a certain period of time), and sent to another server.

The device group described in the present embodiment is merely one of a plurality of computing environments for implementing the embodiment disclosed in the present specification. In certain embodiments, the content management server 6 includes a plurality of computing devices, such as a server cluster. The plurality of computing devices are configured to communicate with each other via any type of communication link, including networks, shared memory, and the like, and perform the processes disclosed herein. Similarly, the personal terminal 2 and the permanent terminal 4 can include a plurality of computing devices configured to communicate with each other.

Further, the content management server 6, the personal terminal 2, and the permanent terminal 4 can be configured to share the disclosed processing steps, such as FIG. 18, in various combinations. For example, a part of the process executed by the content management server 6 can be executed by the personal terminal 2 and the permanent terminal 4. Further, each element of the content management server 6, the personal terminal 2, and the permanent terminal 4 may be integrated into one device, or may be divided into a plurality of devices.

<Display example of personal board screen>
A display example of the personal board screen 1000 until the end of the conference will be described with reference to FIGS. 9 to 11. The personal board screen 1000 is a screen for displaying information to be presented to the user by GUI (Graphical User Interface) and accepting operations from the user, and is a display form of a web browser or application software. ..

As shown in FIGS. 9 to 11, the personal board screen 1000 until the end of the conference has a projection area for displaying the projection screen on the left side and a memo area on the right side. The shared screen ss is displayed as a projection screen in the projection area. In the memo area, each combination of the captured image 1022 of the projection screen and the text memo area 1024 accompanying the captured image 1022 is displayed on the sheet 1020.

By pressing the capture button 1016, the user takes a picture of the projection screen displayed in the projection area, and puts the sheet 1020 on which the set of the captured image 1022 and the text memo area 1024 is displayed in the memo area. Can be additionally displayed in. The operation of pressing the capture button 1016 is an example, and may be, for example, an operation of pressing a shortcut key from the keyboard or a gesture operation from the touch panel.

FIG. 9 is an image diagram of an example of the personal board screen 1000 before the projection screen is displayed and the first screen capture is performed. For example, on the personal board screen 1000 of FIG. 9, a guidance message "display the projection screen here" is displayed in the projection area. Further, in the memo area of FIG. 9, a guidance message "The captured image is displayed" is displayed as the captured image 1022. By displaying such a guidance message, on the personal board screen of FIG. 9, the user can imagine the screen image after the screen capture is performed before the first screen capture is performed. The guidance message may not be displayed. Further, even before the first screen capture is performed, the input from the user to the text memo area 1024 may be accepted.

When content data such as stream data is transmitted to the shared screen ss, the personal board screen 1000 of FIG. 9 becomes the personal board screen 1000 as shown in FIG. FIG. 10 is an image diagram of an example of the personal board screen 1000 on which the projection screen 1040 is displayed. In the projection area of FIG. 10, a screen of contents such as stream data transmitted to the shared screen ss is displayed as a projection screen 1040.

When the personal board screen 1000 of FIG. 10 receives an operation of pressing the capture button 1016 from the user, the personal board screen 1000 of FIG. 11 becomes the personal board screen 1000 of FIG. FIG. 11 is an image diagram of an example of the personal board screen 1000 in which the first screen capture was performed.

By pressing the capture button 1016, the user can take a picture of the projection screen 1040 at that time and display the captured image 1022 of the projection screen 1040 in the memo area. In addition, the user can display the text memo area 1024 attached to the captured image 1022 in the memo area. The captured image 1022 and the text memo area 1024 attached to the captured image 1022 are displayed on, for example, one sheet 1020, so that the combination of the captured image 1022 and the text memo area 1024 is displayed in an easy-to-understand manner. .. When the user accepts the operation of pressing the capture button 1016, the projection screen 1040 at that time is compared with the captured image 1022 of the captured projection screen 1040 displayed in the memo area, and the same content is projected. The screen 1040 may not be captured.

In addition, the mouse cursor is set to the first line of the newly displayed text memo area 1024 by pressing the capture button 1016 of the user. Therefore, the user can easily shift from the operation of pressing the capture button 1016 to the text memo operation to the text memo area 1024. The text memo area 1024 extends downward finitely or infinitely according to the input of the text memo by the user.

An object can also be drawn on the captured image 1022 using a pen tool or the like. On the personal board screen 1000, a tool palette displaying a hand tool button 1002, a pen tool button 1004, a text tool button 1006, an Endo button 1008, a Redo button 1010, an HTML save button 1012, a PDF save button 1014, and a capture button 1016 is displayed. Has been done.

The hand tool button 1002 is a button for the user to start using the hand tool. By using the hand tool, the user can select the object drawn on the captured image 1022 and move it by dragging and dropping. The pen tool button 1004 is a button for the user to start using the pen tool. By using the pen tool, the user can select a color and a line thickness and draw an object on the captured image 1022.

The text tool button 1006 is a button for the user to start using the text tool. By using the text tool, the user can generate a text area on the captured image 1022 and input text. The Undo button 1008 is a button for undoing the work done. The Redo button 1010 is a button for advancing the work returned by the Endo button 1008.

The HTML save button 1012 is a button for locally saving the information of the personal board screen 1000 as an HTML file. The PDF save button 1014 is a button for locally saving the captured image 1022 and the text memo area 1024 displayed in the memo area of the personal board screen 1000 as a PDF file. The capture button 1016 is a button for photographing the projection screen 1040 displayed in the projection area and additionally displaying the sheet 1020 in which the set of the captured image 1022 and the text memo area 1024 is displayed in the memo area.

<DB configuration>
<< Personal memo management DB >>
FIG. 12 is a configuration diagram of an example of the personal memo management DB 6004. A personal memo management DB 6004 as shown in FIG. 12 is constructed in the storage unit 6000 of the content management server 6. The personal memo management DB 6004 of FIG. 12 stores and manages the personal memo ID, the user ID, the room ID, the sheet ID, and the captured image in association with each other.

The item "personal memo ID" is an example of personal memo identification information that identifies the personal memo dm of the personal board dc. The item "user ID" is an example of user identification information that identifies a user. The item "room ID" is an example of room identification information that identifies a room (even if the conference is in the same conference room, the ID of the conference is different for each conference). The item "sheet ID" is an example of sheet identification information that identifies the sheets 1020 and 1120. The item “captured image” is an example of image file identification information that identifies an image file in which the projection screen 1040 is captured.

If the user ID of the user who operates the personal terminal 2 is specified by the personal memo management DB 6004 of FIG. 12, the room ID and personal memo ID of the room in which the user participates can be specified. Further, the personal memo management DB 6004 of FIG. 12 can identify, for example, the sheet 1020 displayed on the personal board screen 1000 and the image file of the captured image 1022 displayed on the sheet 1020, which are specified from the personal memo ID.

<< Shared memo management DB >>
FIG. 13 is a configuration diagram of an example of the shared memo management DB 6005. A shared memo management DB 6005 as shown in FIG. 13 is constructed in the storage unit 6000 of the content management server 6. The shared memo management DB 6005 of FIG. 13 manages the shared memo ID, the room ID, and the bibliographic information of the meeting in association with each other.

The item "shared memo ID" is an example of shared memo identification information for specifying the shared memo cm on the shared screen ss. The item "room ID" is an example of room identification information that identifies a room. The item "meeting bibliographic information" is the bibliographic information of the meeting held in the room specified by the room ID. If the room ID is specified by the shared memo management DB 6005 of FIG. 13, the bibliographic information of the meeting can be specified.

<< Personal memo DB >>
FIG. 14 is a configuration diagram of an example of the personal memo DB 2001a. A personal memo DB 2001a as shown in FIG. 14 is constructed in the storage unit 2000a of the personal terminal 2a. Since the personal memo DB 2001a is created in the cache of the web browser, it exists only while the web browser is running.

The data managed by the personal memo DB 2001a is the same as the data for each personal terminal 2 managed by the personal memo DB 6001 in the content management server 6. The personal terminal 2a acquires the data for the personal terminal 2a out of the data of each personal terminal 2 managed by the content management server 6 and manages it in the personal memo DB 2001a.

The personal memo DB 2001a of FIG. 14 manages the personal memo ID, the sheet ID, the content ID, the content data, and the display position in association with each other. The item "personal memo ID" is an example of personal memo identification information that identifies the personal memo dm of the personal board dc. The item "sheet ID" is an example of sheet identification information that identifies the sheets 1020 and 1120. The item "content ID" is an example of content identification information that identifies each content such as a text memo or a drawn object input to the sheets 1020 and 1120.

The item "content data" is data such as text memos and drawn objects input to the sheets 1020 and 1120. For example, the content data of the content ID "c101" is a "text memo" whose type is input to the text memo area 1024 or the like, a font type is "Mincho", a size is "20", and the input text. An example is shown in which the character is "aiueo".

Further, the content data of the content ID "c103" is vector data whose type is drawn on the captured image 1022 or the like, and shows an example in which the vector data is represented by numerical data. In the case of text input to the captured image 1022 or the like by using the text tool, for example, by expressing the type of content data by "text" or the like, the text input to the captured image 1022 or the like and the text memo area 1024 It can be distinguished from the text memo entered in.

The item "display position" indicates the display position of the text memo or the drawn object input to the sheets 1020 and 1120. For example, the display position of the content input to the captured image 1022 or the like is represented by, for example, coordinates. Further, the display position of the text memo input in the text memo area 1024 or the like is represented by, for example, the number of lines and the number of characters.

Since the personal memo DB 6001 has the same data structure as the personal memo DB 2001a, the description thereof will be omitted. However, the personal memo DB 6001 has all the data of the personal memo DB 2001a, 2001b, 2001c.

<< Shared memo DB >>
FIG. 15 is a configuration diagram of an example of the shared memo DB 2002a. A shared memo DB 2002a as shown in FIG. 15 is constructed in the storage unit 2000a of the personal terminal 2a. Since the shared memo DB 2002a is created in the cache of the web browser, it exists only while the web browser is running. Since the shared memo DB 6002 has the same data structure as the shared memo DB 2002a, the description thereof will be omitted.

The data managed by the shared memo DB 2002a is the same as the data managed by the shared memo DB 6002 in the content management server 6. The personal terminal 2a acquires the data managed by the shared memo DB 6002 of the content management server 6 and manages it by the shared memo DB 2002a.

In the shared memo DB 2002a, a set of the content ID, the content data, and the display position of the content data is managed in association with the shared memo ID for specifying the shared memo on the shared screen ss.

The item "content ID" is an example of content identification information for identifying each content. The item "content data" is content data such as text input to the shared screen ss, pasted images, and vector data obtained by drawing. The item "display position" indicates the display position of the content on the shared screen ss.

<< Interest level management DB >>
FIG. 16 is a configuration diagram of an example of interest level management DB 6003. The interest level management DB 6003 as shown in FIG. 16 is constructed in the storage unit 6000 of the content management server 6. The interest level management DB 6003 of FIG. 16 manages DLs in association with each other by personal memo ID, number of captures of streaming, number of capture references, number of writes, and PDF.

The item "personal memo ID" is an example of personal memo identification information that identifies the personal memo dm of the personal board dc. The item "number of captures of streaming" is the number of times the user has taken the projection screen 1040 on the personal board screen 1000 of the room specified by the personal memo ID.

The item "capture reference count" is an example of the reference count of the total number of captures and the reference for each capture as an example of the reference count of the user referring to the sheet 1120 on the personal board screen 1100 of the room specified by the personal memo ID after the meeting. It represents the number of times and the reference time.

The number of references and the reference time for each capture are the number of times and the date and time for each sheet 1120 in which the user referred to the sheet 1120 on the personal board screen 1100 of the room specified by the personal memo ID. The number of references to the total number of captures is the total number of times for each sheet 1120 that the user referred to the sheet 1120.

The item "number of writes" is the number of writes written by the user on the sheet 1020 or 1120 on the personal board screen 1100 of the room specified by the personal memo ID. In the interest level management DB 6003 of FIG. 16, as an example of the number of writes, the total number of text characters for each personal memo, the number of text characters for each capture in the personal memo, the number of handwritten objects (lines, stamps, etc.), and the capture in the personal memo. It represents the number of handwritten objects for each and the number of handwritten characters for the capture.

The total number of text characters for each personal memo is the total number of text characters obtained by adding the number of text characters for each text memo area 1024 such as the sheet 1020. The number of text characters for each capture in the personal memo is the number of text characters for each text memo area 1024 such as the sheet 1020.

The number of handwritten objects (lines, stamps, etc.) is the total number of objects obtained by adding the number of handwritten objects for each captured image 1022 such as the sheet 1020. The number of handwritten objects for each capture in the personal memo is the number of handwritten objects for each captured image 1022 such as the sheet 1020. The number of handwritten characters for the capture is the total number of characters obtained by adding the number of handwritten characters for each captured image 1022 such as the sheet 1020.

The item "DL in PDF" indicates whether or not the captured image 1022 and the text memo area 1024 displayed in the memo area of the personal board screen 1000 are locally saved (downloaded) as a PDF file by the above-mentioned PDF save button 1014. Represents.

<Processing or operation>
Subsequently, the processing or operation of the present embodiment will be described. In the present embodiment, the presenter, who is an example of a user who operates the personal terminal 2a, performs streaming distribution to the shared screen ss at a conference held by the user in the room, and is an example of a user who operates the personal terminal 2b. An example of a participant attending a conference will be described.

FIG. 17 is a flowchart of an example showing the processing of the information processing system according to the present embodiment. In step S10, the information processing system prepares for the meeting. In the conference preparation, the information processing system prepares the room based on the request from the personal terminal 2 or the permanent terminal 4 by the presenter or the organizer of the conference, and connects the personal terminal 2a and the personal terminal 2b to the room. .. The personal board screen 1000 as shown in FIG. 9 is displayed on the personal terminal 2a and the personal terminal 2b connected to the room.

In step S12, a meeting is held in the information processing system. In response to the request from the presenter's personal terminal 2, the information processing system performs streaming distribution to the shared screen ss of the room, and causes each personal terminal 2 to display the projection screen 1040 as shown in the personal board screen 1000 of FIG. .. Participants perform an operation of pressing the capture button 1016 displayed on the personal board screen 1000 at the timing when they want to take a picture while referring to the projection screen 1040 displayed on the personal board screen 1000.

The personal board screen 1000 that has received the operation of pressing the capture button 1016 by the participant captures the captured image 1022 of the projection screen 1040 at that time. Then, for example, like the memo area of the personal board screen 1000 shown in FIG. 11, the captured image 1022 and the text memo area 1024 attached to the captured image 1022 are displayed on one sheet 1020.

In this way, the participant can additionally display the captured image 1022 of the projection screen 1040 and the text memo area 1024 attached to the captured image 1022 in the memo area at an arbitrary timing. Participants enter memos such as inputting text memos in the text memo area 1024 displayed in the memo area as shown in FIG. 11 and drawing objects (inputting handwritten memos) in the captured image 1022 displayed in the memo area. I do. The contents of the various DBs described above are updated according to the contents of the memo entry.

In step S14, in the information processing system, the personal memo dm in which the participant wrote a memo during the meeting can be viewed (review of the memo), and the memo of the meeting can be reviewed. The information on the review of the memo of the meeting by the participants is transmitted to the content management server 6 for updating the interest level management DB 6003.

Further, in step S16, the information processing system displays and utilizes the degree of interest of the participants in the contents of the meeting based on the method of taking the personal memo dm by the participants and the review of the personal memo dm as described later. To do.

For example, the display of the degree of interest of participants in the content of the conference may be viewed not only by the presenter and the organizer but also by the participants by abstracting the content. In addition, the display of the degree of interest of the participants in the content of the conference may be viewed only by the presenter or the organizer by restricting access. Presenters and organizers can view the degree of interest of participants in the content of the conference and utilize it for approaching participants (sales, etc.) and feedback to the next conference as described later.

According to steps S12 and S14, the operability and usability of the user can be improved by the UI that makes it easy to take the personal memo dm. Further, according to step S16, by visualizing and providing the degree of interest of the participants in the content of the meeting, it is possible to promote the utilization for the approach to the participants (sales, etc.) and the feedback to the next meeting.

FIG. 18 is a sequence diagram of an example of the process from the preparation for the meeting to the display of the projection screen 1040 on the personal board screen 1000. Steps S20 to S24 are steps executed at the end of the meeting (when leaving the room). In step S20, the permanent terminal 4 automatically makes a room creation request to the content management server 6 when leaving the previous meeting. .. The content management server 6 creates a room and responds to the permanent terminal 4 with the room information (including the access destination) of the created room. In step S24, the permanent terminal 4 displays the access destination of the room included in the response from the content management server 6 by a URL, a two-dimensional code, or the like. In addition, when the participants who participate in the conference are registered in advance in the content management server 6 and the address for connecting each personal terminal 2 to the room is transmitted from the content management server 6, the participants can connect to the room in advance. When the address is recognized, the permanent terminal 4 is not necessarily an indispensable configuration. When a user who participates in a conference wants to display a shared screen on a large screen in a configuration without a permanent terminal 4, the personal terminal 2 sharing the screen can output to a projector, a display, an electronic whiteboard, or the like.

In step S26, the presenter who operates the personal terminal 2a inputs the access destination of the room on which the permanent terminal 4 is displayed into the web browser. Proceeding to step S28, the personal terminal 2a accesses the access destination input to the web browser, transmits the room information, and makes a personal board creation request and a WebSocket communication establishment request. WebSocket communication is a communication method different from HTTP for performing two-way communication (socket communication) between a web server and a web browser. According to the WebSocket communication, a TCP connection is established between the content management server 6 and the personal terminal 2 while the page (here, the personal board) to be the target of the WebSocket communication is displayed, and the content management server 6 and the individual are displayed. Both web browsers of the terminal 2 continue to communicate at the timing. In other words, when accessing the personal board, it communicates by HTTP including the handshake, after opening the personal board, it switches to WebSocket communication and performs two-way communication, and when the personal board is closed, WebSocket communication of the page ends. ..

Proceeding to step S30, the content management server 6 transmits the personal board screen data and the room ID to the personal terminal 2a, and approves the establishment of WebSocket communication. Proceeding to step S32, the personal terminal 2a responds to the establishment approval of the WebSocket communication in step S30. In steps S28 to S30, a handshake by the HTTP protocol is performed between the personal terminal 2a and the content management server 6, and while the personal board screen 1000 is displayed, bidirectional communication can be performed by WebSocket communication.

In step S34, the participant who operates the personal terminal 2b inputs the access destination of the room displayed by the permanent terminal 4 into the web browser. Proceeding to step S36, the personal terminal 2b accesses the access destination input to the web browser, transmits the room information, and makes a personal board creation request and a WebSocket communication establishment request.

Proceeding to step S38, the content management server 6 transmits the personal board screen data and the room ID to the personal terminal 2b, and approves the establishment of WebSocket communication. Proceeding to step S40, the personal terminal 2b responds to the establishment approval of the WebSocket communication in step S38. In steps S36 to S38, a handshake by the HTTP protocol is performed between the personal terminal 2b and the content management server 6, and while the personal board screen 1000 is displayed, bidirectional communication can be performed by WebSocket communication.

Proceeding to step S42, the presenter who operates the personal terminal 2a selects a target to be screen-transmitted from the screen 1200 as shown in FIG. 19, for example, to the shared screen ss. FIG. 19 is an image diagram of an example of a screen for selecting a target to be screen-transmitted to the shared screen ss. The screen of FIG. 19 shows an example of targeting the target to be transmitted to the shared screen ss from "Share full screen", "Share application window", and "Share browser tab".

The screen 1200 in FIG. 19 is an example in which the presenter selects "share the entire screen". On the screen 1200, an example is shown in which the "screen 1" of the entire desktop or another screen "screen 2" displayed on the dual display is an option. In addition, when "Share application window" is selected on the screen 1200, a plurality of running applications (including an application for displaying a conference / presentation material file) are displayed as options. In addition, when "Share browser tab" is selected on the screen 1200, the tab of the running web browser is displayed as an option.

Proceeding to step S44, the personal terminal 2a specifies the room ID or the personal board ID, and transmits the streaming of the target to be screen-transmitted to the shared screen ss of the specific room by WebRTC. WebRTC is a standard that realizes high-speed data communication via a web browser, and is one of HTML APIs. WebRTC can send and receive large-capacity data such as video and audio in real time.

Proceeding to step S46, the content management server 6 performs streaming distribution by WebRTC to the personal terminal 2a, the personal terminal 2b, and the personal board screen 1000 of the permanent terminal 4 associated with the room ID specified in step S44.

In step S48, the personal terminal 2a can display the streamed projection screen 1040 in the projection area of the personal board screen 1000 displayed by the web browser, for example, as shown in FIG. Further, in step S50, the personal terminal 2b can display the streamed projection screen 1040 in the projection area of the personal board screen 1000 displayed by the web browser, for example, as shown in FIG. Further, in step S52, the permanent terminal 4 can display the streamed projection screen 1040 in the projection area of the personal board screen 1000 displayed by the web browser, for example, as shown in FIG.

For example, a participant who operates the personal terminal 2b can capture the captured image 1022 of the projection screen 1040 and make a memo to the captured image 1022 and the text memo area 1024 by the procedure shown in the sequence diagram of FIG.

<About capture>
Next, a method of switching the capture device will be described. In the present embodiment, for example, the content management server 6 determines to which network the personal terminal 2 requested to capture is connected based on the IP address of the personal terminal 2, and captures according to the connected network. Switch devices.

FIG. 20 is an example of a subnet mask setting screen of the content management server. The subnet mask is a numerical value for identifying the network address and the host address among the IP addresses. An administrator or the like sets a subnet mask in advance from the permanent terminal 4 or the administrator terminal. In addition, the network address assigned to the company having the conference room X is also set. When the content management server 6 determines the network to which the personal terminal 2 is connected, the IP address of the company is a global IP address.

FIG. 20A is an example of a subnet mask, and FIG. 20B is an example of an IP address of a personal terminal 2. The device determination unit 71 can extract the IP address (10.60.100.0) of the 255 part by the subnet mask by the logical product of the subnet mask and the IP address. This is called a network address. Since the network address is assigned to an organization such as a company, it can be seen that the personal terminal 2 including the pre-registered network address in the IP address is connected to the second network. Since the personal terminal 2 connected to the second network can use a high-speed LAN (either wired or Wi-Fi), there is little disadvantage in the personal terminal 2 transmitting the captured image to the content management server 6. ..

In this way, the device determination unit 71 calculates the logical product of the IP address of the personal terminal 2 requested to capture and the subnet mask to obtain the network address, and in the case of the network address of a company or the like in which the network address is registered in advance. , Determine the device that captures the personal terminal 2 that requested the capture. If not, for example, the device that captures the content management server 6 is determined.

FIG. 21 is an example of a flowchart illustrating a method of determining a device to be captured by the device determination unit 71 of the content management server. The process of FIG. 21 starts when a capture request is made.

First, the device determination unit 71 determines the type of network to which the personal terminal 2 requested to capture is connected (S101). As described above, the device determination unit 71 determines whether or not the device is connected to the second network based on the network address. If it is not connected to the second network, it is presumed that it is connected to the telephone communication network 10.

When connected to the second network, the device determination unit 71 determines that the personal terminal 2 that requested the capture captures (S102).

When not connected to the second network (when connected to the telephone communication network 10), the device determination unit 71 determines that the content management server 6 captures (S103).

In this way, by determining the device to be captured by the content management server 6, it is possible to capture with a device having few disadvantages.

In FIG. 21, it is determined that the content management server 6 captures when connected to the telephone communication network 10, but in this case, the permanent terminal 4 or the personal terminal 2a may perform the capture. For example, in a situation where the processing load of the content management server 6 is high, the device that captures the permanent terminal 4 or the personal terminal 2a can be determined. Also in this case, it is not necessary for the personal terminal 2c connected to the telephone communication network 10 to transmit the captured image to the content management server 6.

-Capture by the content management server First, the flow of the process captured by the content management server will be described with reference to FIG. FIG. 22 is a sequence diagram of an example of a process of capturing the captured image 1022 of the projection screen 1040 and making a memo in the captured image 1022 and the text memo area 1024. FIG. 22 describes a case where the content management server 6 captures. Further, it is assumed that the personal terminal 2c is connected to the telephone communication network 10 as shown in FIG.

Participants who operate the personal terminal 2c in step S60 perform an operation of pressing the capture button 1016. The personal terminal 2c makes a capture shooting request to the content management server 6.

In step S61, the device determination unit 71 of the content management server 6 determines the device to be captured. In FIG. 22, the content management server 6 is determined.

In step S62, the image processing unit 63 of the content management server 6 photographs the projection screen 1040 at that time, and returns the image file of the captured captured image 1022 to the personal terminal 2c.

The personal terminal 2c that has received the image file of the captured image 1022 from the content management server 6 proceeds to step S66, and additionally displays the captured image 1022 of the projection screen 1040 in the memo area using the received image file, for example, as shown in FIG. To do. Further, the personal terminal 2c displays the text memo area 1024 attached to the captured image 1022 on the same sheet 1020 as the additionally displayed captured image 1022, as shown in FIG. 11, for example.

Proceeding to step S68, the participant who operates the personal terminal 2c draws an object on the captured image 1022 on the additionally displayed sheet 1020 and inputs a text memo in the text memo area 1024.

In the sequence of FIG. 22, only the image file of the captured image 1022 is transmitted from the content management server 6 to the personal terminal 2c. In this case, JavaScript (registered trademark) included in the data of the web page and operating on the browser of the personal terminal 2c may create the text memo area.

Further, the content management server 6 may transmit the data of the sheet 1020 including the image of the captured image 1022 and the text memo area 1024. Further, the display position of the captured image 1022 and the sheet 1020 at this time may be determined by a program such as a script executed by the browser of the personal terminal 2c, or the content management server specifies the display position and the personal terminal. It may be transmitted to 2c.

Further, a sheet 1020 in which the acquired image and the text memo area are combined may be generated on the server side. The area of the sheet 1020 other than the captured image and the text memo area (the area represented by black in the drawing) can be an area where handwriting can be performed or an area where handwriting cannot be performed.

When a sheet other than the additionally displayed sheet 1020 (selected sheet 1020) is clicked, the personal terminal 2b determines that the editing is temporarily completed, and proceeds to step S70. In step S70, the personal terminal 2b transmits the information regarding the additionally displayed sheet 1020 and the edited contents for the sheet 1020 to the content management server 6. In step S70, the personal terminal 2b may transmit the information about all the sheets 1020 and the edited contents for all the sheets 1020 to the content management server 6.

The content management server 6 updates the contents of the various DBs described above with the received information about the sheet 1020 and the edited contents for the sheet 1020. In this way, the captured image 1022 captured on the personal board screen 1000, the handwritten memo on the captured image 1022, and the text memo input in the text memo area 1024 are associated with each other by the sheet 1020 and saved in the content management server 6. Will be done.

-Capture by the personal terminal 2b Next, the capture process when the personal terminal 2b is determined to be the device to capture will be described with reference to FIG. 23. FIG. 23 is an example of a sequence diagram illustrating a procedure for capturing by the personal terminal 2b. According to FIG. 2, the personal terminal 2b is connected to the in-house network (second network in FIG. 1).

First, when the user B presses the capture button 1016 on the personal terminal 2b, the reception unit 22b accepts the screen capture process by the web browser (web application) on the personal terminal 2b (S111). As a result, the reception unit 22b requests the communication management unit 30b for screen capture (S112). This request includes the URL of the shared screen ss. As a result, the communication management unit 30b receives the screen capture request.

The communication management unit 30b transmits a capture shooting request to the content management server 6 (S112-2). The device determination unit 71 of the content management server 6 determines that the personal terminal 2b is connected to the second network, and determines the device to capture the personal terminal 2b that has transmitted the capture request. The content management server 6 transmits a shooting instruction to the personal terminal 2b.

Next, the communication management unit 30b performs a screen capture process (S113). Subsequently, the communication management unit 30b outputs the data of the content of the screen capture image to the transmission / reception unit 21b realized by the web browser (S117). As a result, the transmission / reception unit 21b inputs the data of the content of the screen capture image.

Then, the transmission / reception unit 21b transmits the data of the content of the screen capture image to the content management server 6 (S118). As a result, the transmission / reception unit 61 of the content management server 6 receives the data of the content of the screen capture image.

Next, the content management server 6 performs storage processing of content data (S120).

The transmission / reception unit 61 transmits an update notification, which is a notification to update the personal board, to the personal terminal 2b (S121). When the user B has a plurality of personal terminals and each of them accesses the same personal board (URL), the content management server 6 notifies the plurality of personal terminals of the user B of the update. To send. As a result, the transmission / reception unit 21b receives the update notification in the web browser of the personal terminal 2b.

Next, in the web browser of the personal terminal 2b, the transmission / reception unit 21b transmits a request for updated content data and display position information to the content management server 6 (S122). As a result, the transmission / reception unit 61 of the content management server 6 receives the update notification.

Next, in the content management server 6, the storage / reading processing unit 69 reads the content ID 11, the content data, and the display position information p1 related to the updated content from the personal memo DB 6001 (S123). Then, the transmission / reception unit 61 transmits the content ID 11, the content data, and the display position information p1 to the personal terminal 2b (S124). As a result, in the web browser of the personal terminal 2b, the transmission / reception unit 21b receives the content ID 11, the content data, and the display position information p1.

Next, in the personal terminal 2b, the storage / reading processing unit 29b stores the content ID 11, the content data, and the display position information p1 received in step S124 in association with the personal memo management DB 2001b (S125). ). As a result, in the web browser of the personal terminal 2b, the display control unit 24b updates the web page so as to display the content which is the captured image (entire screen) (S126). From the above, the user B can capture the shared screen as a personal image.

-Capture by the permanent terminal 4 Next, the capture process when the permanent terminal 4 is determined to be the device to capture will be described with reference to FIG. 24. FIG. 24 is a sequence diagram of an example of a process of capturing the captured image 1022 of the projection screen 1040 and making a memo in the captured image 1022 and the text memo area 1024. FIG. 24 describes a case where the permanent terminal 4 captures.

Participants who operate the personal terminal 2c in step S160 perform an operation of pressing the capture button 1016. The personal terminal 2c makes a capture shooting request to the content management server 6.

In step S161, the device determination unit 71 of the content management server 6 determines the device to be captured. In FIG. 24, for example, since the processing load of the content management server 6 is high, it is assumed that the permanent terminal 4 is selected.

In step S162, the content management server 6 transmits a shooting instruction to the permanent terminal 4. In step S164, the permanent terminal 4 receives a shooting instruction, and the permanent terminal 4 shoots the projection screen 1040 at that time.

In step S166, the permanent terminal 4 returns the image file of the captured image of the captured captured image 1022 to the content management server 6.

In step S168, the content management server 6 transmits the image file of the captured image to the personal terminal 2c that requested the capture.

In step S170, the personal terminal 2c that has received the image file of the captured image 1022 from the content management server 6 additionally displays the captured image 1022 of the projection screen 1040 in the memo area using the received image file, for example, as shown in FIG. To do. Further, the personal terminal 2c displays the text memo area 1024 attached to the captured image 1022 on the same sheet 1020 as the additionally displayed captured image 1022, as shown in FIG. 11, for example.

Proceeding to step S172, the participant who operates the personal terminal 2c draws an object on the captured image 1022 on the additionally displayed sheet 1020 and inputs a text memo to the text memo area 1024.

When a sheet other than the additionally displayed sheet 1020 (selected sheet 1020) is clicked, the personal terminal 2b determines that the editing is temporarily completed, and proceeds to step S174. In step S174, the personal terminal 2c transmits the information regarding the additionally displayed sheet 1020 and the edited contents for the sheet 1020 to the content management server 6.

Although an example of capturing by the permanent terminal 4 has been described with reference to FIG. 24, the processing flow when the personal terminal 2a (transmitting terminal) captures is also the same.

Further, in the present embodiment, the content management server 6 determines the device to be captured, but the device to be captured by the personal terminal 2 that has received the capture operation may be determined. In this case, if the IP address of the own machine is a local IP address, it is determined that the capture is performed by the own machine because it is connected to the second network. If not, it is presumed that the user is connected to the telephone communication network 10, and the device that captures the content management server 6 is determined. When the device for capturing the content management server 6 is determined, the device for capturing may be determined by the content management server 6. The local IP address is in the range of 10.0.0.0 to 1.255.255.255, 172.16.0.0 to 172.31.255.255, and 192.168.0.0 to 192.168.255.255.

Further, in the present embodiment, if it is not the second network, it is determined that it is a telephone communication network, but if it is not the second network, whether or not the personal terminal 2c is further connected to Wi-Fi is determined by the content management server 6. May judge. Since the communication by Wi-Fi is relatively high speed and the communication fee is low, the content management server 6 can be handled in the same manner as the second network. Therefore, when the personal terminal 2c is connected to Wi-Fi, the content management server 6 determines the device that captures the personal terminal 2c. Whether or not the personal terminal 2c is connected to Wi-Fi may be determined by the personal terminal 2c by transmitting that fact to the content management server 6.

<Summary>
As described above, the information processing system 100 of the present embodiment can switch the device for capturing depending on the situation. For example, when the personal terminals 2a and 2b connected to the second network request capture, the content management server 6 causes the personal terminals 2a and 2b to capture. When the personal terminal 2c connected to the telephone communication network requests capture, the content management server 6 captures itself. Therefore, the device for appropriately capturing can be determined according to the situation of the personal terminal 2 requesting capture, and the demerit can be suppressed.

In this embodiment, the information processing system 100 in which each user can specify the device to be captured will be described.

In this embodiment, the hardware configuration diagrams of FIGS. 4 to 7 and the functional block diagram shown in FIG. 8 described in the first embodiment will be described as being applicable.

<Example of setting screen>
FIG. 25 is an example of the setting screen 300 of the capture device. The setting screen 300 can be displayed when each user displays a personal portal on the personal terminal 2 and performs a predetermined operation. The setting screen 300 has buttons for "permanent terminal" 301, "transmission terminal" 302, "server" 303, and "own terminal" 304, and a device desired by the user as a device for capturing from these buttons can be selected. select. The "permanent terminal" 301, the "transmitting terminal" 302, the "server" 303, and the "own terminal" 304 correspond to A to D in FIG. 1, respectively.

Since the personal terminal 2 transmits the device for capturing in response to the pressed button to the content management server 6, the content management server 6 stores the device for capturing in association with the user ID.

The setting screen 300 of FIG. 25 may be displayed at the timing when the user performs a predetermined operation after the user logs in, or may be displayed at the timing when the user performs a capture operation.

FIG. 26 is an example of a sequence diagram showing a procedure in which each personal terminal 2 transmits a capture device to the content management server 6.

In steps S180 and S182, since the users A and C have selected the devices to be captured on the setting screen 300, the personal terminals 2a and 2c receive the devices to be captured, respectively.

In steps S184 and S186, the personal terminals 2a and 2c transmit the capture device to the content management server 6, respectively.

When the users A and C perform the capture operation, in steps S188 and S190, the personal terminals 2a and 2c transmit the capture shooting request to the content management server 6, respectively.

In steps S192 and S194, the content management server 6 determines the device to be captured based on the device to be captured registered in the content management server 6 by the users A and C.

Although the device for capturing is determined by the content management server 6 in FIG. 26, the personal terminal 2 may determine the device. In this case, the personal terminal 2 holds the capture device set on the setting screen, and when the users A and C perform the capture operation, the instruction of the capture device is transmitted to the content management server 6. As a matter of course, when the device for capturing is set in the own machine, it is not necessary to transmit the device for capturing to the content management server 6.

<Summary>
According to this embodiment, the user can determine the situation of the personal terminal 2 and set the device for capturing.

In this embodiment, the information processing system 100 that changes the capturing device according to the comparison result between the number of personal terminals 2 connected to the content management server 6 and the threshold value will be described.

<Functional configuration>
FIG. 27 is a functional block diagram of each terminal and server constituting the information processing system 100. In addition, in this embodiment, since the components having the same reference numerals in FIG. 8 perform the same function, only the main components of this embodiment may be mainly described.

In this embodiment, the session management DB 6006 is constructed in the storage unit 6000. The session management DB 6006 is a database in which users participating in each room are registered.

FIG. 28 schematically shows the information registered in the session management DB 6006. A user ID is associated with the room ID in the session management DB 6006. Therefore, the content management server 6 can grasp the number of personal terminals 2 connected to each room. The number of personal terminals 2 can be said to be the number of users and the number of sessions.

FIG. 29 is a flowchart showing a procedure in which the device determination unit 71 of the content management server 6 determines the device to be captured. The process of FIG. 29 starts when, for example, a capture shooting request is transmitted from the personal terminal 2 to the content management server 6.

The device determination unit 71 determines whether or not the number of terminals participating in the conference is greater than the threshold value (S201). As shown in FIG. 30, this threshold value can be set by the user.

If the number of terminals participating in the conference is larger than the threshold value, the load on the content management server 6 may increase. Therefore, the device determination unit 71 determines that the personal terminal 2 that requested the capture captures the content (S202).

When the number of terminals participating in the conference is equal to or less than the threshold value, the load on the content management server 6 is not large, and the device determination unit 71 determines that the content management server 6 captures (S203).

FIG. 30 is an example of the setting screen 310 for the number of terminals serving as a threshold value. The setting screen 310 can be displayed when each user displays a personal portal on the personal terminal 2 and performs a predetermined operation. The terminal number setting screen 310 has a number input field 311. Since the personal terminal 2 transmits the numerical value set by the user to the content management server 6, the content management server 6 stores the threshold value.

In this embodiment, the device to capture is determined based on the number of terminals participating in the conference, but it may be determined based on the bandwidth of the network. The bandwidth of a network is the amount of information that can be exchanged within a unit time. In addition, the bandwidth of the network has a large correlation with the communication speed.

Further, the device for capturing may be determined according to the communication speed between the personal terminal 2 and the content management server 6. In this case, if the communication speed is higher than the threshold value, the content management server 6 determines that each individual terminal 2 captures, and if not, the content management server 6 determines that the capture is performed.

In this embodiment as well, the personal terminal 2 that has received the capture operation may determine the device to capture. In this case, the content management server 6 transmits the number of personal terminals 2 connected to the personal terminal 2 that has received the capture operation.

<Summary>
According to this embodiment, the device for capturing can be switched according to the situation regarding the load of the content management server 6 (the number of connected personal terminals 2).

In this embodiment, an information processing system that switches the device to be captured by the content management server 6 according to the comparison result between the number of captures and the threshold value will be described.

In this embodiment, the hardware configuration diagrams of FIGS. 4 to 7 and the functional block diagram shown in FIG. 8 described in the first embodiment will be described as being applicable.

FIG. 31 is an example of a flowchart showing a procedure in which the content management server 6 determines a device to be captured according to the number of captures. The process of FIG. 31 is repeatedly executed at regular intervals.

The device determination unit 71 determines whether or not the number of captures is larger than the threshold value (S301). As shown in FIG. 12, the personal memo management DB 6004 is associated with the captured image in association with the room ID. Therefore, the device determination unit 71 can count the number of captures of this room at regular intervals.

If the number of captures per fixed time is larger than the threshold value, the load on the content management server 6 may increase. Therefore, the device determination unit 71 determines that the personal terminal 2 that requested the capture captures (S302).

When the number of captures is equal to or less than the threshold value, the device determination unit 71 determines that the content management server 6 captures, assuming that the load on the content management server 6 is not large (S303). The threshold value in FIG. 31 can be set by the user from the same setting screen as in FIG. 30.

Further, also in this embodiment, the personal terminal 2 that has received the capture operation may determine the device to capture. In this case, the content management server 6 transmits the number of captures at regular intervals and the number of personal terminals 2 connected to the personal terminal 2 that has received the capture operation.

<Summary>
According to this embodiment, the device for capturing can be switched according to the situation regarding the load based on the number of captures of the content management server 6.

In this embodiment, the information processing system 100 in which the user can set the determination criteria for determining the device to be captured by the content management server 6 will be described.

In this embodiment, the hardware configuration diagrams of FIGS. 4 to 7 and the functional block diagram shown in FIG. 8 described in the first embodiment will be described as being applicable.

FIG. 32 is an example of a determination criterion setting screen 320 for determining a device to be captured. This setting screen 320 can be displayed when each user displays a personal portal on the personal terminal 2 and performs a predetermined operation. The judgment standard setting screen 320 displays a list of judgment criteria, and in FIG. 32, "the network to which the terminal is connected" 321 and "user setting" 322, "the number of personal terminals" 323, "network band" 324, and " It has buttons for "line speed" 325 and "capture count" 326.

The "network to which the terminal is connected" 321 corresponds to the judgment criteria described in the first embodiment, the "user setting" 322 corresponds to the judgment criteria described in the second embodiment, and the "number of personal terminals" 323, " The "network band" 324 and the "line speed" 325 correspond to the judgment criteria described in the third embodiment, and the "capture count" 326 corresponds to the judgment criteria described in the fourth embodiment. Since the personal terminal 2 transmits the determination criteria selected by the user to the content management server 6, the content management server 6 stores the determination criteria for each user.

Therefore, by setting the judgment criteria for determining the device to be captured on the setting screen 320 as shown in FIG. 32, the content management server 6 sets the device to be captured according to the judgment criteria described in Examples 1 to 4. Can be decided.

Alternatively, as shown in FIG. 33, a determination criterion that the user gives priority to may be set. FIG. 33 is an example of the determination criterion setting screen 330 for determining the device to be captured. This setting screen 330 can be displayed when each user displays a personal portal on the personal terminal 2 and performs a predetermined operation. The determination standard setting screen 330 has buttons for "network band priority" 331 and "display speed priority" 332. Since the personal terminal 2 transmits the determination criteria selected by the user to the content management server 6, the content management server 6 stores the determination criteria in association with the user ID.

“Network band priority” 331 is a setting when the content management server 6 selects a device for capturing based on the network band. Therefore, it corresponds to the judgment criteria described in the third embodiment. The "display speed priority" 332 determines the device to be captured by the content management server 6 so that the time until the personal terminal 2 displays the captured image is shortened. When "display speed priority" 332 is selected, the content management server 6 determines that the personal terminal 2 that requested the capture captures.

Since the personal terminal 2 can also determine the "network band priority" 331 and the "display speed priority" 332, the personal terminal 2 may select the device to capture according to the selection result.

<Summary>
According to this embodiment, the user can set a determination criterion for determining the device to be captured by the content management server 6.

In Examples 1 to 5, when each personal terminal 2 accepts the capture operation, the personal terminal 2 transmits a capture request to the content management server 6 to determine a device for the content management server 6 to capture. When the content management server 6 instructed the personal terminal 2 to take a picture, the personal terminal 2 performed the capture process. Alternatively, the personal terminal 2 can determine the device to be captured, and the personal terminal 2 can capture the device or request the content management server 6 to take a picture. This is because the Web page transmitted from the content management server 6 to the personal terminal 2 includes a JavaScript (registered trademark) process for capturing in response to a request from the content management server 6.

On the other hand, when the JavaScript (registered trademark) transmitted to the personal terminal 2 does not capture by the instruction from the content management server 6, but the content management server 6 performs a capture operation (capture operation is performed on each of the personal terminals 2a to 2c). It is also possible to send the corresponding JavaScript (registered trademark) to the personal terminal 2 when deciding which device captures the data. In this case, since the JavaScript (registered trademark) changes, the personal terminal 2 reloads the Web page (including the JavaScript (registered trademark)) when the capture method is determined.

In this case, when the user performs the capture operation, the personal terminal 2 does not inquire the content management server 6 for the device to be captured, but captures by itself or manages the content based on the already determined capture method. Request capture from server 6. For example, the personal terminals 2a and 2b capture by their own device, and the personal terminal 2c (connected to the telephone communication network 10) requests the content management server 6 to capture. By doing so, the communication between the personal terminal 2 and the content management server 6 can be reduced, and the time until the captured image is displayed can be shortened.

FIG. 34 is an example of a sequence diagram showing a procedure in which the content management server 6 transmits a script to the personal terminal 2 according to the capture method.

In step S401, the device determination unit 71 of the content management server 6 determines the device to be captured. The determination method may be any of Examples 1 to 5.

In steps S403 and S405, the content management server transmits JavaScript (registered trademark) to each personal terminal 2. For example, when the device determination unit 71 determines that the personal terminal 2a is to be captured by the personal terminal 2, the web page creation unit 66 transmits JavaScript (registered trademark) to be captured by the personal terminal 2a to the personal terminal 2a. When the device determination unit 71 determines that the personal terminal 2c is to be captured by the content management server 6, the web page creation unit 66 transmits the JavaScript (registered trademark) requested by the personal terminal 2c to the content management server 6 to the personal terminal 2c. To do.

<Summary>
According to this embodiment, each personal terminal 2 executes an appropriate script on the personal terminal 2, so that the communication between the personal terminal 2 and the content management server 6 can be reduced and the time until the captured image is displayed can be shortened. ..

[Other Embodiments]
Although the embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and substitutions can be added without departing from the gist of the present invention. For example, the information processing system according to this embodiment can be used in the following situations.

In the general seminar, the customer corresponds to the participant of this embodiment, and the sales person corresponds to the presenter or organizer of this embodiment, and the interest of the participant can be visualized. In school use, the student corresponds to the participant of this embodiment, the teacher corresponds to the presenter or organizer of this embodiment, and the degree of concentration of the student in the lesson can be analyzed. In the general meeting, the employee corresponds to the participant of this embodiment, the manager corresponds to the presenter and the organizer of this embodiment, and the manager can grasp the situation of the meeting in the company.

Each function of the embodiment described above can be realized by one or more processing circuits. Here, the "processing circuit" in the present specification is a processor programmed to execute each function by software such as a processor implemented by an electronic circuit, or a processor designed to execute each function described above. It shall include devices such as ASIC (Application Specific Integrated Circuit), DSP (digital signal processor), FPGA (field programmable gate array) and conventional circuit modules.

2a, 2b, 2c Personal terminal 4 Permanent terminal 6 Content management server 8 Schedule management server 9 Communication network 100 Information processing system

Japanese Unexamined Patent Publication No. 2006-146415

Claims (16)

An information processing system including an information processing device that transmits data for displaying a web page to web browsers of a plurality of communication terminals, and a plurality of communication terminals that accept an import operation for the web page displayed by the web browser.
A device determination unit that determines a device that captures a screen displayed on the web page according to the status of the information processing system, and a device determination unit.
In the device determined by the device determination unit, the device has a capture unit that captures at least a part of the screen.
The communication terminal is
An information processing system including a display control unit that displays captured images captured by the capture unit. The device determination unit is a permanent terminal that displays the web page arranged in the local network, a transmission terminal that transmits the screen to be captured to the information processing device, the information processing device, or the communication that receives the capture operation. The information processing system according to claim 1, wherein any of the terminals is determined as a device for capturing. The information according to claim 1 or 2, wherein the device determination unit determines, as the situation, a device to be captured according to the type of network to which the communication terminal that has received the capture operation is connected. Processing system. When the communication terminal that has received the capture operation is connected to the local network, the device determination unit determines the device that captures the communication terminal.
The information processing system according to claim 3, wherein when the communication terminal that has received the capture operation is not connected to the local network, the device determination unit determines the device that captures the information processing device. The user accepts a setting as to which of the permanent terminal, the transmitting terminal, the information processing device, and the communication terminal that has received the import operation is to be the device to capture.
The information processing system according to claim 2, wherein the device determination unit determines a device to be captured according to a user's setting. The information processing system according to claim 1 or 2, wherein the device determination unit determines a device to be captured according to a comparison result of a number of communication terminals connected to the information processing device and a threshold value. The information processing system according to claim 6, wherein a setting screen for accepting the threshold value setting is displayed, and the threshold value setting is accepted. The information processing according to claim 1 or 2, wherein the device determination unit determines a device that captures the screen according to the network band or line speed to which the information processing device or the communication terminal is connected. system. The information processing device stores the captured images captured by the communication terminal accepting the capture operation.
The information processing system according to claim 1 or 2, wherein the device determination unit determines a device to be captured according to a comparison result between the number of captured images per fixed time and a threshold value. The information processing system according to claim 9, wherein a setting screen for accepting the threshold value setting is displayed, and the threshold value setting is accepted. The device determination unit displays a list of judgment criteria for determining the device that captures the screen.
The information processing system according to any one of claims 3 to 10, wherein the device determination unit determines a device that captures the screen based on the determination criteria, although it receives a selection from the user. When the device determination unit determines the device for capturing the screen,
When the communication terminal accepts the import operation, the information processing device transmits a script language for performing processing related to the import to the communication terminal.
When the plurality of communication terminals receive an import operation, they either import by themselves or request the information processing device to import according to the script language transmitted from the information processing device. The information processing system according to any one of claims 1 to 11. An information processing device that accepts an import operation for a web page acquired from a second information processing device that transmits data for displaying a web page to web browsers of a plurality of communication terminals.
A device determination unit that determines a device that captures a screen displayed on the web page according to the status of the information processing device.
In the device determined by the device determination unit, a capture unit that captures at least a part of the screen and a capture unit.
A display control unit that displays the captured image captured by the capture unit, and
An information processing device characterized by having. An information processing device that transmits data that displays a web page on the web browsers of multiple communication terminals.
It has a device determination unit that determines a device for capturing a screen displayed on the web page according to the status of the information processing device when receiving a message that the capture operation has been accepted from the communication terminal.
If the information processing device decides to capture the screen, it captures at least a portion of the screen.
An information processing device characterized in that when the communication terminal determines to capture the screen, the communication terminal is instructed to capture the screen. An image display performed by an information processing system having an information processing device that transmits data for displaying a web page on the web browsers of a plurality of communication terminals and a plurality of communication terminals that accept an import operation for the web page displayed by the web browser. It's a method
A step in which the device determination unit determines a device for capturing a screen displayed on the web page according to the status of the information processing system.
In the device determined by the device determination unit, the step of the capture unit capturing at least a part of the screen and
The step of displaying the captured image captured by the capture unit by the communication terminal, and
An image display method characterized by having. An information processing device that accepts an import operation for the web page acquired from a second information processing device that transmits data for displaying a web page on the web browsers of a plurality of communication terminals.
A device determination unit that determines a device that captures a screen displayed on the web page according to the status of the information processing device.
In the device determined by the device determination unit, a capture unit that captures at least a part of the screen and a capture unit.
A display control unit that displays the captured image captured by the capture unit,
A program to function as.

Images