JP2021077111A - Communication system, information processing method, information processor and program - Google Patents

Abstract

To provide a communication system, an information processing method, an information processing device and a program that can efficiently search for minutes of a target meeting.SOLUTION: A communication system includes a management server that manages text data converted by a conversion server for sound data recorded in a video conference, and an information processor. The management server includes a storage processing unit that associates the text data with a speaker who emitted sound data corresponding to the text data and stores it in a storage unit as at least a part of the minutes. The information processor includes a first transmission unit that transmits at least a keyword and a speaker specified as search conditions according to an input operation received by a reception unit to the management server. In addition, the management server includes a search unit that searches for the minutes including the keyword transmitted from the first transmission unit and corresponding to the speaker transmitted from the first transmission unit, from the minutes stored in the storage unit.SELECTED DRAWING: Figure 8

Description

The present invention relates to communication systems, information processing methods, information processing devices and programs.

A technique is known in which audio data recorded in a video conference is converted into text using a cloud service, automatically generated as minutes, and then stored as minutes. There is a need for a technology that can efficiently search for the desired content of the minutes stored in this way.

As a video conferencing system that can search such minutes, a system that can search the displayed minutes by speaker and keyword is disclosed (for example, Patent Document 1).

However, in the conventional video conferencing system, it is possible to perform a text search of the period, the conference name, the participants, the minutes, etc., and a large amount of search results are obtained, and the searcher finds the minutes of the desired conference. There is a problem that it takes time and effort. Further, the technique described in Patent Document 1 merely makes it possible to search the relevant part of the minutes that have already been opened by the speaker and the keyword.

The present invention has been made in view of the above, and an object of the present invention is to provide a communication system, an information processing method, an information processing device, and a program capable of efficiently searching the minutes of a target meeting. To do.

In order to solve the above-mentioned problems and achieve the object, the present invention has a communication having a management server for managing text data converted by a conversion server for sound data recorded in a video conference, and an information processing device. In the system, the management server is a storage processing unit that associates the text data with the speaker who has emitted the sound data corresponding to the text data and stores the text data in the storage unit as at least a part of the minutes. The information processing apparatus includes a first transmission unit that transmits at least a keyword and a speaker specified as search conditions according to an input operation received by the reception unit to the management server. , A search unit that searches the minutes stored in the storage unit for the minutes that include the keyword transmitted from the first transmission unit and that corresponds to the speaker transmitted from the first transmission unit. It is characterized by being prepared.

According to the present invention, the minutes of a target meeting can be efficiently searched.

FIG. 1 is a schematic view of a shared system according to an embodiment. FIG. 2 is a diagram showing an example of the hardware configuration of the electronic blackboard according to the embodiment. FIG. 3 is a diagram showing an example of the hardware configuration of the video conferencing terminal according to the embodiment. FIG. 4 is a diagram showing an example of the hardware configuration of the car navigation device according to the embodiment. FIG. 5 is a diagram showing an example of the hardware configuration of the PC and the server according to the embodiment. FIG. 6 is a diagram showing an example of the software configuration of the electronic blackboard according to the embodiment. FIG. 7 is a diagram showing an example of the software configuration of the PC according to the embodiment. FIG. 8 is a functional block diagram of the shared system according to the embodiment. FIG. 9 is a conceptual diagram of a user authentication management table, an access management table, and a schedule management table. FIG. 10 is a conceptual diagram showing a content management table and a speaker information management table. FIG. 11 is a conceptual diagram showing a user authentication management table, a user management table, and a union management table. FIG. 12 is a conceptual diagram showing a union reservation management table and an event management table. FIG. 13 is a conceptual diagram showing a server authentication management table and an execution event history management table. FIG. 14 is a conceptual diagram showing an execution event management table and a related information management table. FIG. 15 is a conceptual diagram showing a text information management table. FIG. 16 is a conceptual diagram showing a user voice information management table. FIG. 17 is a sequence diagram showing an example of the flow of the schedule registration process of the shared system according to the embodiment. FIG. 18 is a diagram showing a sign-in screen. FIG. 19 is a diagram showing an example of an initial screen of a PC. FIG. 20 is a diagram showing a schedule input screen. FIG. 21 is a sequence diagram showing an example of the flow of the user voice data registration process of the shared system according to the embodiment. FIG. 22 is a diagram showing a sign-in screen for registering user voice data. FIG. 23 is a diagram showing a user voice data registration screen. FIG. 24 is a sequence diagram showing an example of a flow of event start processing of the shared system according to the embodiment. FIG. 25 is a diagram showing a sign-in screen displayed on the electronic blackboard. FIG. 26 is a diagram showing a reservation list screen of the union. FIG. 27 is a sequence diagram showing an example of a flow of event start processing of the shared system according to the embodiment. FIG. 28 is a diagram showing a detailed information screen of the event. FIG. 29 is a diagram showing a display screen displayed on the electronic blackboard at the start of the event. FIG. 30 is an image diagram showing a state in which the user uses the electronic blackboard. FIG. 31 is a sequence diagram showing an example of a flow of registration processing of the execution event history of the shared system according to the embodiment. FIG. 32 is a flowchart showing an example of the flow of the voice-text conversion process of the shared system according to the embodiment. FIG. 33 is a flowchart showing an example of the flow of the speaker identification process of the shared system according to the embodiment. FIG. 34 is a sequence diagram showing an example of the flow of the voice-text conversion process of the shared system according to the embodiment. FIG. 35 is a diagram showing an example of a data file including text data. FIG. 36 is a sequence diagram showing an example of a flow of registration processing of the execution event history of the shared system according to the embodiment. FIG. 37 is a sequence diagram showing an example of a flow of end processing of an event of the shared system according to the embodiment. FIG. 38 is a sequence diagram showing an example of a flow of end processing of an event of the shared system according to the embodiment. FIG. 39 is a diagram showing an event end screen displayed on the electronic blackboard. FIG. 40 is a diagram showing an upload screen of file data displayed on the electronic blackboard. FIG. 41 is a diagram showing an upload completion screen of file data displayed on the electronic blackboard. FIG. 42 is a sequence diagram showing an example of a flow of browsing processing of the execution event history of the shared system according to the embodiment. FIG. 43 is a sequence diagram showing an example of a flow of browsing processing of the execution event history of the shared system according to the embodiment. FIG. 44 is a diagram showing an execution event list screen displayed on the PC according to the embodiment. FIG. 45 is a diagram showing an execution event history screen displayed on the PC according to the embodiment. FIG. 46 is a diagram showing an execution event history screen displayed on the PC according to the embodiment. FIG. 47 is a sequence diagram showing an example of a flow of selection processing of the execution event history of the shared system according to the embodiment. FIG. 48 is a diagram showing an execution event history screen displayed on the PC according to the embodiment. FIG. 49 is a diagram showing an execution event history screen displayed on the PC according to the embodiment. FIG. 50 is a sequence diagram showing an example of a flow of selection processing of the execution event history of the shared system according to the embodiment. FIG. 51 is a sequence diagram showing an example of a flow of selection processing of the execution event history of the shared system according to the embodiment. FIG. 52 is a sequence diagram showing an example of the flow of the text data editing process of the shared system according to the embodiment. FIG. 53 is a diagram showing an execution event history screen displayed on the PC according to the embodiment. FIG. 54 is a diagram showing a text editing screen displayed on the PC according to the embodiment. FIG. 55 is a conceptual diagram showing an updated text information management table. FIG. 56 is a diagram showing an execution event history screen displayed on the PC according to the embodiment. FIG. 57 is a sequence diagram showing an example of a flow of browsing processing of the editing history of the shared system according to the embodiment. FIG. 58 is a diagram showing an edit history screen displayed on the PC according to the embodiment. FIG. 59 is a sequence diagram showing an example of the flow of the search process of the minutes of the shared system according to the embodiment. FIG. 60 is a diagram showing a minutes search screen displayed on the PC according to the embodiment. FIG. 61 is a diagram showing a minutes search screen including search results displayed on the PC according to the embodiment. FIG. 62 is a diagram showing a minutes search screen including search results displayed on the PC according to the embodiment.

The shared system 1 according to the present embodiment will be described in detail below with reference to the drawings. In the present embodiment, the "file" indicates an "electronic file".

(Outline of system configuration)
FIG. 1 is a schematic view of a shared system according to an embodiment. The outline of the configuration of the shared system 1 will be described with reference to FIG.

As shown in FIG. 1, the shared system 1 of the present embodiment includes an electronic whiteboard 2, a video conferencing terminal 3, a car navigation device 4, a PC (Personal Computer) 5, a shared support server 6, and the like. It includes a schedule management server 8 (an example of a management server) and a voice-text conversion server 9 (an example of a conversion server).

Further, the electronic blackboard 2, the video conferencing terminal 3, the car navigation device 4, the PC 5, the sharing support server 6, the schedule management server 8 and the voice text conversion server 9 can communicate with each other via the communication network 10. The communication network 10 is constructed by the Internet, a mobile communication network, a LAN (Local Area Network), or the like. The communication network 10 may include not only wired communication but also wireless communication networks such as 3G (3rd Generation), WiMAX (Worldwide Interoperability for Microwave Access), and LTE (Long Term Evolution).

Further, the electronic blackboard 2 (an example of the information processing device) is used in the conference room X. The video conferencing terminal 3 (an example of an information processing device) is used in the conference room Y. In addition, the union is subject to reservation by each user. The car navigation device 4 (an example of an information processing device) is used in the vehicle α. The vehicle α in this case is a vehicle that is subject to car sharing. Vehicles include cars, motorcycles, bicycles, wheelchairs and the like.

A "union" refers to an object, service, space (room), location, or information shared by multiple people or groups. The conference room X, the conference room Y, and the vehicle α are examples of a union shared by a plurality of users. An example of the information given to a union is an account. For example, in a specific service provided on the Web, the account used is limited to one.

The electronic blackboard 2, the video conferencing terminal 3, and the car navigation device 4 are examples of communication terminals. The "communication terminal" is, for example, a terminal that can be used by a user signing in (see S32 described later). The communication terminal used in the vehicle α includes not only the car navigation device 4 but also a smartphone or a smart watch in which a car navigation application is installed.

The PC 5 (an example of an information processing device) is an example of a display terminal, and is an example of a registration device for registering a reservation for use of each union and registering an event scheduled to be executed by each user to the schedule management server 8. Is. Events include, for example, meetings, meetings, gatherings, gatherings, consultations, meetings, driving, boarding or traveling.

The sharing support server 6 is a computer, and provides support when sharing a union remotely to each communication terminal.

The schedule management server 8 is a computer and manages reservations for each union and schedules for each user.

The voice-text conversion server 9 is a computer, and converts sound (voice) data received from an external computer (for example, a shared support server 6) into text data. The voice-text conversion server 9 may be connected by WebSockt and realized by an external service (for example, Watson or Bing Speech) that converts voice into text.

Here, the shared support server 6, the schedule management server 8, and the voice-text conversion server 9 are referred to as a management system. The management system may be, for example, a computer in which all or part of the functions of the shared support server 6, the schedule management server 8, and the voice-text conversion server 9 are integrated. Further, each of the shared support server 6, the schedule management server 8, and the voice-text conversion server 9 may be configured to distribute and realize each function among a plurality of computers. Further, the shared support server 6, the schedule management server 8 and the voice text conversion server 9 will be described as being server computers existing in the cloud environment, but the shared support server 6 and the voice text conversion server 9 are electronic blackboards 2 and the like. It may be a server that operates on the communication terminal of the above, or it may be a server that exists in the on-premises environment. Further, the schedule management server 8 may be a server existing in the on-premises environment.

(Hardware configuration)
Subsequently, the hardware configuration of the device or terminal for constructing the shared system 1 will be described with reference to FIGS. 2 to 5.

<Hardware configuration of electronic blackboard>
FIG. 2 is a diagram showing an example of the hardware configuration of the electronic blackboard according to the embodiment. As shown in FIG. 2, the electronic blackboard 2 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, and an SSD (Solid State Drive) 204. , Network I / F 205 and external device connection I / F (Interface) 206.

Of these, the CPU 201 controls the operation of the entire electronic blackboard 2. The ROM 202 stores a program used for driving the CPU 201 such as the CPU 201 and the IPL (Initial Program Loader). The RAM 203 is used as a work area of the CPU 201. The SSD 204 stores various data such as a program for an electronic blackboard. The network I / F 205 controls communication with the communication network 10. The external device connection I / F 206 controls communication with a USB (Universal Bus) memory 2600, a PC 2700, and an external device (microphone 2200, speaker 2300, camera 2400).

Further, the electronic blackboard 2 includes a capture device 211, a GPU 212, a display controller 213, a contact sensor 214, a sensor controller 215, an electronic pen controller 216, a short-range communication circuit 219, an antenna 219a of the short-range communication circuit 219, and a power switch 222. I have.

Of these, the capture device 211 acquires the image data displayed on the display 220, which is an example of the display unit, via the display controller 213, and stores the image data in the RAM 203 or the like. The GPU (Graphics Processing Unit) 212 is a semiconductor chip that specializes in graphics. The display controller 213 controls and manages the screen display in order to output the output image from the capture device 211 or the GPU 212 to the display 220 or the like. The contact sensor 214 detects that the electronic pen 2500, the user's hand H, or the like has come into contact with the display 220. The sensor controller 215 controls the processing of the contact sensor 214. The contact sensor 214 inputs and detects the coordinates by the infrared blocking method. In this method of inputting coordinates and detecting coordinates, two light receiving and emitting devices installed at both upper ends of the display 220 emit a plurality of infrared rays in parallel with the display 220 and are provided around the display 220. This is a method of receiving light that is reflected by a reflecting member and returns on the same optical path as the light path emitted by the light receiving element. The contact sensor 214 outputs an infrared ID emitted by two light emitting / receiving devices blocked by the object to the sensor controller 215, and the sensor controller 215 specifies a coordinate position which is a contact position of the object. The electronic pen controller 216 determines whether or not there is a touch of the pen tip or a touch of the pen tip on the display 220 by communicating with the electronic pen 2500. The short-range communication circuit 219 is a communication circuit such as NFC (Near Field Communication) and Bluetooth (registered trademark). The power switch 222 is a switch for switching the power ON / OFF of the electronic blackboard 2.

Further, the electronic blackboard 2 is provided with a bus line 210. The bus line 210 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 201 shown in FIG.

The contact sensor 214 is not limited to the infrared blocking method, but is a capacitive touch panel that specifies a contact position by detecting a change in capacitance, and a contact position is specified by a voltage change between two opposing resistance films. Various detection means such as a resistance film type touch panel, an electromagnetic induction type touch panel that detects electromagnetic induction generated by contact of a contact object with a display unit, and specifies a contact position may be used. Further, the electronic pen controller 216 may determine whether or not there is a touch not only on the pen tip and pen end of the electronic pen 2500, but also on a portion held by the user of the electronic pen 2500 and other electronic pen portions.

<Hardware configuration of video conferencing terminal>
FIG. 3 is a diagram showing an example of the hardware configuration of the video conferencing terminal according to the embodiment. As shown in FIG. 3, the video conferencing terminal 3 includes a CPU 301, a ROM 302, a RAM 303, a flash memory 304, an SSD 305, a media I / F 307, an operation button 308, a power switch 309, and a bus. Line 310, network I / F311 and CMOS (Complementary Metal Switch Sensor) sensor 312, image pickup element I / F313, microphone 314, speaker 315, sound input / output I / F316, display I / F317, and so on. It includes an external device connection I / F 318, a short-range communication circuit 319, and an antenna 319a of the short-range communication circuit 319. Of these, the CPU 301 controls the operation of the entire video conferencing terminal 3. The ROM 302 stores a program used to drive the CPU 301 such as an IPL. The RAM 303 is used as a work area of the CPU 301. The flash memory 304 stores various data such as a communication program, image data, and sound data. The SSD 305 controls reading or writing of various data to the flash memory 304 according to the control of the CPU 301. An HDD may be used instead of the SSD. The media I / F 307 controls reading or writing (storage) of data to the recording medium 306 such as a flash memory. The operation button 308 is a button that is operated when selecting a destination of the video conferencing terminal 3. The power switch 309 is a switch for switching the power ON / OFF of the video conferencing terminal 3.

Further, the network I / F311 is an interface for data communication using a communication network 10 such as the Internet. The CMOS sensor 312 is a kind of built-in imaging means that images a subject and obtains image data under the control of the CPU 301. The image sensor I / F 313 is a circuit that controls the drive of the CMOS sensor 312. The microphone 314 is a kind of built-in sound collecting means for inputting voice. The sound input / output I / F 316 is a circuit that processes sound signal input / output between the microphone 314 and the speaker 315 under the control of the CPU 301. The display I / F 317 is a circuit that transmits image data to an external display 320 (an example of a display unit) under the control of the CPU 301. The external device connection I / F 318 is an interface for connecting various external devices. The short-range communication circuit 319 is a communication circuit such as NFC or Bluetooth.

Further, the bus line 310 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 301 shown in FIG.

The display 320 is a kind of display unit (display means) composed of a liquid crystal or an organic EL for displaying an image of a subject, an operation icon, and the like. Further, the display 320 is connected to the display I / F 317 by a cable 320c. The cable 320c may be a cable for analog RGB (VGA) signals, a cable for component video, DisplayPort, HDMI® (High-Definition Multimedia Interface) or DVI ( It may be a cable for a Digital Visual Interface) signal.

Instead of the CMOS sensor 312, it may be an image sensor such as a CCD (Charge Coupled Device) sensor. External devices such as an external camera, an external microphone, and an external speaker can be connected to the external device connection I / F318 by a USB cable or the like. When an external camera is connected, the external camera is driven in preference to the built-in CMOS sensor 312 according to the control of the CPU 301. Similarly, when an external microphone is connected, or when an external speaker is connected, the external microphone or the external speaker is given priority over the built-in microphone 314 or the built-in speaker 315, respectively, according to the control of the CPU 301. The external speaker is driven.

Further, the recording medium 306 has a structure that is detachable from the video conferencing terminal 3. Further, as long as it is a non-volatile memory that reads or writes data under the control of the CPU 301, not only the flash memory 304 but also an EEPROM or the like may be used.

<Hardware configuration of car navigation device>
FIG. 4 is a diagram showing an example of the hardware configuration of the car navigation device according to the embodiment. As shown in FIG. 4, the car navigation device 4 includes a CPU 401, a ROM 402, a RAM 403, an EEPROM 404, a power switch 405, an acceleration / orientation sensor 406, a media I / F 408, and a GPS receiver 409. And have.

Of these, the CPU 401 controls the operation of the entire car navigation device 4. The ROM 402 stores a program used to drive the CPU 401 such as an IPL. The RAM 403 is used as a work area for the CPU 401. The EEPROM 404 reads or writes various data such as a program for the car navigation device 4 under the control of the CPU 401. The power switch 405 is a switch for switching the power ON / OFF of the car navigation device 4. The acceleration / orientation sensor 406 is a variety of sensors such as an electronic magnetic compass, a gyro compass, and an acceleration sensor that detect the geomagnetism. The media I / F 408 controls reading or writing (storage) of data to a recording medium 407 such as a flash memory. The GPS receiving unit 409 receives a GPS signal from a GPS satellite.

Further, the car navigation device 4 includes a long-distance communication circuit 411, an antenna 411a of the long-distance communication circuit 411, a CMOS sensor 412, an image pickup element I / F 413, a microphone 414, a speaker 415, and a sound input / output I / output. It includes an F416, a display 417 (an example of a display unit), a display I / F418, an external device connection I / F419, a short-range communication circuit 420, and an antenna 420a of the short-range communication circuit 420.

Of these, the long-distance communication circuit 411 receives congestion information, road construction information, traffic accident information, etc. provided from the infrastructure outside the vehicle, and conversely transmits the vehicle position, emergency lifesaving signal, etc. to the outside. It is a circuit to do. The external infrastructure is, for example, a road information guidance system such as a VICS (registered trademark) (Vehicle Information and Communication System) system. The CMOS sensor 412 is a kind of built-in imaging means that images a subject and obtains image data under the control of the CPU 401. The image sensor I / F 413 is a circuit that controls the drive of the CMOS sensor 412. The microphone 414 is a kind of built-in sound collecting means for inputting voice. The sound input / output I / F 416 is a circuit that processes sound signal input / output between the microphone 414 and the speaker 415 under the control of the CPU 401. The display 417 is a kind of display unit (display means) such as a liquid crystal or an organic EL for displaying an image of a subject, various icons, and the like, and has a touch panel function. The touch panel is a kind of input means for the user to operate the car navigation device 4. The display I / F 418 is a circuit for displaying an image on the display 417. The external device connection I / F419 is an interface for connecting various external devices. The short-range communication circuit 420 is a communication circuit such as NFC or Bluetooth. Further, the car navigation device 4 includes a bus line 410. The bus line 410 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 401 shown in FIG.

<Hardware configuration of PC and server>
FIG. 5 is a diagram showing an example of the hardware configuration of the PC and the server according to the embodiment. The PC 5 is constructed by a computer, and as shown in FIG. 5, the CPU 501, the ROM 502, the RAM 503, the HD 504, the HDD (Hard Disk Drive) controller 505, the media I / F 507, and the display 508. (An example of a display unit), a network I / F 509, a keyboard 511, a mouse 512, a CD-RW drive 514, a speaker 515, and a bus line 510 are provided.

Of these, the CPU 501 controls the operation of the entire PC 5. The ROM 502 stores a program used to drive the CPU 501 such as an IPL. The RAM 503 is used as a work area of the CPU 501. The HD504 stores various data such as programs. The HDD controller 505 controls reading or writing of various data to the HD 504 according to the control of the CPU 501. The media I / F 507 controls reading or writing (storage) of data to a recording medium 506 such as a flash memory. The display 508 displays various information such as cursors, menus, windows, characters or images. The display 508 is an example of a display unit (display means). The network I / F 509 is an interface for performing data communication using the communication network 10. The keyboard 511 is a kind of input means including a plurality of keys for inputting characters, numerical values, various instructions, and the like. The mouse 512 is a kind of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like. The CD-RW drive 514 controls reading or writing of various data to a CD-RW (Compact Disk-Rewritable) 513 as an example of a removable recording medium. The speaker 515 outputs a sound signal according to the control of the CPU 501.

The PC 5 also includes a bus line 510. The bus line 510 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 501 shown in FIG.

Further, the shared support server 6 is constructed by a computer, and as shown in FIG. 5, the CPU 601 and the ROM 602, the RAM 603, the HD 604, the HDD controller 605, the recording media 606, and the media I / It includes an F607, a display 608, a network I / F609, a keyboard 611, a mouse 612, a CD-RW drive 614, and a bus line 610. These are CPU 501, ROM 502, RAM 503, HD 504, HDD controller 505, recording media 506, media I / F 507, display 508, network I / F 509, keyboard 511, mouse 512, CD-RW drive 514 and bus line 510 in PC5, respectively. It has the same configuration as.

Further, the schedule management server 8 is constructed by a computer, and as shown in FIG. 5, the CPU 801 and the ROM 802, the RAM 803, the HD 804, the HDD controller 805, the recording media 806, and the media I / It includes an F807, a display 808, a network I / F809, a keyboard 811, a mouse 812, a CD-RW drive 814, and a bus line 810. These are CPU 501, ROM 502, RAM 503, HD 504, HDD controller 505, recording media 506, media I / F 507, display 508, network I / F 509, keyboard 511, mouse 512, CD-RW drive 514 and bus line 510 in PC5, respectively. It has the same configuration as.

Further, the voice-to-text conversion server 9 is constructed by a computer, and as shown in FIG. 5, the CPU 901, the ROM 902, the RAM 903, the HD 904, the HDD controller 905, the recording medium 906, and the media I It includes a / F907, a display 908, a network I / F909, a keyboard 911, a mouse 912, a CD-RW drive 914, and a bus line 910. These are CPU 501, ROM 502, RAM 503, HD 504, HDD controller 505, recording media 506, media I / F 507, display 508, network I / F 509, keyboard 511, mouse 512, CD-RW drive 514 and bus line 510 in PC5, respectively. It has the same configuration as.

Each of the above programs may be a file in an installable format or an executable format, and may be recorded and distributed on a computer-readable recording medium. Examples of the recording medium include a CD-R (Compact Disk Recordable), a DVD (Digital Versaille Disk), a Blu-ray disc, an SD card, and the like. In addition, the recording medium can be provided domestically or internationally as a program product (Program Product).

Further, the shared support server 6 may be constructed by a single computer, or may be constructed by a plurality of computers arbitrarily assigned by dividing each unit (function, means or storage unit). The same applies to the schedule management server 8 and the voice-to-text conversion server 9.

(Software configuration of electronic blackboard)
Subsequently, the computer software installed on the electronic blackboard 2 will be described with reference to FIG. Here, the computer software (hereinafter referred to as software) is a program related to the operation of the computer and other information used for processing by the computer, and is similar to the program. A program is a command to a computer that is combined so that one result can be obtained. In addition, a program-like thing cannot be called a program because it is not a direct command to the computer, but it has properties similar to a program in that it regulates the processing of the computer. For example, a data structure (a logical structure of data represented by an interrelationship between data elements) corresponds to a program.

In addition, application is a general term for software used to perform a specific process in the classification of software. On the other hand, an operating system (hereinafter referred to as an OS) is software for controlling a computer and making computer resources available to applications and the like. The OS performs basic computer management and control such as input / output control, hardware management such as memory and hard disk, and process management. The application operates by using the function provided by the OS.

FIG. 6 is a diagram showing an example of the software configuration of the electronic blackboard according to the embodiment. As shown in FIG. 6, the OS 101, the Launcher 102, the schedule viewer 103a, the file viewer 103b, and the browser application 103c operate on the work area 15 of the RAM 203. OS101 is basic software that provides the basic functions of the electronic blackboard 2 and manages the entire electronic blackboard 2.

Launcher 102 is a launcher application that runs on OS 101. The Launcher 102 manages, for example, the start and end of an event such as a conference executed on the electronic blackboard 2, or the management of external applications such as the schedule viewer 103a, the file viewer 103b, and the browser application 103c used during the execution of the event.

The schedule viewer 103a, the file viewer 103b, and the browser application 103c are external applications (hereinafter, referred to as an external application 103 when it is not necessary to distinguish them) that operate on the Launcher 102. The external application 103 executes the process independently of the Launcher 102, and executes the service or function provided on the OS 101. Note that FIG. 6 shows an example in which three external applications, a schedule viewer 103a, a file viewer 103b, and a browser application 103c, are installed on the electronic blackboard 2, but the number of external applications is not limited to this.

(PC software configuration)
FIG. 7 is a diagram showing an example of the software configuration of the PC according to the embodiment. The computer software installed on the PC 5 will be described with reference to FIG. 7. As shown in FIG. 7, the OS5501, the minutes application 5502a and the browser 5502b operate on the work area 5500 of the RAM 503. OS5501 is basic software that provides the basic functions of the PC5 and manages the entire PC5.

The minutes application 5502a and the browser 5502b are applications for generating and displaying an execution event history screen which is a minutes of an event executed by the electronic blackboard 2 using various data transmitted from the schedule management server 8. is there. Note that FIG. 7 shows an example in which two external applications, the minutes application 5502a and the browser 5502b, are installed on the PC 5, but the number of external applications is not limited to this. The minutes application 5502a may be a Web application executed on the browser 5502b.

(Functional configuration of shared system)
Subsequently, the functional configuration of the present embodiment will be described with reference to FIGS. 8 to 15.

FIG. 8 is a functional block diagram of the shared system according to the embodiment. Note that, in FIG. 8, among the terminals, devices, and servers shown in FIG. 1, those related to the processing or operation described later are shown.

<Functional configuration of electronic blackboard>
As shown in FIG. 8, the electronic blackboard 2 includes a transmission / reception unit 21, a reception unit 22, an image processing unit 23-1, a sound processing unit 23-2, a display control unit 24, and a determination unit 25. It has a sound control unit 26, an acquisition / provision unit 28, and a storage / reading processing unit 29. Each of these parts is a function or means of functioning that any of the components shown in FIG. 2 is operated by an instruction from the CPU 201 according to a program developed on the RAM 203 from the SSD 204. is there. Further, the electronic blackboard 2 has a storage unit 2000 constructed by the RAM 203, SSD 204 or USB memory 2600 shown in FIG.

<< Each function configuration of the electronic blackboard >>
Next, each component of the electronic blackboard 2 will be described. The transmission / reception unit 21 is realized by the command from the CPU 201 shown in FIG. 2 and the network I / F205 and the external device connection I / F206, and various data (various data) with other terminals, devices or systems via the communication network 10. Or information) is sent and received.

The reception unit 22 is mainly realized by the command from the CPU 201 shown in FIG. 2, the contact sensor 214, and the electronic pen controller 216, and receives various inputs by the user.

The image processing unit 23-1 stores the image data displayed on the display 220, which is realized by the command from the CPU 201 shown in FIG. 2 and the capture device 211. Further, the image processing unit 23-1 is realized by the command from the CPU 201 shown in FIG. 2 and the GPU 212, and performs image processing for displaying an image on the display 220. In addition, the image processing unit 23-1 performs a process of converting the drawn image data obtained by drawing on the display 220 with the electronic pen 2500 or the hand H by the user into coordinate data. For example, when the electronic blackboard (2a) of an arbitrary base transmits coordinate data to the electronic blackboard (2b) of another base, the electronic blackboard (2b) is on the side of the other electronic blackboard (2b) based on the coordinate data. The same drawing is displayed on the display 220. Further, the image processing unit 23-1 performs image processing on the image data obtained by the camera 2400 taking an image of the subject.

The sound processing unit 23-2 is realized by the command from the CPU 201 shown in FIG. 2, converts the user's voice acquired by the microphone 2200 into a voice signal, and voices the sound data based on the voice signal. Perform processing.

The display control unit 24 is realized by a command from the CPU 201 shown in FIG. 2 and a display controller 213 shown in FIG. 2, and displays a drawn image on the display 220 or supports sharing using a Web browser. Access the server 6 to display various screen data. Specifically, the display control unit 24 starts and executes the Launcher 102 running on the OS 101 shown in FIG. 6 and the external application 103, and various types of drawings are drawn by the API (Application Programming Interface) provided by the OS 101. The screen is displayed on the display 220.

The determination unit 25 is realized by an instruction from the CPU 201 shown in FIG. 2 and makes various determinations.

The sound control unit 26 is a function realized by a command from the CPU 201 shown in FIG. 2 and outputs a sound signal from the speaker 2300. The sound control unit 26 sets the sound data to be output from the speaker 2300, and outputs the sound signal related to the set sound data from the speaker 2300 to reproduce the sound data.

The acquisition / provision unit 28 is executed by the instruction from the CPU 201 shown in FIG. 2 and the short-range communication circuit 219 and the antenna 219a, and is executed by the short-range communication with a dedicated terminal such as an IC card or a smartphone. Acquires and provides data.

The storage / reading processing unit 29 is executed by the instruction from the CPU 201 and SSD204 shown in FIG. 2, and is a process of storing various data in the storage unit 2000 and reading various data stored in the storage unit 2000. I do. Further, the storage unit 2000 overwrites and stores the image data and the sound data received when communicating with another electronic blackboard or the video conferencing terminal each time it is received. Of these, the image is displayed on the display 220 by the image data before being overwritten, and the sound is output from the speaker 2300 by the sound data before being overwritten.

Since the video conferencing terminal 3 and the car navigation device 4 have the same functions as the electronic blackboard 2, the description thereof will be omitted here.

Further, the transmission / reception unit 21, the reception unit 22, the image processing unit 23-1, the sound processing unit 23-2, the display control unit 24, the judgment unit 25, the sound control unit 26, and the acquisition / provision unit of the electronic blackboard 2 shown in FIG. The 28 and the storage / reading processing unit 29 conceptually show the functions, and are not limited to such a configuration. For example, a plurality of functional blocks illustrated as independent functional blocks on the electronic blackboard 2 shown in FIG. 8 may be configured as one functional block. On the other hand, in the electronic blackboard 2 shown in FIG. 8, the function of one function block may be divided into a plurality of functions and configured as a plurality of function blocks.

<Functional configuration of PC>
As shown in FIG. 8, the PC 5 stores the transmission / reception unit 51 (first transmission unit, transmission unit), the reception unit 52, the display control unit 54, the generation unit 56, the sound control unit 58, and the storage unit. -Has a read processing unit 59. Each of these parts is a function or a functioning means realized by operating any of the components shown in FIG. 5 by an instruction from the CPU 501 according to a program developed on the RAM 503 from the HD 504. .. Further, the PC 5 has a storage unit 5000 constructed by the HD 504 shown in FIG.

<< PC function configuration >>
Next, each component of the PC 5 will be described. The transmission / reception unit 51 is realized by the command from the CPU 501 and the network I / F 509 shown in FIG. 5, and transmits / receives various data (or information) to / from another terminal, device, or system via the communication network 10.

The reception unit 52 is mainly realized by the instructions from the CPU 501 shown in FIG. 5 and the keyboard 511 and the mouse 512, and receives various inputs by the user.

The display control unit 54 is realized by a command from the CPU 501 shown in FIG. 5, and displays an image on the display 508 or accesses the sharing support server 6 using a Web browser to display various screen data. To do. Specifically, the display control unit 54 accesses the shared support server 6 or the schedule management server 8 by, for example, starting and executing the minutes application 5502a or the browser 5502b running on the OS 5501 shown in FIG. .. Then, the display control unit 54 downloads, for example, HTML (HyperText Markup Language) and downloads WebAPP (WebApplication) including CSS (Cascading Style Sheets) or Javascript (registered trademark) and the like, and generates the WebAPP. The image data is displayed on the display 508. The display control unit 54 displays, for example, HTML5 (Extendable Markup Language), JSON (JavaScript Object Notification), SOAP (Simple Object Access Protocol) format, or the like.

The generation unit 56 is a function realized by a command from the CPU 501 shown in FIG. 5 and generates various image data to be displayed on the display 508. The generation unit 56 generates various image data using the content data received by the transmission / reception unit 51. The generation unit 56, for example, renders the text data which is the content data and generates the image data related to the text data which is the content image data in order to display the rendered data. Rendering is a process of interpreting data described in a language for describing a Web page (HTML, CSS, XML, etc.) and calculating the arrangement of characters, image data, and the like actually displayed on the screen. The generation unit 56 is an example of the generation means.

The sound control unit 58 is a function realized by a command from the CPU 501 shown in FIG. 5 and outputs a sound signal from the speaker 515. The sound control unit 58 sets the sound data to be output from the speaker 515, and outputs the sound signal related to the set sound data from the speaker 515 to reproduce the sound data. The sound control unit 58 is an example of the reproduction means.

The storage / reading processing unit 59 is executed by an instruction from the CPU 501 shown in FIG. 5 and an HDD controller 505 or the like to store various data in the storage unit 5000 or read various data from the storage unit 5000. Perform processing.

The transmission / reception unit 51, the reception unit 52, the display control unit 54, the generation unit 56, the sound control unit 58, and the storage / reading processing unit 59 of the PC 5 shown in FIG. 8 conceptually show the functions. It is not limited to such a configuration. For example, a block of a plurality of functions shown as a block of an independent function in the PC 5 shown in FIG. 8 may be configured as a block of one function. On the other hand, in the PC 5 shown in FIG. 8, the function of one function block may be divided into a plurality of functions and configured as a plurality of function blocks.

<Functional configuration of shared support server>
The shared support server 6 has a transmission / reception unit 61, an authentication unit 62, a creation unit 63, a generation unit 64, a determination unit 65, and a storage / reading processing unit 69. Each of these parts functions or functions as any of the components shown in FIG. 5 operates by an instruction from the CPU 601 according to the sharing support program developed on the RAM 603 from the HD 604. It is a means. Further, the shared support server 6 has a storage unit 6000 constructed by the HD 604 or the like shown in FIG.

<< User Authentication Management Table >>
FIG. 9A is a conceptual diagram showing a user authentication management table. In the storage unit 6000, a user authentication management DB 6001 configured by a user authentication management table as shown in FIG. 9A is constructed. In this user authentication management table, a user ID for identifying a user, a user name, an organization ID for identifying an organization to which the user belongs, and a password are associated and managed. The organization ID also includes a domain name representing a group or organization for managing a plurality of computers on a communication network.

<< Access control table >>
FIG. 9B is a conceptual diagram showing an access management table. In the storage unit 6000, an access management DB 6002 configured by an access management table as shown in FIG. 9B is constructed. In this access management table, the organization ID, the access ID required for authentication when accessing the schedule management server 8, and the access password are associated and managed. The access ID and access password are provided by the shared support server 6 by the schedule management server 8 via WebAPI or the like using the protocol of HTTP (Hypertext Transfer Protocol) or HTTPS (Hypertext Transfer Protocol). It is necessary to use the function). The schedule management server 8 manages a plurality of schedulers, and the schedulers used may differ depending on the organization. Therefore, it is necessary to manage the schedule management server 8 using the access management table.

<< Schedule management table >>
FIG. 9C is a conceptual diagram showing a schedule management table. In the storage unit 6000, a schedule management DB 6003 composed of a schedule management table as shown in FIG. 9C is constructed. In this schedule management table, for each scheduled event ID and execution event ID, the organization ID, the user ID of the reservation person, the presence / absence of participation of this reservation person, the reservation person name, the scheduled start time, the scheduled end time, the event name, etc. The user ID of the participant, the presence / absence of participation of other participants, the names of other participants, and the file data are associated and managed.

Of these, the scheduled event ID is identification information for identifying a reserved event. The scheduled event ID is an example of scheduled event identification information for identifying an event scheduled to be executed. The execution event ID is identification information for identifying an event that is actually executed or an event that is being executed among the reserved events. The execution event ID is an example of execution event identification information for identifying an executed event or an executed event. The reservation person's name is the name of the person who reserved the union, for example, the name of the organizer when the union is a conference room, and the name of the driver, for example, when the union is a vehicle. The scheduled start time indicates the scheduled start time of the union. The scheduled end time indicates the scheduled end time of use of the union. The event name indicates the name of the event scheduled to be executed by the reservation person. The user IDs of other participants are identification information for identifying participants other than the reservation person. Other participant names are the names of participants other than the booker, and include unions. That is, the users in this case include unions as well as reservations and other participants. The file data is the file data of the material file used in the event corresponding to the scheduled event ID registered by the user A using the schedule input screen 550 (see FIG. 20) described later. The file data is data in a predetermined file format created by using various applications. The file format of the file data is, for example, a PowerPoint file, an Excel file, or the like.

<< Content management table >>
FIG. 10A is a conceptual diagram showing a content management table. In the storage unit 6000, a content management DB 6005 composed of a content management table as shown in FIG. 10 is constructed. In this content management table, the content processing ID, the type of content processing, the content of the content, and the start date and time and the end date and time of the content processing are managed in association with each execution event ID. Here, the content is the content of an execution event that occurred in an event such as a meeting, or the material used in the event. Types of content processing include recording, snapshots, voice-to-text conversion, action item generation, and material delivery. Further, the content processing ID is identification information for identifying the processing of the content generated in each event.

Here, the content includes historical information indicating the execution content of the event and action items generated by the executed event. The history information is data such as recorded data, snapshots, voice texts or materials. A snapshot is a process of acquiring a display screen as image data at a certain point in time of an event being executed. Note that snapshots are also referred to as, for example, capture or image recognition.

When the type of content processing is "recording", the content of the content includes a URL indicating a storage destination of the recorded sound data. When the type of content processing is "snapshot", the content of the content includes a URL indicating a save destination of the image data of the screen acquired by the snapshot (capture). The capture is to save the image (still image, moving image) displayed on the display 220 as image data. Further, when the type of content processing is "voice-text conversion", the content of the content includes a URL indicating a storage destination of the text data of the received voice text.

Here, the action item indicates the content of an action that occurs at an event such as a meeting and should be dealt with by a person related to the event. When the type of content processing is "action item occurrence", the content content is the user ID of the performer of the action item, the deadline date for completing the action item, and the URL indicating the storage destination of the image data indicating the action item. including.

<< Speaker Information Management Table >>
FIG. 10B is a conceptual diagram showing a speaker information management table. In the storage unit 6000, a speaker information management DB 6007 composed of a speaker information management table as shown in FIG. 10B is constructed. In this speaker information management table, the user ID and the speaker ID are managed in association with each other. Here, the speaker ID is identification information for identifying a speaker of sound data generated by an event such as a conference.

<< Each function configuration of the shared support server >>
Next, each functional configuration of the shared support server 6 will be described in detail. In the following description of each functional configuration of the shared support server 6, among the constituent elements shown in FIG. 5, the main constituent elements for realizing each functional configuration of the shared support server 6 are used. The relationship is also explained.

The transmission / reception unit 61 of the shared support server 6 shown in FIG. 8 is realized by the instruction from the CPU 601 shown in FIG. 5 and the network I / F 609 shown in FIG. 5, and is realized via the communication network 10. Sends and receives various data (or information) to and from other terminals, devices, or systems.

The authentication unit 62 is realized by the command from the CPU 601 shown in FIG. 5, and the information (user ID, organization ID and password) sent from the communication terminal is registered in the user authentication management DB 6001 in advance. Authentication is performed by determining whether the information is present.

The creation unit 63 is realized by the instruction from the CPU 601 shown in FIG. 5, and is as shown in FIG. 26, which will be described later, based on the reservation information and the schedule information sent from the schedule management server 8. Create a reservation list screen 230.

The generation unit 64 is realized by the instruction from the CPU 601 shown in FIG. 5, and generates the execution event ID, the content processing ID, and the URL of the storage destination.

The determination unit 65 is realized by an instruction from the CPU 601 shown in FIG. 5 and makes various determinations. This determination will be described later.

The storage / reading processing unit 69 is realized by the command from the CPU 601 shown in FIG. 5 and the HDD controller 605 shown in FIG. 5, and various data can be stored in the storage unit 6000 or stored in the storage unit 6000. Performs processing such as reading various stored data.

The transmission / reception unit 61, the authentication unit 62, the creation unit 63, the generation unit 64, the judgment unit 65, and the storage / reading processing unit 69 of the shared support server 6 shown in FIG. 8 conceptually show the functions. , The configuration is not limited to this. For example, a block of a plurality of functions shown as a block of an independent function in the shared support server 6 shown in FIG. 8 may be configured as a block of one function. On the other hand, in the shared support server 6 shown in FIG. 8, the functions of one function block may be divided into a plurality of functions and configured as a plurality of function blocks.

<Functional configuration of schedule management server>
The schedule management server 8 has a transmission / reception unit 81 (second transmission unit), an authentication unit 82, a generation unit 83, a search unit 84, and a storage / reading processing unit 89 (memory processing unit). .. Each of these parts functions or functions as any of the components shown in FIG. 5 operates by an instruction from the CPU 801 according to a schedule management program developed on the RAM 803 from the HD 804. It is a means. Further, the schedule management server 8 has a storage unit 8000 constructed by the HD 804 shown in FIG.

<< User Authentication Management Table >>
FIG. 11A is a conceptual diagram showing a user authentication management table. In the storage unit 8000, a user authentication management DB 8001 composed of a user authentication management table as shown in FIG. 11A is constructed. In this user authentication management table, a user ID for identifying a user is managed by associating an organization ID and a password for identifying the organization to which the user belongs.

<< User management table >>
FIG. 11B is a conceptual diagram showing a user management table. In the storage unit 8000, a user management DB 8002 configured by a user management table as shown in FIG. 11B is constructed. In this user management table, the user ID and the user name (user name) indicated by the user ID are associated and managed for each organization ID.

<< Union management table >>
FIG. 11C is a conceptual diagram showing a union management table. In the storage unit 8000, a union management DB 8003 composed of a union management table as shown in FIG. 11C is constructed. In this union management table, a union ID for identifying a union and a union name (union name) are associated and managed for each organization ID.

<< Union reservation management table >>
FIG. 12A is a conceptual diagram showing a union reservation management table. In the storage unit 8000, a union reservation management DB 8004 configured by a union reservation management table as shown in FIG. 12A is constructed. In this union reservation management table, reservation information in a state in which each information is associated is managed. The reservation information includes a union ID, a union name, a user ID of the reservation person, a scheduled start date and time of use, a scheduled date and time of end of use, and an event name for each organization ID. Of these, the scheduled start date and time of use indicates the scheduled start date and time of use of the union. The scheduled end date and time of use indicates the scheduled end date and time of use of the union. Each date and time shows the year, month, day, hour, minute, second, and time zone, but in FIG. 12A, due to space limitations, the year, month, day, hour, and minute are also shown.

<< Event Management Table >>
FIG. 12B is a conceptual diagram showing an event management table. In the storage unit 8000, an event management DB 8005 composed of an event management table as shown in FIG. 12B is constructed. In this event management table, schedule information in a state in which each information is associated is managed. The schedule information is managed by associating an organization ID, a user ID, a user name, an event start scheduled date and time, an event end scheduled date and time, and an event name for each scheduled event ID. Of these, the scheduled start date and time of the event indicates the scheduled start date and time when the event is executed. The scheduled end date and time of the event indicates the scheduled end date and time when the event is executed. Each date and time shows the year, month, day, hour, minute, second, and time zone, but in FIG. 12B, due to space limitations, the year, month, day, hour, and minute are also shown. Further, in the event management table, the file data of the material file used in the event shown in the schedule information is managed in association with the schedule event ID.

<< Server Authentication Management Table >>
FIG. 13A is a conceptual diagram showing a server authentication management table. In the storage unit 8000, a server authentication management DB 8006 configured by a server authentication management table as shown in FIG. 13A is constructed. In this server authentication management table, an access ID and an access password are associated and managed. The access ID and access password have the same concept as the access ID and access password managed by the access management DB 6002 of the shared support server 6.

<< Execution event history management table >>
FIG. 13B is a conceptual diagram showing an execution event history management table. In the storage unit 8000, an execution event history management DB 8008 configured by an execution event history management table as shown in FIG. 13B is constructed. In this execution event history management table, the content processing ID, the type of content processing, the content of the content, and the start date and time and the end date and time of the content processing are managed in association with each execution event ID. The execution event history management DB 8008 manages a part of the same data as the content management DB 6005. The same data is the execution event ID, the content processing ID, the type of content processing, the start date and time and the end date and time of the content processing. The contents of the content are the same, except that the notation method (http: // or c: //) of the save destination of the content data is different.

<< Execution event management table >>
FIG. 14A is a conceptual diagram showing an execution event management table. In the storage unit 8000, an execution event management DB 8009 configured by an execution event management table as shown in FIG. 14A is constructed. In this execution event management table, the event name, the start date and time of the event, and the end date and time are associated with the execution event ID and managed. The execution event management DB 8009 manages information related to an actually executed event among the schedule information managed in the event management DB 8005.

<< Related information management table >>
FIG. 14B is a conceptual diagram showing a related information management table. In the storage unit 8000, a related information management DB 8010 configured by a related information management table as shown in FIG. 14B is constructed. In this related information management table, related information in a state in which each information (data) is associated is managed for each execution event ID. The related information is managed in association with the time zone in which the content is generated, the sound data, the voice text data, and the screen data. The content occurrence time zone indicates the elapsed time from the start date and time of the event at the time when the content is generated in the executed event. Here, the content generation time zone is generated by the generation unit 83 based on the event start date and time stored in the event management DB 8005 and the content processing start date and time and end date and time stored in the execution event history management DB 8008. To. The time zone in which the content is generated is an example of time information. Further, the sound data includes a content processing ID and a content type. The voice text data and the screen data include a content processing ID, a content type, and a sequence number. The sequence No. indicates the order of occurrence time when the content processing occurs in each of the voice text data and the screen data.

<< Text information management table >>
FIG. 15 is a conceptual diagram showing a text information management table. In the storage unit 8000, a text information management DB 8012 composed of a text information management table as shown in FIG. 15 is constructed. In this text information management table, text information including voice text data generated in the executed event is managed for each execution event ID. The text information is associated with a content processing ID, a text ID for identifying the text data, a transcrit indicating the content of the text data, a user ID, and state information indicating the state of the text data. Of these, the content processing ID identifies the content processing whose type of content processing is "speech-to-text conversion". Further, the transcript (text data) is text data indicating the content of the content associated with the content processing ID similar to the content processing ID associated with the text information in the execution event history management DB 8008. Further, the state information is information indicating whether or not the text data is edited. The state information indicates "Original" information indicating that the text data has not been edited when the associated text data has not been edited from the text data generated by the voice-to-text conversion server 9.

<< Each function configuration of the schedule management server >>
Next, each functional configuration of the schedule management server 8 will be described in detail. In the following description of each functional configuration of the schedule management server 8, among the constituent elements shown in FIG. 5, the main constituent elements for realizing each functional configuration of the schedule management server 8 are used. The relationship is also explained.

The transmission / reception unit 81 of the schedule management server 8 shown in FIG. 8 is realized by the instruction from the CPU 801 shown in FIG. 5 and the network I / F809 shown in FIG. 5, and is realized via the communication network 10. Sends and receives various data (or information) to and from other terminals, devices, or systems.

The authentication unit 82 is realized by the command from the CPU 801 shown in FIG. 5, and the information (user ID, organization ID and password) sent from the common entity is registered in the user authentication management DB 8001 in advance. Authentication is performed by determining whether the information is present. Further, the authentication unit 82 authenticates by determining whether the information (access ID and access password) sent from the shared support server 6 is the information registered in advance in the server authentication management DB 8006.

The generation unit 83 is a function of generating related information registered in the related information management DB 8010, which is realized by an instruction from the CPU 801 shown in FIG.

The search unit 84 is realized by a command from the CPU 801 shown in FIG. 5, and a search specifying at least one of a keyword, a period, and a speaker input on the minutes search screen 590 shown in FIG. 60, which will be described later. Search for minutes that match the conditions. For example, the search may be mandatory by at least the keyword and speaker among the keywords, period and speaker.

The storage / reading processing unit 89 is realized by the command from the CPU 801 shown in FIG. 5 and the HDD controller 805 shown in FIG. 5, and various data can be stored in the storage unit 8000 or stored in the storage unit 8000. Performs processing such as reading various stored data.

The transmission / reception unit 81, the authentication unit 82, the generation unit 83, the search unit 84, and the storage / reading processing unit 89 of the schedule management server 8 shown in FIG. 8 conceptually show the functions as described above. It is not limited to the configuration. For example, a block of a plurality of functions shown as a block of an independent function in the schedule management server 8 shown in FIG. 8 may be configured as a block of one function. On the other hand, the schedule management server 8 shown in FIG. 8 may divide the function of one function block into a plurality of functions and configure the schedule management server 8 as a block of a plurality of functions.

<Functional configuration of voice-text conversion server>
The voice-text conversion server 9 includes a transmission / reception unit 91, a conversion unit 93, a speaker identification unit 95, a generation unit 97, and a storage / reading processing unit 99. Each of these parts is a function or a functioning means realized by operating any of the components shown in FIG. 5 by an instruction from the CPU 901 according to a program developed on the RAM 903 from the HD 904. .. Further, the voice-text conversion server 9 has a storage unit 9000 constructed by the HD904 shown in FIG.

<< User voice information management table >>
FIG. 16 is a conceptual diagram showing a user voice information management table. In the storage unit 9000, a user voice information management DB 9001 configured by a user voice information management table as shown in FIG. 16 is constructed. In this user voice information management table, user voice data, which is voice data for identifying a user, and a speaker ID are managed in association with each other. The user voice data includes a voice file in which the voice of the user recorded using the electronic blackboard 2 or the like is stored. The voice-text conversion server 9 executes a speaker identification process for identifying a speaker of the sound data by using the sound data generated by an event such as a conference and the user voice data.

<< Each function configuration of the voice-text conversion server >>
Next, each functional configuration of the voice-text conversion server 9 will be described in detail. In the following, in explaining each functional configuration of the voice-text conversion server 9, among the constituent elements shown in FIG. 5, the main constituent elements for realizing each functional configuration of the voice-text conversion server 9. The relationship with is also explained.

The transmission / reception unit 91 of the voice-to-text conversion server 9 shown in FIG. 8 is realized by the command from the CPU 901 shown in FIG. 5 and the network I / F 909 shown in FIG. Sends and receives various data (or information) to and from other terminals, devices, or systems via.

The conversion unit 93 is realized by the instruction from the CPU 901 shown in FIG. 5, and converts the sound data received by the transmission / reception unit 91 via the communication network 10 into text data.

The speaker identification unit 95 identifies the speaker of the sound data realized by the instruction from the CPU 901 shown in FIG. 5 and received by the transmission / reception unit 91 via the communication network 10.

The generation unit 97 is realized by the instruction from the CPU 901 shown in FIG. 5, and generates a speaker ID for identifying the speaker of the sound data generated by an event such as a conference.

The storage / reading processing unit 99 is realized by the command from the CPU 901 shown in FIG. 5 and the HDD controller 905 shown in FIG. 5, and various data can be stored in the storage unit 9000 or stored in the storage unit 9000. Performs processing such as reading various stored data.

Each of the above IDs is an example of identification information. In addition, the organization ID includes a company name, a business establishment name, a department name, an area name, and the like. The user ID includes an employee number, a driver's license number, my number in the Japanese social security / tax number system, and the like.

Further, the transmission / reception unit 91, the conversion unit 93, the speaker identification unit 95, the generation unit 97, and the storage / reading processing unit 99 of the voice-text conversion server 9 shown in FIG. 8 conceptually show the functions. It is not limited to such a configuration. For example, a block of a plurality of functions shown as a block of an independent function in the voice-text conversion server 9 shown in FIG. 8 may be configured as a block of one function. On the other hand, in the voice-text conversion server 9 shown in FIG. 8, the function of one function block may be divided into a plurality of functions and configured as a plurality of function blocks.

(Processing or operation of the embodiment)
Hereinafter, the processing or operation of each embodiment will be described.

<Schedule registration process>
FIG. 17 is a sequence diagram showing an example of the flow of the schedule registration process of the shared system according to the embodiment. FIG. 18 is a diagram showing a sign-in screen. FIG. 19 is a diagram showing an example of an initial screen of a PC. FIG. 20 is a diagram showing a schedule input screen. A process in which the user A (Rikotaro) registers his / her own schedule from the PC 5 to the schedule management server 8 will be described with reference to FIGS. 17 to 20.

First, when the user A operates the keyboard 511 or the like of the PC 5, the display control unit 54 of the PC 5 displays the sign-in screen 530 for signing in on the display 508 as shown in FIG. Display (step S11). On the sign-in screen 530, an input field 531 for inputting a user ID and an organization ID of a user, an input field 532 for inputting a password, a sign-in button 538 pressed when signing in, and sign-in are displayed. It includes a cancel button 539 that is pressed when canceling. Here, the user ID and the organization ID are the e-mail addresses of the user A. The user name part of the e-mail address is the user ID, and the domain name part is the organization ID. The input field 531 may be an input field for separately inputting the user ID and the organization ID instead of the e-mail address.

Next, when the user A inputs his / her user ID and organization ID in the input field 531, enters his / her password in the input field 532, and presses the sign-in button 538, the reception unit 52 signs in. (Step S12). Then, the transmission / reception unit 51 of the PC 5 transmits the sign-in request information indicating the sign-in request to the schedule management server 8 (step S13). The sign-in request information includes the information (user ID, organization ID, and password) received in step S12. As a result, the transmission / reception unit 81 of the schedule management server 8 receives the sign-in request information.

Next, the authentication unit 82 of the schedule management server 8 authenticates the user A by using the user ID, the organization ID, and the password (step S14). Specifically, the storage / reading processing unit 89 is a user corresponding to the set of the user ID, the organization ID, and the password received in step S13 in the user authentication management DB 8001 (see FIG. 11 (A)). Search for ID, organization ID and password pairs. If there is a corresponding pair, the authentication unit 82 determines that the requesting user A is a legitimate user. If there is no corresponding pair, the authentication unit 82 determines that the user A is an unjust (illegal) user. If it is not valid, the transmission / reception unit 81 notifies the PC5 that it is not valid, but here, the case where it is valid will be continued.

Next, the transmission / reception unit 81 transmits the authentication result to the PC 5 (step S15). As a result, the transmission / reception unit 51 of the PC 5 receives the authentication result.

Next, when the authentication result to the effect that it is valid is received in step S15, the generation unit 56 of the PC 5 generates the initial screen 540 as shown in FIG. 19 (step S16). Then, the display control unit 54 of the PC 5 displays the initial screen 540 as shown in FIG. 19 on the display 508 (step S17). The initial screen 540 includes a "schedule registration" button 541 pressed when registering a schedule, an "execution event history browsing" button 543 pressed when viewing the execution event history, and a closing of the initial screen. A "close" button 549 is included. Here, when the user presses the "schedule registration" button 541, the reception unit 52 accepts the schedule registration (step S18). Then, the transmission / reception unit 51 transmits a schedule registration request to the schedule management server 8 (step S19). As a result, the transmission / reception unit 81 of the schedule management server 8 receives the schedule registration request.

Next, the storage / reading processing unit 89 of the schedule management server 8 searches the user management DB 8002 (see FIG. 11B) using the organization ID received in step S13 as a search key, and all the corresponding items. Read the user ID and all user names (step S20). Then, the transmission / reception unit 81 transmits the schedule input screen information to the PC 5 (step S21). The schedule input screen information includes all user IDs and all user names read out in step S20. All the user names also include the name (name) of the reservation person who is the user A who input for sign-in in step S12. As a result, the transmission / reception unit 51 of the PC 5 receives the schedule input screen information.

Next, in the PC 5, the generation unit 56 generates the schedule input screen 550 using the schedule input screen information received in step S21. (Step S22). Then, the display control unit 24 of the PC 5 displays the schedule input screen 550 as shown in FIG. 20 on the display 508 (step S23).

In this schedule input screen 550, input the input field 551 for inputting the event name, the input field 552 for inputting the common entity ID or the common entity name, and the scheduled start date and time of the event execution (use of the common entity). Input field 553 for inputting, input field 554 for inputting scheduled end date and time of event execution (use of common entity), input field 555 for inputting memos such as agenda, display area for displaying the name of the person making the reservation 556, selection menu 557 for selecting the participant name of a participant other than the reservation person, "OK" button 558 pressed when registering the reservation, when inputting or canceling the input contents Includes a "CANCEL" button 559 that is pressed. The reservation person name is the name of the user who entered the PC5 for sign-in in step S12. The mouse pointer p1 is also displayed.

An e-mail address may be entered in the input field 552. Also, when a union name is selected in the selection menu 557, the union is also added as another participant.

Next, the user A inputs a predetermined item in the input fields 551 to 555, selects the name (user name) of the user who wants to participate in the event from the selection menu 557 using the mouse pointer p1. When the "OK" button 558 is pressed, the reception unit 52 accepts the input of the schedule information (step S24). Then, the transmission / reception unit 51 transmits the schedule information to the schedule management server 8 (step S25). This schedule information includes an event name, a union ID (or union name), a scheduled start date and time, a scheduled end date and time, a user ID of each participant, and a memo. When the union ID is entered in the input field 552 on the schedule input screen 550, the union ID is transmitted, and when the union name is entered in the input field 552, the union name is entered. Is sent. Further, on the schedule input screen 550, the user name is selected in the selection menu 557, but since the user ID is also received in step S21, the user ID corresponding to the user name is transmitted. As a result, the transmission / reception unit 81 of the schedule management server 8 receives the schedule information.

Next, the schedule management server 8 searches the union management DB 8003 (see FIG. 11 (C)) using the union ID (or union name) received in step S25 as a search key to search for the corresponding union. Read the name (or union ID) (step S26).

Next, the storage / reading processing unit 89 stores the reservation information in the union reservation management DB 8004 (see FIG. 12A) (step S27). In this case, the storage / reading processing unit 89 adds the reservation information for one record to the union reservation management table of the union reservation management DB 8004 managed by the scheduler registered in advance. The reservation information is configured based on the schedule information received in step S25 and the union name (or union ID) read in step S26. The scheduled start date and time in the union reservation management DB 8004 corresponds to the scheduled start date and time in the schedule information. Further, the scheduled end date and time of use in the union reservation management DB 8004 corresponds to the scheduled end date and time in the schedule information.

Further, the storage / reading processing unit 89 stores the schedule information in the event management DB 8005 (see FIG. 12B) (step S28). In this case, the storage / reading processing unit 89 adds schedule information for one record to the event management table of the event management DB 8005 managed by the scheduler registered in advance. The schedule information is configured based on the schedule information received in step S25. The scheduled event start date and time in the event management DB 8005 corresponds to the scheduled start date and time in the schedule information. The scheduled event end date and time in the event management DB 8005 corresponds to the scheduled end date and time in the schedule information.

As described above, the user A can register his / her own schedule in the schedule management server 8. Although the case where the schedule is registered by the PC 5 is described in FIGS. 17 to 20, the schedule may be registered by the electronic blackboard 2, the video conferencing terminal 3, or the car navigation device 4 by the same processing.

<User voice data registration process>
FIG. 21 is a sequence diagram showing an example of the flow of the user voice data registration process of the shared system according to the embodiment. FIG. 22 is a diagram showing a sign-in screen for registering user voice data. FIG. 23 is a diagram showing a user voice data registration screen. A process of registering user voice data used for identifying a speaker with respect to sound data generated by an event such as a conference will be described with reference to FIGS. 21 to 23.

First, when the user A operates the contact sensor 214 or the like of the electronic blackboard 2, the display control unit 24 of the electronic blackboard 2 registers the speaker on the display 220 as shown in FIG. The sign-in screen 520 for sign-in at the time of execution is displayed (step S231). The sign-in screen 520 has an input field 521 for entering the user ID and organization ID of the user, an input field 522 for entering the password, a sign-in button 528 pressed when signing in, and a sign. Includes a cancel button 529, which is pressed when canceling the login. Here, the user ID and the organization ID are the e-mail addresses of the user A. The user name part of the e-mail address is the user ID, and the domain name part is the organization ID. The input field 521 may be an input field for separately inputting the user ID and the organization ID instead of the e-mail address.

Next, when the user A inputs his / her user ID and organization ID in the input field 521, enters his / her password in the input field 522, and presses the sign-in button 528, the reception unit 22 signs in. (Step S232). Then, the transmission / reception unit 21 of the electronic blackboard 2 transmits the sign-in request information indicating the sign-in request to the schedule management server 8 (step S233). The sign-in request information includes the information (user ID, organization ID, and password) received in step S232. As a result, the transmission / reception unit 81 of the schedule management server 8 receives the sign-in request information.

Next, the authentication unit 82 of the schedule management server 8 authenticates the user A by using the user ID, the organization ID, and the password (step S234). Specifically, the storage / reading processing unit 89 is a user corresponding to the set of the user ID, the organization ID, and the password received in step S233 in the user authentication management DB 8001 (see FIG. 11 (A)). Search for ID, organization ID and password pairs. If there is a corresponding pair, the authentication unit 82 determines that the requesting user A is a legitimate user. If there is no corresponding pair, the authentication unit 82 determines that the user A is an unjust (illegal) user. If it is not valid, the transmission / reception unit 81 notifies the electronic blackboard 2 that it is not valid, but here, the case where it is valid will be continued.

Next, the transmission / reception unit 81 transmits the authentication result to the electronic blackboard 2 (step S235). As a result, the transmission / reception unit 21 of the electronic blackboard 2 receives the authentication result.

Next, when the display control unit 24 of the electronic blackboard 2 receives the authentication result to the effect that it is valid in step S235, the display control unit 24 generates the user voice data registration screen 560 as shown in FIG. 23 (step). S236). Then, the display control unit 24 of the PC 5 displays the user voice data registration screen 560 as shown in FIG. 23 on the display 508 (step S237). The user voice data registration screen 560 has a voice data display area 561 for displaying the file name of the user voice data in which the user's voice is recorded, and a recording button 562 pressed when recording the user's voice. The play button 563 pressed when playing back the recorded user voice data, the "register" button 568 pressed when registering the recorded user voice data, and the registration of the user voice data. Includes a "Cancel" button 569 that is pressed when the process is cancelled.

Next, when the user A selects the record button 562 and records his / her own voice using the microphone 2200, the sound processing unit 23-2 causes the user voice including the recorded voice of the user A. Acquire data (step S238). Then, when the user A presses the "register" button 568, the reception unit 22 accepts the user voice data registration request (step S239).

Then, the transmission / reception unit 21 transmits the user voice data acquired in step S238 to the voice text conversion server 9 (step S240). As a result, the transmission / reception unit 91 of the voice-text conversion server 9 receives the user voice data.

Next, the generation unit 97 of the voice-text conversion server 9 generates a speaker ID for identifying the speaker of the sound data generated by the event (step S241). Further, the storage / reading processing unit 99 stores the user voice information in the user voice information management DB 9001 (see FIG. 16) (step S242). The user voice information includes the user voice data received in step S240 and the speaker ID generated in step S241.

Then, the transmission / reception unit 91 transmits the speaker ID generated in step S241 to the sharing support server 6 (step S243). As a result, the transmission / reception unit 61 of the sharing support server 6 receives the speaker ID. The storage / reading processing unit 69 of the sharing support server 6 stores speaker information in the speaker information management DB 6007 (see FIG. 10B) (step S244). This speaker information includes the user ID of the user A authenticated in step 234 and the speaker ID received in step S243.

As described above, the user A can register the user voice data, which is the data of his / her own voice, in the voice text conversion server 9 in advance before the event such as the conference is started. Although the case where the speaker information is registered by the electronic blackboard 2 is described in FIGS. 21 to 23, the schedule registration may be performed by the video conferencing terminal 3, the car navigation device 4, or the PC 5 by the same processing. ..

<Event start processing>
24 and 27 are sequence diagrams showing an example of a flow of event start processing of the shared system according to the embodiment. FIG. 25 is a diagram showing a sign-in screen displayed on the electronic blackboard. FIG. 26 is a diagram showing a reservation list screen of the union. FIG. 28 is a diagram showing a detailed information screen of the event. FIG. 29 is a diagram showing a display screen displayed on the electronic blackboard at the start of the event. FIG. 30 is an image diagram showing a state in which the user uses the electronic blackboard. A process of having a meeting with other participants using the electronic blackboard 2 in the meeting room X reserved by the user A (Rikotaro) will be described with reference to FIGS. 24 to 30.

First, when the user A presses the power switch 222 of the electronic blackboard 2, the reception unit 22 of the electronic blackboard 2 receives the power ON from the user (step S31). When the power ON is received by the reception unit 22, the Launcher 102 shown in FIG. 6 is activated. Then, as shown in FIG. 25, the display control unit 24 of the electronic blackboard 2 displays the sign-in screen 110 for signing in on the display 220 (step S32). On the sign-in screen 110, a selection icon 111 pressed when user A signs in using his / her own IC card, user A's own e-mail address (user ID and organization ID), and It includes a selection icon 113 that is pressed when signing in by entering a password, and a power icon 115 that is pressed when the power is turned off without signing in.

Next, the user A presses the selection icon 111 to communicate with the short-range communication circuit 219 such as an IC card reader, or the user A presses the selection icon 113 to perform the communication with the user A. The reception unit 22 accepts the sign-in request by inputting its own e-mail address and password (step S33). Then, the transmission / reception unit 21 transmits the sign-in request information indicating the sign-in request to the shared support server 6 (step S34). The sign-in request information includes information received in step S33 (user ID, organization ID and password), time zone information of the country or region where the electronic blackboard 2 is installed, and a communication terminal (here, the electronic whiteboard). The user ID, organization ID and password of 2) are included. As a result, the transmission / reception unit 61 of the sharing support server 6 receives the sign-in request information.

Next, the authentication unit 62 of the shared support server 6 authenticates the user A by using the user ID, the organization ID, and the password of the user A received in step S34 (step S35). Specifically, the storage / reading processing unit 69 uses the user ID, organization ID, and password of the user A received in step S34 as search keys in the user authentication management DB 6001 (see FIG. 9A). , The set of the user ID, the organization ID and the password corresponding to the set of the user ID, the organization ID and the password of the received user A is searched. If there is a corresponding pair, the authentication unit 62 determines that the requesting user A is a legitimate user. If there is no corresponding pair, the authentication unit 62 determines that the requesting user A is an unjust (illegal) user. If it is not valid, the transmission / reception unit 61 notifies the electronic blackboard 2 that it is not valid, but here, the case where it is valid will be continued.

Next, the storage / reading processing unit 69 of the shared support server 6 responds by searching the access management DB 6002 (see FIG. 9B) using the organization ID of the user A received in step S34 as a search key. The access ID and access password to be used are read (step S36).

Next, the transmission / reception unit 61 transmits the reservation request information indicating the request for the reservation information of the union and the schedule request information indicating the request for the user's schedule information to the schedule management server 8 (step S37). The reservation request information and the schedule request information include the time zone information received in step S34, the user ID and organization ID of the communication terminal, and the access ID and access password read in step S36. .. As a result, the transmission / reception unit 81 of the schedule management server 8 receives the reservation request information and the schedule request information.

Next, the authentication unit 82 of the schedule management server 8 authenticates the shared support server 6 by using the access ID and the access password (step S38). Specifically, the storage / reading processing unit 89 sets the access ID and the access password corresponding to the set of the access ID and the access password received in step S37 in the server authentication management DB 8006 (see FIG. 13 (A)). To search for. If there is a corresponding pair, the authentication unit 82 determines that the requesting shared support server 6 is a legitimate accessor. If there is no corresponding pair, the authentication unit 82 determines that the requesting shared support server 6 is an unjust (illegal) accessor. If it is not valid, the transmission / reception unit 81 notifies the shared support server 6 that it is not valid, but here, the case where it is valid will be continued.

The storage / reading processing unit 89 of the schedule management server 8 uses the user ID of the communication terminal received in step S37 as a search key, and uses the union reservation management DB 8004 managed by the scheduler (see FIG. 12A). By searching, the corresponding reservation information is read out (step S39). In this case, the storage / reading processing unit 89 reads out the reservation information whose scheduled start date and time is today.

Further, the storage / reading processing unit 89 uses the user ID of the communication terminal received in step S37 as a search key to search the event management DB 8005 (see FIG. 12B) managed by the scheduler. Read the corresponding schedule information (step S40). In this case, the storage / reading processing unit 89 reads out the schedule information whose scheduled event start date and time is today. If the schedule management server 8 is located in a country or region different from the communication terminal such as the electronic blackboard 2, the time zone is set according to the country or region where the communication terminal is installed based on the time zone information. I'm adjusting.

Next, the transmission / reception unit 81 transmits the reservation information read in step S39 and the schedule information read in step S40 to the sharing support server 6 (step S41). As a result, the transmission / reception unit 61 of the sharing support server 6 receives the reservation information and the schedule information.

Next, the creation unit 63 of the sharing support server 6 creates a reservation list based on the reservation information and the schedule information received in step S41 (step S42). Then, the transmission / reception unit 61 transmits the reservation list information indicating the contents of the reservation list to the electronic blackboard 2 (step S43). As a result, the transmission / reception unit 21 of the electronic blackboard 2 receives the reservation list information.

Next, on the electronic blackboard 2, the display control unit 24 displays the reservation list screen 230 as shown in FIG. 22 on the display 220 (step S44). The reservation list screen 230 includes a display area 231 for displaying a union name (here, a place name) and a display area 232 for displaying today's date and time. Further, on the reservation list screen 230, event information 235, 236, 237 and the like indicating an event using today's union (here, conference room X) are displayed. Each event information includes the scheduled start time and end time of use of the union, the event name, and the name of the person who reserved the union (reserver name) for each event. The event information includes start buttons 235s, 236s, 237s, and the like that are pressed when the user identifies an event to be started.

Next, in FIG. 27, when the user A presses the start button 235s with, for example, the electronic pen 2500, the reception unit 22 accepts the selection of the event indicated by the event information 235 (step S51). Next, the transmission / reception unit 21 of the electronic blackboard 2 transmits a scheduled event ID indicating the scheduled event selected in step S51 to the shared support server 6 (step S52). The process of step S52 is the process of requesting the transmission of the execution event identification information. As a result, the transmission / reception unit 61 of the shared support server 6 receives the selected scheduled event ID.

Next, in the shared support server 6, the generation unit 64 generates a unique execution event ID (step S53). Then, the storage / reading processing unit 69 manages the execution event ID generated in step S53, the reservation event ID received in step S52, the user ID and organization ID of the reservation person, and the event information in association with each other (step). S54). The user ID and organization ID of the reservation person, and the event information are IDs and information based on the reservation information and the schedule information received in step S41. At this point, there is no entry in the participation / non-participation column in the reservation management table (see FIG. 9C).

Next, in the sharing support server 6, the transmission / reception unit 61 transmits the file data transmission request information indicating the transmission request of the file data registered in the schedule management server 8 to the schedule management server 8 (step S55). The file data transmission request information includes the scheduled event ID received in step S52, the user ID and organization ID of the communication terminal received in step S34, and the access ID and access password read in step S36. ing. As a result, the transmission / reception unit 81 of the schedule management server 8 receives the file data transmission request information.

Next, the storage / reading processing unit 89 of the schedule management server 8 searches the event management DB 8005 (see FIG. 12B) using the scheduled event ID received in step S55 as a search key to obtain the scheduled event ID. Read the associated file data (step S56). Then, the transmission / reception unit 81 transmits the file data read in step S56 to the sharing support server 6 (step S57). As a result, the transmission / reception unit 61 of the sharing support server 6 receives the file data.

Next, the storage / reading processing unit 69 of the shared support server 6 associates the scheduled event ID received in step S52 and the execution event ID generated in step S53 with the file data received in step S57. It is stored and managed in the schedule management DB 6003 (see FIG. 9C) (step S58).

Then, the transmission / reception unit 61 transmits the execution event ID generated in step S53 and the file data received in step S57 to the electronic blackboard 2 (step S59). As a result, the transmission / reception unit 21 of the electronic blackboard 2 receives the execution event ID and the file data.

Next, in the electronic blackboard 2, the storage / reading processing unit 29 stores the execution event ID and the file data in the storage unit 2000 (step S60). Here, the file data transmitted from the shared support server 6 is stored in a specific storage area of the storage unit 2000. As a result, the electronic blackboard 2 accesses the specific storage area during the execution of the event, so that the display control unit 24 displays the file data stored in the specific storage area on the display 220. Here, the specific storage area is a temporary storage destination of data provided for each event being executed, and is a storage area specified by an arbitrary path (character string) indicating the location in the storage unit 2000. Is. The specific storage area is not limited to the configuration provided inside the electronic blackboard 2, but is present in an external storage device connected to the electronic blackboard 2, or a local server that exists in the on-premises environment and can communicate with the electronic blackboard 2. It may be provided.

Next, the display control unit 24 displays the detailed information screen 250 of the selected event on the display 220 as shown in FIG. 28 (step S61). The detailed information screen 250 of this event includes a display area 251 for the event name, a display area 252 for the scheduled event implementation time (scheduled start time and scheduled end time), and a display area 253 for the reserved person name. Further, on the event detailed information screen 250, a display area 256 for displaying the contents of the memo, a display area 257 for displaying the names of prospective participants, and a file stored in a specific storage area of the storage unit 2000 are displayed. A display area 258 for displaying identification information (for example, a file name) for identifying data is displayed. The display area 257 displays the names of the bookers and other selected participants shown in FIG. 20, and each prospective participant name checks who is actually attending the meeting. Check box is displayed. Further, in the display area 258, the file data stored in the specific storage area of the storage unit 2000, that is, the file name of the file data downloaded from the shared support server 6 is displayed, and the file data is downloaded from the shared support server 6. The file name of the file data inside is also displayed. Further, the lower right side of the event detailed information screen 250 includes a "close" button 259 for closing the detailed information screen 250.

Next, when the user checks the check box of the user who is actually participating among the names of the prospective participants and presses the "close" button 259, the reception unit 22 accepts the selection of the participant. (Step S62). Then, the transmission / reception unit 21 transmits the user ID of each prospective participant and information indicating the presence / absence of participation to the shared support server 6 (step S63). As a result, the transmission / reception unit 61 of the shared support server 6 receives the user ID of each prospective participant and information indicating the presence / absence of participation.

Next, in the shared support server 6, in the schedule management DB 6003, the presence / absence of participation is stored and managed in the participation presence / absence column that has not been entered (step S64). Further, the storage / reading processing unit 69 of the sharing support server 6 reads out the corresponding speaker ID by searching the speaker information management DB 6007 using the user ID of the participant received in step S63 as a search key (step). S65). Then, the transmission / reception unit 61 transmits the speaker ID of the participant read in step S65 to the voice / text conversion server 9 (step S66). As a result, the transmission / reception unit 91 of the voice-text conversion server 9 receives the speaker ID of the participant. The voice-text conversion server 9 uses the information on the speaker ID of the participant received in step S66 as a speaker candidate in the speaker identification process (see FIG. 33 and the like) described later.

As described above, the user A starts the event (here, the policy decision meeting) by using the union (here, the conference room X) and the communication terminal (here, the electronic blackboard 2). When the event is started, the display control unit 24 of the electronic blackboard 2 causes the display 220 to display the display screen 100a as shown in FIG. 29. On the display screen 100a shown in FIG. 29, the menu bar 121, the time information 124 indicating the elapsed time from the start of the event or the remaining time until the end of the event, and the power icon 117 pressed when the power of the electronic blackboard 2 is turned off. It is included. The menu bar 121 is an example of the operation display screen (window) of the Launcher 102. The menu bar 121 includes a plurality of operation icons 125 (125a to 125j) that are selected (pressed) when performing various processes during event execution. The operation icon 125a is selected (pressed) to view detailed information about the event being executed. The operation icon 125b is selected (pressed) when the various external applications 103 are started. The operation icon 125c is selected (pressed) when browsing the file data stored in the specific storage area of the storage unit 2000. The operation icon 125d is selected (pressed) when switching the display of the application display screen of the running external application 103. The operation icon 125e is selected (pressed) when changing the screen size of the application display screen of the external application 103. The operation icon 125f is selected (pressed) when performing various operations related to the event being executed. The operation icon 125g is selected (pressed) when capturing the display screen 100a displayed on the display 220. The operation icon 125h is selected (pressed) when ending the event being executed. The operation icon 125i is selected (pressed) when the browser application 103c is started to perform a search using the browser. The operation icon 125j is selected (pressed) when inputting characters, numerical values, or the like.

Here, the various icons included in the display screen displayed on the electronic blackboard 2 are examples of the “reception area”. The reception area may be not only an image such as an icon or a button, but also a character such as "change" or a combination of the image and the character. The image here may be an image that can be visually recognized by the user, such as an illustration or a pattern, as well as a symbol or a figure. Further, the selection (pressing) of various icons is an example of operations for various icons. The operations for various icons include, for example, an input operation to the display 220 using the electronic pen 2500, a double click or a single click with a mouse which is an example of the input means of the PC 2700, and a keyboard which is an example of the input means of the PC 2700. The input operation used can be mentioned.

Then, as shown in FIG. 30, the user A can hold a conference in the conference room X by using the electronic blackboard 2. The display control unit 24 of the electronic blackboard 2 causes the display 220 to display the menu bar 120 in which the menu bar 121 included in the display screen 100a is reduced and the file data r4 of the material file as the event execution screen R. For example, the user A of the electronic blackboard 2 presses the operation icon 125c, the reception unit 22 accepts the selection of the operation icon 125c, and the display control unit 24 stores the material in the specific storage area of the storage unit 2000. Display the file data r4 of the file. The display control unit 24 of the electronic blackboard 2 stores not only the file data received in step S59, but also the file data stored in advance in the storage unit 2000 or the file data newly generated at the started event. It may be displayed. In this case, the storage / reading processing unit 29 of the electronic blackboard 2 stores the file data generated or updated in the started event in a specific storage area of the storage unit 2000.

<Registration process of execution event history>
Subsequently, the process of registering the execution event history will be described with reference to FIGS. 31 to 36.

31 and 36 are sequence diagrams showing an example of the flow of registration processing of the execution event history of the shared system according to the embodiment. FIG. 32 is a flowchart showing an example of the flow of the voice-text conversion process of the shared system according to the embodiment. FIG. 33 is a flowchart showing an example of the flow of the speaker identification process of the shared system according to the embodiment.

First, the determination unit 25 of the electronic blackboard 2 determines the type of content processing in the started event (step S71). Specifically, when the content is sound data generated by recording by the sound processing unit 23-2, the determination unit 25 determines that the type of content processing is "recording". When the content is image data acquired by a snapshot (capture) by the image processing unit 23-1, the determination unit 25 determines that the processing type of the content is “snapshot”. When the content is the material file data transmitted by the transmission / reception unit 21, the determination unit 25 determines that the processing type of the content is "material transmission".

Next, the transmission / reception unit 21 transmits the registration request information indicating the registration request of the generated content to the sharing support server 6 (step S72). In this case, the transmission / reception unit 21 automatically transmits the registration request information each time the content is generated. The registration request information includes an execution event ID, a user ID of a content sender, content data, and content processing type information. As a result, the transmission / reception unit 61 of the sharing support server 6 receives the registration request information.

Next, the determination unit 65 of the sharing support server 6 determines the type of content processing received based on the type information included in the registration request information received by the transmission / reception unit 61 (step S73). Then, when the transmission / reception unit 61 of the sharing support server 6 determines that the content processing type is "recording" by the determination unit 65, the transmission / reception unit 61 transmits the sound data which is the content data to the voice text conversion server 9 (step). S74). As a result, the transmission / reception unit 91 of the voice-text conversion server 9 receives the sound data. When the type of content processing is other than "recording", the sharing support server 6 shifts to the processing of step S79 without performing the processing of steps S74 to S78.

Next, the conversion unit 93 of the voice-text conversion server 9 converts the sound data received by the transmission / reception unit 91 into text data (step S75). Here, the voice-text conversion process in the voice-text conversion server 9 in step S75 will be described with reference to FIG. 32. First, the conversion unit 93 acquires information indicating the date and time when the sound data was received by the transmission / reception unit 91 (step S75-1). The information acquired in step S75-1 may be information indicating the date and time when the shared support server 6 received the sound data or the date and time when the shared support server 6 transmitted the sound data. In this case, the transmission / reception unit 91 of the voice-text conversion server 9 receives the sound data transmitted from the sharing support server 6 and the information indicating the date and time in step S74.

Next, the conversion unit 93 executes a text conversion process of the sound data received by the transmission / reception unit 91 (step S75-2). Then, when the text conversion process of the sound data is completed (YES in step S75-3), the conversion unit 93 shifts the process to step S75-4. On the other hand, the conversion unit 93 repeats the process of step S75-2 until the text conversion process of the sound data is completed. Here, in step S75-3, the conversion unit 93 determines that the text conversion process of the sound data is completed when a predetermined amount of text conversion is performed based on the sound data received by the transmission / reception unit 91. To do. For example, when the text conversion for one sentence is performed, the conversion unit 93 determines that the text conversion process for the sound data is completed. Then, the conversion unit 93 generates the text data converted from the sound data (step S75-4). As a result, the voice-text conversion server 9 converts the sound data transmitted from the sharing support server 6 into text data. Since the voice-text conversion server 9 receives the sound data transmitted from the sharing support server 6 at any time, the process shown in FIG. 32 is repeatedly executed.

Here, with reference to FIG. 34, the voice-to-text conversion process described in steps S74, S75 and S77 of FIG. 31 and FIG. 32 will be described in more detail. FIG. 34 is a sequence diagram showing an example of the flow of the voice-text conversion process of the shared system according to the embodiment. In FIG. 34, the process of step S76 in FIG. 31 will be omitted.

As shown in step S74 of FIG. 31, the transmission / reception unit 21 of the electronic blackboard 2 transmits sound data generated by the event being executed to the shared support server 6 at any time (steps Sa1 and Sa2). As a result, the transmission / reception unit 61 of the sharing support server 6 receives the sound data transmitted from the electronic blackboard 2 at any time. Then, the transmission / reception unit 61 of the sharing support server 6 transmits the sound data transmitted from the electronic blackboard 2 to the voice-text conversion server 9 at any time (steps Sb1 and Sb2). As a result, the transmission / reception unit 91 of the voice-text conversion server 9 receives the sound data transmitted from the sharing support server 6 at any time.

Here, the sound data transmitted from the electronic blackboard 2 to the shared support server 6 in step Sa1 corresponds to the sound data transmitted from the shared support server 6 to the voice text conversion server 9 in step Sb1, and corresponds to the sound data transmitted from the shared support server 6 to the voice text conversion server 9 in step Sb1. The sound data transmitted from 2 to the shared support server 6 corresponds to the sound data transmitted from the shared support server 6 to the schedule management server 8 in step Sb2. In this way, the sound data collected and processed by the sound processing unit 23-2 of the electronic blackboard 2 is transmitted to the voice-text conversion server 9 at any time.

The conversion unit 93 of the voice-text conversion server 9 converts the sound data received by the transmission / reception unit 91 into text data. Then, the transmission / reception unit 91 of the voice-text conversion server 9 transmits the text data converted by the conversion unit 93 to the sharing support server 6 (step Sc1). In this case, the text data transmitted in step Sc1 is text data obtained by converting the sound data received in steps Sb1 and Sb2 into text.

Here, the text data transmitted from the voice-text conversion server 9 to the sharing support server 6 in step Sc1 is undetermined text data. When the text conversion of a predetermined amount is performed, the conversion unit 93 of the voice-text conversion server 9 determines that the text conversion of the sound data is completed and the text data is confirmed. For example, when the text conversion for one sentence is performed, the conversion unit 93 determines that the text conversion of the sound data is completed and the text data is confirmed. In step Sc1, since the text data is unconfirmed, the transmission / reception unit 91 has not determined the text data together with the text data (“Inthis flag ear, more”) converted by the conversion unit 93 (undecided). Send an unconfirmed flag to indicate that.

Then, the transmission / reception unit 61 of the sharing support server 6 transmits the text data (unconfirmed text data) transmitted from the voice text conversion server 9 to the electronic blackboard 2 (step Sc2). As a result, the transmission / reception unit 21 of the electronic blackboard 2 receives the text data (unconfirmed text data) transmitted from the sharing support server 6. Then, the display control unit 24 of the electronic blackboard 2 causes the display 508 to display the text data received by the transmission / reception unit 21. The electronic blackboard 2 can display text data as a subtitle display of sound data generated by the event being executed on the event execution screen R as shown in FIG. By receiving and displaying even unconfirmed text data on the electronic blackboard 2, the user of the electronic blackboard 2 is informed that the voice-text conversion process by the voice-text conversion server 9 is being performed at any time. You can check it.

The shared system 1 repeats the processing as shown above (steps Sa3 to Sa7, Sb3 to Sb7, Sc3 to Sc4), so that the sound is collected and processed by the sound processing unit 23-2 of the electronic blackboard 2. Repeat the data text conversion process. The number of times (frequency) that undetermined text data is transmitted by the voice-to-text conversion server 9 can be set as appropriate.

When the transmission / reception unit 91 of the voice text conversion server 9 completes the text conversion of the sound data by the conversion unit 93 and determines that the text data is confirmed, the transmission / reception unit 91 transmits the confirmed text data to the shared support server 6. (Step Sc5). In this case, the transmission / reception unit 91 transmits a confirmation flag indicating that the text data has been confirmed together with the text data converted by the conversion unit 93 (“Inthis fiscal year, more TWICE the last year.”). .. As a result, the transmission / reception unit 61 of the sharing support server 6 receives the text data (confirmed text data) transmitted from the voice-text conversion server 9.

Here, the text data received in step Sc5 is stored and managed in the content management DB 6005 by the process of step S81 in FIG. In this case, the date and time related to the processing of the text data received in step Sc5 is the date and time when the text data was received in step Sc5 from the date and time when the sound data was transmitted in step Sb1. That is, the start date and time of the content processing stored in the content management DB 6005 is the date and time when the sound data is transmitted in step Sb1, and the end date and time of the content processing is the date and time when the text data is received in step Sc5. In this way, the sharing support server 6 starts transmitting the sound data corresponding to the text data to the voice-text conversion server 9 together with the text data (fixed text data) transmitted from the voice-text conversion server 9. The date and time when the text data confirmed from is received can be managed as the start and end date and time of the content processing.

Then, the transmission / reception unit 61 of the sharing support server 6 transmits the text data (confirmed text data) transmitted from the voice-text conversion server 9 to the electronic blackboard 2 (step Sc6). As a result, the transmission / reception unit 21 of the electronic blackboard 2 receives the text data transmitted from the sharing support server 6. Then, the transmission / reception unit 21 of the electronic blackboard 2 causes the display 220 to display the text data transmitted from the sharing support server 6.

In this way, the shared system 1 can convert the sound data generated by the event being executed into text data by using the voice-text conversion server 9, and also provides information on the generation time of the text data (content processing). The start date and time and the end date and time) can be stored and managed in the shared support server 6 and the schedule management server 8.

Returning to FIG. 31, the description of the execution event history registration process will be continued. The speaker identification unit 95 identifies the speaker of the voice related to the sound data received by the transmission / reception unit 91 (step S76). Here, the speaker identification process in the voice-to-text conversion server 9 in step S76 will be described with reference to FIG. 33.

First, the storage / reading processing unit 99 reads the user voice data of the speaker candidate of the event being executed from the user voice information management DB 9001 (step S76-1). Specifically, the storage / reading processing unit 99 searches the user voice information management DB 9001 (see FIG. 16) using the speaker ID of the participant received in step S66 as a search key, and thereby corresponds to the corresponding user. Read voice data.

Next, the speaker identification unit 95 extracts the user voice data corresponding to the sound data received in step S74 from the user voice data read in step S76-1 (step S76-2). .. That is, the speaker identification unit 95 compares the user voice data read in step S76-1 with the sound data received in step S74. Here, for the extraction process of the user voice data by the speaker identification unit 95, for example, a machine learning program by artificial intelligence (AI) is used. The speaker identification unit 95 determines the degree of agreement with the user voice data stored in advance based on the characteristics such as voice pitch, voice quality, intonation, emphatic consonant, sound length, and rhythm contained in the received sound data. Determine and extract the user voice data with the highest degree of matching. Since the extraction process of the user voice data by the speaker identification unit 95 uses a general-purpose technique, detailed description thereof will be omitted. Further, the extraction process of the user voice data by the speaker identification unit 95 is not limited to the one by machine learning.

Next, the speaker identification unit 95 determines whether or not the corresponding user voice data exists in the process of step S76-2 (step S76-3). When the corresponding user voice data exists, the speaker identification unit 95 shifts the process to step S76-4 (YES in step S76-3). Then, the storage / reading processing unit 99 reads out the corresponding speaker ID by searching the user voice information management DB 9001 using the user voice data extracted in step S76-2 as a search key (step S76-4). ). As a result, the voice-text conversion server 9 can identify the speaker ID of the received sound data.

On the other hand, the speaker identification unit 95 ends the process when the corresponding user voice data does not exist (NO in step S76-3). In this way, the voice-text conversion server 9 can specify the speaker ID of the sound data generated by the event by using the user voice data registered in advance. Further, the shared system 1 transmits the information of the participants of the event to the voice-text conversion server 9 that performs the speaker identification process, so that only the participants of the event are candidates for the speaker to be used for the speaker identification process. Can be done. As a result, the voice-to-text conversion server 9 can limit the speaker candidates, which conventionally had to be all registered user voice data, to only the participants of the event, so that the speaker identification process can be performed. It can be improved to improve the accuracy.

The order of the voice-to-text conversion process of FIG. 32 and the speaker identification process of FIG. 33 may be changed back and forth, or may be performed in parallel. Further, in steps S75 and S76 of FIG. 31, and FIGS. 32 to 34, an example in which the voice-text conversion server 9 executes both the voice-text conversion process and the speaker identification process has been described. It may be configured to be executed on a server different from the speaker identification process. In this case, the shared support server 6 transmits sound data to a plurality of servers, and the server responsible for each process transmits text data or a speaker ID to the shared support server 6. Of these, the server responsible for the speaker identification process transmits the speaker ID specified for the received sound data to the shared support server 6 at any time. Then, when the determination unit 65 of the shared support server 6 receives the confirmed text data, the speaker ID that has been received most frequently (high proportion) among the speaker IDs that have been received at any time is selected. It is specified as a speaker ID corresponding to the confirmed text data.

Returning to FIG. 31, the description of the execution event history registration process will be continued. The transmission / reception unit 91 transmits the text data converted by the conversion unit 93 and the speaker ID specified by the speaker identification unit 95 to the sharing support server 6 (step S77). Here, the transmission / reception unit 91 transmits the text data and the speaker ID, as well as the information indicating the date and time acquired in step S75-1, to the sharing support server 6. As a result, the transmission / reception unit 61 of the sharing support server 6 receives the text data and the speaker ID. Next, the storage / reading processing unit 69 of the sharing support server 6 reads out the corresponding user ID by searching the speaker information management DB 6007 using the speaker ID received in step S77 as a search key (step S78). ).

Next, the generation unit 64 generates a unique content processing ID for identifying the content processing generated by the event (step S79). In addition, the generation unit 64 generates a URL of content data indicating the content of the content (step S80). Then, the storage / reading processing unit 69 informs the content management DB 6005 (see FIG. 10) of the type of content processing, the start date / time and end date / time of the content processing, and step S79 for each execution event ID received in step S72. The content processing ID generated by the above and the URL indicating the storage destination of the content generated by the step S80 are associated and managed (step S81).

Here, when the type of content processing is "speech-text conversion", the start date and time and end date and time of the content processing are the date and time when the sound data is converted into text data. Here, the date and time when the sound data is converted into text data are the date and time when the sound data was transmitted by the transmission / reception unit 61 of the shared support server 6 and the transmission / reception unit of the shared support server 6, as shown in FIG. It is the date and time when the text data was received by 61. The date and time when the sound data is converted into text data is the date and time when the sound data is received by the transmission / reception unit 91 of the voice text conversion server 9 and the date and time when the text data is transmitted by the transmission / reception unit 91 of the voice text conversion server 9. There may be. When the type of content processing is "voice text conversion", the start date and time and end date and time of the content processing may be the same as the start date and time and end date and time of the content processing related to the sound data to be text-converted. ..

When the types of content processing are "recording", "snapshot", and "material transmission", the start date and time and end date and time of the content processing are determined by the transmission / reception unit 61 of the shared support server 6 as content data (sound data, sound data, It is the date and time when the image data and file data) were received. When the types of content processing are "recording", "snapshot", and "material transmission", the start date and time and end date and time of the content processing are the date and time when the content data is transmitted by the transmission / reception unit 21 of the electronic blackboard 2. It may be. When the type of content processing is "recording", it may be the start date and time and the end date and time when the recording by the sound processing unit 23-2 was performed. Further, when the type of content processing is "snapshot", the start date and time and end date and time of the content processing may be the date and time when the snapshot (capture) was performed by the image processing unit 23-1.

Next, in FIG. 36, the storage / reading processing unit 69 of the sharing support server 6 searches the user authentication management DB 6001 (see FIG. 9A) using the user ID received in step S72 as a search key. Reads the corresponding organization ID (step S91).

Next, the storage / reading processing unit 69 reads the corresponding access ID and access password by searching the access management DB 6002 (see FIG. 9B) using the organization ID read in step S91 as a search key. (Step S92).

Next, the transmission / reception unit 61 transmits the execution event history registration request information indicating the execution event history registration request to the schedule management server 8 (step S93). The registration request information of the execution event history includes the execution event ID received in step S72, the user ID of the sender, the content data, the content processing ID generated in step S77, and the content data generated in step S78. The URL, the access ID and access password read in step S92, and the start date and time and end date and time of the content processing are included. Here, when the text data received in step S77 is transmitted as content data, the transmission / reception unit 61 transmits the user ID read in step S78 together with the text data. As a result, the transmission / reception unit 81 of the schedule management server 8 receives the registration request information of the execution event history.

Next, in the schedule management server 8, the authentication unit 82 authenticates the shared support server 6 by using the access ID and the access password (step S94). Since this authentication is the same process as in step S38, the description thereof will be omitted. Here, the case where it is valid will be continued.

Next, the storage / reading processing unit 89 stores and manages various data (information) received in step S93 in the execution event history management DB 8008 (see FIG. 13B) (step S95). The storage / reading processing unit 89 stores various data (information) in the execution event history management DB 8008 in association with the execution event ID received in step S93. As a result, the schedule management server 8 manages data having the same contents as the shared support server 6 side.

In addition, the generation unit 83 of the schedule management server 8 generates related information in which the content data received in step S93 is associated with each content generation time zone (step S96). Here, the occurrence time zone of the content included in the related information is generated by using the start date and time of the event stored in the event management DB 8005 and the start date and time and end date and time of the content processing stored in the execution event history management DB 8008. To. That is, the content generation time zone indicates the elapsed time from the start date and time of the event at the time when the content is generated in the executed event. Then, the storage / reading processing unit 89 of the schedule management server 8 is generated by the generation unit 83 in association with the execution event ID received in step S93 with respect to the related information management DB 8010 (see FIG. 14B). The related information is stored and managed (step S97). As a result, the schedule management server 8 manages the content data having different types of content processing in association with each content generation time.

Then, the storage / reading processing unit 89 of the schedule management server 8 associates the text information management DB 8012 (see FIG. 15) with the execution event ID received in step S93 with the text data received in step S93. The text information to be included is stored and managed (step S98). Specifically, the generation unit 83 includes a text data and content processing ID received in step S93, a text ID for identifying the text data received in step S93, and a user ID received together with the text data in step S93. In addition, it generates text information including state information. Then, the storage / reading processing unit 89 stores the text information generated by the generation unit 83 in the text information management DB 8012 in association with the execution event ID received in step S93. In this case, the state information included in the text information is given "Original" information indicating that the associated text data has not been edited.

Here, the data format of the text data stored in the sharing support server 6 and the schedule management server 8 will be described with reference to FIG. 35. FIG. 35 is a diagram showing an example of a data file including text data. The data file d1 shown in FIG. 35 is stored and managed in the URL of the content storage destination shown in the content management DB 6005 and the execution event history management DB 8008.

As shown in FIG. 35, the data file d1 contains 10 text data (transscripts). In addition, each text data (0 to 9) includes a sequence number (seq_num) indicating the order in which content processing occurs, text data (text), a start date and time of content processing (start time), and an end date and time of content processing (end_time). ), Content processing ID (id) and speaker information (speaker) are included, respectively. These pieces of information are the same as the information corresponding to the respective items of the content management DB 6005 and the execution event history management DB 8006. Among them, the speaker information (speaker) indicates the user ID of the user specified as the speaker of the sound data corresponding to the text data. The speaker information (speaker) may be configured to show the speaker ID or the user name together with the user ID or in place of the user ID. Further, the number of text data included in the data file d1 is not limited to this.

In this way, the sharing support server 6 and the schedule management server 8 include text data, information regarding the speaker of the sound data corresponding to the text data (user ID), and time information regarding the processing of the text data (content processing). The start date and time and the end date and time) can be associated and managed in the data file d1 as shown in FIG. 35.

As described above, the electronic blackboard 2 can transmit the execution event ID of the event being executed and the content generated by the corresponding event to the schedule management server 8. Further, the schedule management server 8 can store the received contents in the execution event history management DB 8008 for each execution event ID. As a result, the shared system 1 can store the content generated by the executed event for each event.

<Event end processing>
Subsequently, the process of ending the event being executed will be described with reference to FIGS. 37 to 41.

37 and 38 are sequence diagrams showing an example of a flow of event termination processing of the shared system according to the embodiment. FIG. 39 is a diagram showing an event end screen displayed on the electronic blackboard. FIG. 40 is a diagram showing an upload screen of file data displayed on the electronic blackboard. FIG. 41 is a diagram showing an upload completion screen of file data displayed on the electronic blackboard.

In FIG. 37, when the user A closes the screen displayed on the display 220, the reception unit 22 accepts the end of the event being executed (step S301).

Then, the transmission / reception unit 21 transmits the execution event start / end information indicating the start date / time and the end date / time of the execution event, and the file data registration request information indicating the file data registration request to the shared support server 6 (step). S302). The execution event start / end information includes the execution event ID, the event name, the start date / time of the event, and the end date / time. Further, the registration request information of the file data includes the execution event ID, the user ID of the transmission source, and the file data. As a result, the transmission / reception unit 61 of the sharing support server 6 receives the execution event start / end information and the file data registration request information.

Next, the generation unit 64 of the sharing support server 6 generates a unique content processing ID for identifying the content processing generated by the event (step S303). In addition, the generation unit 64 generates a URL of content data indicating the content of the content (step S304). Then, the storage / reading processing unit 69 refers to the content management DB 6005 (see FIG. 10A) for each execution event ID received in step S302, such as the type of content processing, the start date / time and end date / time of content processing. , The content processing ID generated in step S303, and the URL indicating the storage destination of the content generated in step S304 are associated and managed (step 305).

Next, the storage / reading processing unit 69 of the sharing support server 6 responds by searching the user authentication management DB 6001 (see FIG. 9A) using the user ID received in step S72 as a search key. Read the tissue ID (step S306).

Next, the storage / reading processing unit 69 reads the corresponding access ID and access password by searching the access management DB 6002 (see FIG. 9B) using the organization ID read in step S91 as a search key. (Step S307).

Next, in FIG. 38, the transmission / reception unit 61 transmits the execution event start / end information received in step S302 and the file data registration request information indicating the file data registration request to the schedule management server 8 ( Step S308). The file data registration request information includes the execution event ID received in step S302, the source user ID and file data, the content processing ID generated in step S303, and the URL of the file data generated in step S304. , The access ID and access password read in step S307, and the start date and time and end date and time of the content processing. As a result, the transmission / reception unit 81 of the schedule management server 8 receives the execution event start / end information and the file data registration request information.

Next, in the schedule management server 8, the authentication unit 82 authenticates the shared support server 6 by using the access ID and the access password (step S309). Since this authentication is the same process as in step S38, the description thereof will be omitted. Here, the case where it is valid will be continued.

Next, the storage / reading processing unit 89 of the schedule management server 8 stores the execution event start / end information received in step S308 in the execution event management DB 8009 (see FIG. 14A) (step S310). .. In this case, the storage / reading processing unit 89 adds execution event start / end information for one record to the execution event management table of the execution event management DB 8009.

Next, the storage / reading processing unit 89 stores and manages various data (information) received in step S308 in the execution event history management DB 8008 (step S311). The storage / reading processing unit 89 stores various data (information) including file data in the execution event history management DB 8008 (see FIG. 13B) in association with the execution event ID received in step S308. As a result, the schedule management server 8 manages the same file data as the shared support server 6 side.

Next, the transmission / reception unit 81 transmits the file data registration information indicating that the file data has been registered to the sharing support server 6 (step S312). As a result, the transmission / reception unit 61 of the sharing support server 6 receives the file data registration information.

Then, the transmission / reception unit 61 of the sharing support server 6 transmits the file data registration information transmitted from the schedule management server 8 to the electronic blackboard 2 (step S313). As a result, the transmission / reception unit 21 of the electronic blackboard 2 receives the file data registration information.

Next, when the file data registration information is received by the transmission / reception unit 21, the storage / reading processing unit 29 of the electronic blackboard 2 deletes the file data stored in the specific storage area of the storage unit 2000 (step S314). .. As a result, the electronic blackboard 2 can reduce the risk of leakage of conference information and the like by deleting the file data transmitted to the sharing support server 6.

Here, the screen transition of the screen displayed on the electronic blackboard 2 at the end of the event being executed will be described. First, in step S301, when the end of the event being executed by the reception unit 22 is received, the display control unit 24 displays the event end screen 270 as shown in FIG. 39 on the display 220. .. The event end screen 270 has a file display area 272, a file upload selection area 273, a "Close" button 278 that is pressed when proceeding with the end processing of the event being executed, and a case of canceling the end processing of the event being executed. Includes a "CANCEL" button 279 that is pressed to. In the file display area 272, file data images 272a, 272b, and 272c for identifying the file data of the material file stored in the specific storage area of the storage unit 2000 are displayed. The file upload selection area 273 includes a check box for accepting a selection as to whether or not to upload the file data corresponding to the file data image displayed in the file display area 272 to the sharing support server 6.

When the reception unit 22 accepts the selection of the "Close" button 278 while the file upload selection area 273 is selected, the display control unit 24 displays the file upload screen 280a as shown in FIG. 40 on the display 220. Is displayed. The file upload screen 280a is displayed on the display 220 when the file data stored in the specific storage area of the storage unit 2000 is uploaded to the sharing support server 6. On the file upload screen 280a, the event name 281 of the event to end, the end date and time 282 of the event, the file data upload progress display area 283, and the "CANCEL" button 288 pressed when interrupting (stopping) the file data upload are displayed. include. In the upload progress display area 283, the number of file data to be uploaded (for example, "3" shown in FIG. 40) and the number of file data for which upload has been completed (for example, "0" shown in FIG. 40). ) Is displayed.

Then, when the upload of the file data is completed, the display control unit 24 displays the upload completion screen 280b as shown in FIG. 41. The upload completion screen 280b includes a "Close" button 288 that is pressed when the event is terminated. When the upload completion screen 280b is displayed on the display 220, as shown in step S314, the storage / reading processing unit 29 of the electronic blackboard 2 stores the uploaded file data in a specific storage area of the storage unit 2000. Remove from.

On the other hand, when the file upload screen 280a is displayed on the display 220 and the file data upload fails, the display control unit 24 receives information (for example, a file) for identifying the file data for which the upload failed. First name) is displayed. As a result, for example, when a failure of the communication network 10 occurs, the event participants can print the file data created or edited at the event, or store the file data in the USB memory 2600 connected to the electronic blackboard 2. By doing so, you can bring back the file data that failed to be uploaded.

Further, when the file data remains in the specific storage area of the storage unit 2000 even after the event ends, the storage / reading processing unit 29 of the electronic whiteboard 2 starts, for example, the next event using the electronic whiteboard 2. Sometimes, the file data stored in a specific storage area is deleted. As a result, the electronic blackboard 2 can reduce the risk of leakage of conference information and the like.

<Browsing process of execution event history>
42 and 43 are sequence diagrams showing an example of a flow of browsing processing of the execution event history of the shared system according to the embodiment. FIG. 44 is a diagram showing an execution event list screen displayed on the PC according to the embodiment. 45 and 46 are diagrams showing an execution event history screen displayed on the PC according to the embodiment. The process of browsing the execution event history will be described with reference to FIGS. 42 to 46. The processes of steps S111 to S117 shown in FIG. 42 are the same as the processes of steps S11 to S17 shown in FIG. 17, respectively.

When the user presses the "execution event history viewing" button 543 on the initial screen 540 shown in FIG. 19, the reception unit 52 accepts the viewing of the execution event history (step S118).

Then, the transmission / reception unit 51 transmits the execution event history browsing request information indicating the execution event history browsing request to the schedule management server 8 (step S119). As a result, the transmission / reception unit 81 of the schedule management server 8 receives the execution event history viewing request information.

Next, in the schedule management server 8, the storage / reading processing unit 89 reads the execution event start / end information stored in the execution event management DB 8009 (step S120). The execution event start / end information includes the execution event ID, the event name, the start date / time, and the end date / time.

Then, the transmission / reception unit 81 transmits the execution event ID, the event name, the start date / time, and the end date / time read in step S120 to the PC 5 (step S121). As a result, the transmission / reception unit 51 of the PC 5 receives the execution event ID, the event name, the start date / time, and the end date / time.

Next, the generation unit 56 of the PC 5 uses the data (information) received in step S121 to generate the execution event list screen 570 as shown in FIG. 44 (step S122). Then, in the PC 5, the display control unit 54 displays the execution event list screen 570 generated by the generation unit 56 on the display 508 (step S123). As shown in FIG. 44, the execution event list screen 570 includes information 571 to 573 indicating the execution event. For example, the information 571 to 573 indicating the execution event displays the execution event name, the start date and time of the event, and the end date and time. The information 571 to 573 indicating the execution event is an example of the history information selection area that accepts the selection for displaying the history information indicating the execution content of the event. Further, the lower right portion of the execution event list screen 570 includes a "close" button 575 for closing the execution event list screen 570.

Next, the reception unit 52 of the PC 5 accepts the selection of the execution event included in the execution event list screen 570 (step S124). Specifically, when the user selects any of the information 571 to 573 indicating the execution event included in the execution event list screen 570, the reception unit 52 accepts the selection of the information indicating the execution event. Then, the transmission / reception unit 51 of the PC 5 transmits the execution event ID of the execution event selected by the reception unit 52 to the schedule management server 8 (step S125). As a result, the transmission / reception unit 81 of the schedule management server 8 receives the execution event ID.

Next, the storage / reading processing unit 89 of the schedule management server 8 reads the corresponding execution event history information by searching the execution event history management DB 8008 using the execution event ID received in step S125 as a search key (step). S126). The execution event history information includes the content processing ID, the type of content processing, the start date and time and the end date and time of the content processing.

Further, the storage / reading processing unit 89 of the schedule management server 8 reads out the corresponding related information by searching the related information management DB 8010 using the execution event ID received in step S125 as a search key (step S127). This related information includes a content generation time zone, a content processing ID, a content processing type, and a sequence number. The time zone in which the content included in the related information occurs is an example of the time information.

Then, the storage / reading processing unit 89 reads out the content data indicating the content of the execution event history from the storage destination of the content data indicating the content of the execution event history read in step S126 (step S128). Then, the transmission / reception unit 81 transmits the read related information and content data to the PC 5 (step S129). Here, when the text data is transmitted as the content data, the transmission / reception unit 81 transmits the information of the user name identified by the user ID managed by the text information management DB 8012 together with the text data. Here, the user name information is a user name read by searching the user management DB 8002 (see FIG. 11B) using the user ID managed by the text information management DB 8012 as a search key. .. As a result, the transmission / reception unit 51 of the PC 5 receives the related information, the content data, and the execution event history information.

Next, the sound control unit 58 of the PC 5 sets the reproduction location of the sound data, which is the content data received in step S129 (step S130). In this case, the sound control unit 58 sets the portion of the received related information corresponding to the sound data associated with the content generation time zone “0:00:00” as the reproduction portion of the sound data.

Next, the generation unit 56 of the PC 5 uses the related information and the content data received in step S129 to generate the execution event history screen 580 as shown in FIG. 45 (step S131). As shown in FIG. 45, the execution event history screen 580 shows the data of the content generated by the executed event for each type of content processing. On the execution event history screen 580, a playback location display area 581 for displaying the playback location of the recorded data, a speaker display candidate selection area 670 for selecting a speaker of the recorded data to be displayed as voice text, and a voice text are displayed. It includes a text data display area 582, a snapshot display area 583 for displaying snapshots, and a "material file" button 585 that is pressed when viewing materials used in an event. More specifically, the generation unit 56 causes the execution event to display the image related to the text data, which is the received content data, in the text data display area 582 in the order of the sequence numbers included in the received related information. Generate a history screen 580. In addition, the generation unit 56 generates an execution event history screen 580 so that an image showing speaker information (user name) corresponding to the text data is displayed in association with each of the images related to the text data. Further, the generation unit 56 displays the execution event history screen 580 in the snapshot display area 583 in the order of the sequence numbers included in the received related information so that the image related to the screen data which is the received content data is displayed. Generate. Further, the generation unit 56 displays the execution event history screen so that the reproduction point 581p included in the reproduction location display area 581 is displayed at the position corresponding to the reproduction portion set by step S130 in the total reproduction time of the sound data. Generate 580.

Then, the display control unit 54 of the PC 5 displays the execution event history screen 580 generated by the generation unit 56 on the display 508 (step S132). Further, the sound control unit 58 of the PC 5 reproduces the sound data of the reproduction portion set in step S130.

Here, in the reproduction location display area 581, a reproduction point 581p indicating the reproduction portion in the total reproduction time, a slider 581b indicating the reproduction history (reproduction process) of the recorded data, and a reproduction time display image indicating the reproduction time in the total reproduction time 581t is included. The reproduction point 581p and the slider 581b form a seek bar. The seek bar is an operation area for displaying the reproduction location of the recorded data and for designating the reproduction location of the recorded data. The user can visually grasp which part of the recorded data is being reproduced from the beginning to the end by the positions of the reproduction point 581p and the slider 581b. Further, the user can move the reproduction point 581p by using an input means such as a mouse 507, whereby the recorded data can be reproduced from an arbitrary reproduction point. The reproduction point 581p is an example of a reproduction location specific image. The seek bar including the reproduction point 581p and the slider 581b is an example of the reproduction history display image.

Further, the speaker display candidate selection area 670 includes speaker display candidates 671 (671a to 671e) indicating candidates for speakers to be selected. The speaker display candidate 671 indicates the name (user name) of each participant of the event. Of these, the speaker display candidate 671e (“unknown”) is associated with the text data that the speaker could not identify.

Further, in the text data display area 582, images 582a, 582b, 582c, 582d, and 582e showing the text data are displayed in chronological order from top to bottom. Similarly, in the snapshot display area 583, snapshot (capture) images 583a, 583b, and 583c are displayed in chronological order from top to bottom. In this way, the text data display area 582 and the snapshot display area 583 are displayed in parallel in chronological order. As a result, for example, if the user browses the image 583b, he / she can easily grasp the time zone in which the pie chart was explained at the conference. Therefore, the image showing the text data displayed in parallel with the image 583b. 582c and 582d can be easily found. Here, the images 583a, 583b, and 583c are snapshot images of the entire screen displayed on the display 220.

Further, in the text data display area 582, images 673a to 673e showing speaker information corresponding to each text data are displayed in association with images 582a to 582e showing the text data. The images 673a to 673e showing the speaker information include the information of the user name received together with the text data in step S129.

Here, as shown in FIG. 46, when a specific speaker display candidate 671 (for example, speaker display candidate 671a) is selected using the mouse pointer p3, the reception unit 52 selects the speaker display candidate 671. Accept. Then, the display control unit 54 highlights the text data associated with the speaker information corresponding to the selected specific speaker display candidate 671. In the case of the example of FIG. 46, the display control unit 54 is associated with the images 673d and 673e showing the speaker information (“Riko Taro”) corresponding to the speaker display candidate 671a (“Riko Taro”), and the images 673d and 673e. The images 582d and 582e showing the text data are highlighted. As an example of highlighting, an example of displaying a specific image in different colors has been shown, but the present invention is not limited to this, and any display format that can emphasize a specific image and make the user recognize it. Just do it. Further, on the execution event history screen 580 shown in FIG. 46, an example in which both the images 673d and 673e showing speaker information and the images 582d and 582e showing text data are highlighted has been described. The 54 may have a configuration in which only the images 673d and 673e showing speaker information are highlighted, or a configuration in which only the images 582d and 582e showing text data are highlighted. Further, the display control unit 54 may be configured to filter and display the text data associated with the speaker information corresponding to the selected specific speaker display candidate 671.

As a result, the shared system 1 can easily share the execution event history generated by the executed event among the members (participants) who participated in the event. Further, the PC 5 displays the text data included in the execution event history in association with the speaker information of the sound data corresponding to the text data, so that the contents of the text data can be displayed without re-listening to the sound data. It is possible to let the user confirm who is saying. Further, the PC 5 highlights the text data associated with the speaker information corresponding to the selected specific speaker display candidate 671, so that the text data related to the speaker's remark that the user wants to confirm can be displayed. It can be emphasized and confirmed by the user. The shared system 1 may have a configuration in which the user can edit the reference authority for viewing the execution event history by registering or changing the user after executing the event.

<Selection process of execution event history>
47, 50, and 51 are sequence diagrams showing an example of a flow of selection processing of the execution event history of the shared system according to the embodiment. 48 and 49 are diagrams showing an execution event history screen displayed on the PC according to the embodiment. The execution event history selection process will be described with reference to FIGS. 47 to 51.

First, the reception unit 52 of the PC 5 accepts the selection of a specific image included in the text data display area 582 of the execution event history screen 580 (step S151). Specifically, for example, as shown in FIG. 48, the user uses the mouse pointer p3 to display an image (for example, image 582t) showing specific text data included in the text data display area 582 of the execution event history screen 580. When selected by using, the reception unit 52 accepts the selection for a specific image included in the text data display area 582.

The transmission / reception unit 51 of the PC 5 transmits the selection information related to the selection received by the reception unit 52 to the schedule management server 8 (step S152). Here, the selection information includes a content processing ID corresponding to a specific image (for example, image 582t) for which selection has been accepted. As a result, the transmission / reception unit 81 of the schedule management server 8 receives the selection information transmitted from the PC 5.

Next, the storage / reading processing unit 89 of the schedule management server 8 reads out the corresponding related information by searching the related information management DB 8010 using the content processing ID included in the selection information received in step S152 as a search key. (Step S153). More specifically, the storage / reading processing unit 89 reads out the related information associated with the content processing ID included in the selection information from the related information management DB 8010. This related information includes a content generation time zone, a content processing ID, a content processing type, and a sequence number. The time zone in which the content included in the related information occurs is an example of the time information. As a result, the schedule management server 8 can identify the content generated in the same time zone as the image for which the user has received the selection from the data (information) related to the various contents included in the related information.

Further, the storage / reading processing unit 89 of the schedule management server 8 searches the execution event history management DB 8008 using the content processing ID included in the related information read in step S153 as a search key, and thereby performs the corresponding execution event history. Read the information (step S154). The execution event history information includes the content processing ID, the type of content processing, and the content of the content.

Then, the storage / reading processing unit 89 reads out the content data indicating the content of the execution event history from the storage destination of the content data indicating the content of the content included in the execution event history information read in step S154 (step S155). ). Then, the transmission / reception unit 81 transmits the read related information and content data to the PC 5 (step S156). As a result, the transmission / reception unit 51 of the PC 5 receives the related information and the content data.

Next, the sound control unit 58 of the PC 5 sets the reproduction location corresponding to the sound data, which is the content data received in step S156, as the reproduction location of the sound data (start time of the sound data) (step S157).

Next, the generation unit 56 of the PC 5 generates the execution event history screen 580 based on the content data received in step S156 (step S158). Specifically, the generation unit 56 generates the execution event history screen 580 so that the image related to the text data, which is the received content data, is displayed in the text data display area 582. In addition, the generation unit 56 generates an execution event history screen 580 so that an image showing speaker information (user name) corresponding to the text data is displayed in association with each of the images related to the text data. In addition, the generation unit 56 generates the execution event history screen 580 so that the image related to the screen data, which is the received content data, is displayed in the snapshot display area 583.

Further, the generation unit 56 executes the execution event history screen 580 so that the reproduction point 581p included in the reproduction location display area 581 is displayed at the position related to the sound data set in step S157 in the total reproduction time of the sound data. To generate.

The display control unit 54 displays the execution event history screen 580 generated in step S158 on the display 508 (step S159). FIG. 49 shows an execution event history screen 580 displayed on the display 508 in step S159. In the execution event history screen 580 shown in FIG. 49, the position of the reproduction point 581p included in the reproduction location display area 581 is moved to the time position corresponding to the image 582t selected by the user. In this way, the PC 5 sets the time position indicated by the generation time zone of the content related to the selected specific image by selecting the specific image from the images related to the text data obtained by converting the sound data into text. The reproduction point 581p is moved. As a result, the PC 5 can reproduce the sound data of the reproduction portion corresponding to the image only by selecting the image related to the text data for which the user wants to confirm the voice. For example, the user reproduces the sound data related to the remark of a specific speaker by selecting the image related to the text data associated with the specific speaker information highlighted on the execution event history screen 580. can do. Further, the PC 5 moves the playback point 581p in correspondence with the movement of the image related to the text data, so that the user can easily grasp which playback point of the total playback time the sound data is being played back. Can be made to.

Further, the processes shown in FIGS. 50 and 51 are processes when content data (execution event history) whose type is different from the text data (execution event history) shown in FIG. 47 is selected. FIG. 50 shows a process when an image related to screen data displayed in the snapshot display area 583 is selected. First, the reception unit 52 of the PC 5 accepts the selection of a specific image included in the snapshot display area 583 of the execution event history screen 580 (step S161). Specifically, for example, when the user selects a specific image (for example, snapshot image 583r) included in the snapshot display area 583 of the execution event history screen 580 by using the mouse pointer p3, the reception unit 52 Accepts selections for a particular image contained in the snapshot display area 583. The processing of steps S162 to S169 is the same as the processing of steps S152 to S159 shown in FIG. 47, respectively.

Further, FIG. 51 shows a process when the reproduction point 581p displayed in the reproduction portion display area 581 is selected. First, the reception unit 52 of the PC 5 accepts the movement of the reproduction point 581p included in the reproduction location display area 581 of the execution event history screen 580 (step S181). Specifically, for example, when the user moves the playback point 581p included in the playback location display area 581 of the execution event history screen 580 using the mouse pointer p3, the reception unit 52 moves to the playback location display area 581. Accepts the movement of the included playback point 581p.

The transmission / reception unit 51 of the PC 5 transmits the selection information related to the movement received by the reception unit 52 to the schedule management server 8 (step S182). Here, the selection information includes the content processing ID related to the sound data of the received reproduction destination of the moving destination. As a result, the transmission / reception unit 81 of the schedule management server 8 receives the selection information transmitted from the PC 5. Since the processes of steps S182 to S189 are the same as the processes of steps S152 to S159 shown in FIG. 47, the description thereof will be omitted.

<Editing process of text data>
Subsequently, the process of editing each text data displayed on the execution event history screen will be described with reference to FIGS. 52 to 58.

FIG. 52 is a sequence diagram showing an example of the flow of the text data editing process of the shared system according to the embodiment. 53 and 56 are diagrams showing an execution event history screen displayed on the PC according to the embodiment. FIG. 54 is a diagram showing a text editing screen displayed on the PC according to the embodiment. FIG. 55 is a conceptual diagram showing an updated text information management table. It is assumed that the execution event history screen 580 as shown in FIG. 45 is displayed on the display 508 of the PC 5.

First, when the user presses the text data display area 582 using the mouse pointer p3, the display control unit 54 of the PC 5 displays the execution event history screen 580a shown in FIG. 53 (step S201). Specifically, when the user presses an image (for example, image 582c) indicating specific text data included in the text data display area 582 using the mouse pointer p3, the reception unit 52 selects the specific text data. Accept. Then, the display control unit 54 displays the execution event history screen 580a including the "edit" button 620 displayed at the position corresponding to the specific text data for which the selection has been accepted. This "edit" button 620 is pressed when requesting editing of the selected specific text data. The "edit" button 620 is an example of an edit request image.

Next, when the user presses the "edit" button 620 using the mouse pointer p3, the reception unit 52 accepts the selection of the "edit" button 620 and accepts the text data edit request (step S202). Next, the display control unit 54 causes the text editing screen 630 as shown in FIG. 54 to be displayed on the execution event history screen 580a displayed on the display 508 (step S203). The text editing screen 630 is an editing screen that accepts editing of the contents of the selected specific text data (for example, the text data shown in the image 582c). On the text editing screen 630, an input area 631 for editing specific text data, a "save" button 633 pressed when saving the specific text data edited in the input area 631, and a text data editing process. "Cancel" button 635 pressed when interrupting (stopping), "Play" button 637 pressed when playing sound data corresponding to specific text data to be edited, associating with specific text data The speaker information 638 and the speaker selection menu 639 for changing (editing) the speaker information 638 are included.

Next, when the user inputs to the input area 631, the reception unit 52 accepts editing of specific text data or speaker information (step S204). Thereby, the user can edit the selected specific text data by inputting to the input area 631. When the reception unit 52 accepts the selection of the "play" button 637, the sound control unit 58 reproduces the sound data corresponding to the specific text data to be edited. Further, the user edits the speaker information 638 using the speaker selection menu 639. The speaker information 638 shown on the text editing screen 630 shown in FIG. 54 is a state in which the speaker information has been changed (edited) from "unknown" to "Reiwa Saburo". As a result, the user can edit the text data or the speaker information while listening to the sound data corresponding to the specific text data to be edited.

Next, when the reception unit 52 accepts the selection of the "save" button 633, the transmission / reception unit 51 of the PC 5 transmits the text information update request information indicating the update request of the text information to the schedule management server 8 ( Step S205). The text data and speaker information edited in step S204, the content processing ID received in step S156, and the user ID received in step S112 are transmitted to the text information update request information. The content processing ID in this case is a content processing ID corresponding to the edited text data. As a result, the transmission / reception unit 81 of the schedule management server 8 receives the text information update request information.

The storage / reading processing unit 89 of the schedule management server 8 stores the text data received in step S205 in the text information management DB 8012 (see FIG. 15) (step S206). In this case, the text information management DB 8012 stores a text information management table as shown in FIG. 55. To the text information management table of FIG. 55, text information including the edited text data and speaker information received in step S205 is added to the text information stored in the text information management table shown in FIG. There is. This text information includes a content processing ID similar to the text data before editing, a text ID (for example, text12.2) whose version has been updated from the text ID of the text data before editing (for example, text12), and text data. Is associated with status information indicating that has been edited. Of these, the state information includes an editor name indicating the editor who edited the text data and an edit date and time indicating the date and time when the text data was edited.

The transmission / reception unit 81 of the schedule management server 8 transmits the text data and speaker information stored in step S206 to the PC 5 (step S207). As a result, the transmission / reception unit 51 of the PC 5 receives the text data and speaker information transmitted from the schedule management server 8.

The generation unit 56 of the PC 5 uses the text data received in step S207 to generate the execution event history screen 580b shown in FIG. 56 (step S208). More specifically, the generation unit 56 generates the execution event history screen 580b in which the unedited text data included in the execution event history screen 580a is changed to the edited text data received in step S207.

Then, the display control unit 54 of the PC 5 displays the execution event history screen 580b generated by the generation unit 56 on the display 508 (step S209). As shown in FIG. 53, an image 582c showing the edited text data is displayed in the text data display area 582 included in the execution event history screen 580b. Further, the execution event history screen 580b includes an "edited" button 625 indicating that the text data shown in the image 582c has been edited. The “REDACTED” button 625 is an example of an edited image. As a result, the user can grasp whether or not the text data displayed in the text data display area 582 has been edited by checking the "edited" button 625 included in the execution event history screen 580b. can do.

As described above, the PC 5 causes the user to edit the text data or speaker information displayed on the execution event history screen 580, and at the same time, edits the text data or speaker information in place of the text data before editing. By displaying it, the edited text data or speaker information can be displayed as the minutes. Further, even when the conversion accuracy from the sound data to the text data is low, the PC5 allows the user to edit the displayed text data so that the execution event history screen 580 is used as the minutes of the event. It can be used effectively. Further, the PC 5 can display the edited text data so that the user can see the edited text data and reproduce the sound data of the part that the user wants to hear.

FIG. 57 is a sequence diagram showing an example of a flow of browsing processing of the editing history of the shared system according to the embodiment. FIG. 58 is a diagram showing an edit history screen displayed on the PC according to the embodiment. A process of viewing the editing history of the text data displayed on the execution event history screen 580 will be described with reference to FIGS. 57 and 58. It is assumed that the execution event history screen 580b as shown in FIG. 56 is displayed on the display 508 of the PC 5.

First, when the user presses the "edited" button 625, the reception unit 52 of the PC 5 accepts the selection of the "edited" button 625 and accepts the viewing request of the editing history (step S211). Next, the transmission / reception unit 51 transmits the edit history transmission request information indicating the transmission request of the edit history of the text data to the schedule management server 8 (step S212). The edit history transmission request information includes a content processing ID corresponding to the text data that is the target of the browsing request. As a result, the transmission / reception unit 81 of the schedule management server 8 receives the edit history transmission request information.

Next, the storage / reading processing unit 89 of the schedule management server 8 reads out the corresponding text information by searching the text information management DB 8012 using the content processing ID received in step S212 as a search key (step S213). In this case, the storage / reading processing unit 89 reads out the text information included in the updated text information management table as shown in FIG. 55. For example, when the content processing ID received in step S212 is "c269", the storage / reading processing unit 89 reads out the text information including the text IDs "ext12" and "ext12.1." The transmission / reception unit 81 transmits the text information read in step S213 to the PC 5 (step S214). This text information includes a content processing ID, a text ID, text data, and state information. As a result, the transmission / reception unit 51 of the PC 5 receives the text information transmitted from the schedule management server 8.

Then, the display control unit 54 of the PC 5 displays the edit history screen 640 as shown in FIG. 58 on the execution event history screen 580a displayed on the display 508 (step S215). The edit history screen 640 has an original data display area 641 in which text data before editing is displayed, an edit history display area 643 in which text data after editing is displayed, and a “close” for closing the edit history screen 640. Button 649 is included. Of these, the original data display area 641 includes text data associated with the state information indicating "Original" among the text information received in step S214. Further, the edit history display area 643 includes text data associated with the editor name and the state information indicating the edit date and time among the text information received in step S214. In the edit history display area 643, when the corresponding text data is edited a plurality of times, the edited text data display area (644a, 644b) for displaying the plurality of edited text data is indicated in the state information. It is displayed in chronological order based on. Further, in the edit history display area 643, the text data is displayed at a position corresponding to the edited text data display area (644a, 644b) for displaying each text data based on the state information associated with each text data. Editor information (645a, 645b) indicating the editor who made the edit and edit date / time information (646a, 645b) indicating the time when the text data was edited are displayed, respectively.

As a result, the PC 5 can make the user confirm that the text data has been edited by displaying the execution event history screen 580b on the display 508, and can display the edited history of the edited text data in the editing history. It can be confirmed by the user using the screen 640. In addition, the PC 5 displays the information of the editor who edited the text data and the information of the editing date and time when the editing was performed together with the content of the text data, so that the credibility of the edited text data can be improved by the user. Can be confirmed by.

Although FIGS. 42 to 58 have described the case where the execution event history is browsed by the PC 5, the execution event history can be browsed on the electronic blackboard 2 by the same processing. Further, in the processing and screen examples shown in FIGS. 42 to 58, the same processing and the same screen display are performed not only on the PC 5 but also on the electronic blackboard 2, the video conferencing terminal 3, or the car navigation device 4. May be good.

Further, in FIGS. 42 to 58, the case where the image data related to each screen to be displayed on the PC 5 is generated by the generation unit 56 of the PC 5 has been described, but the image data may be generated by the schedule management server 8. .. In this case, in the schedule management server 8, image data is generated by the generation unit 83, and the generated image data is transmitted to the PC 5 by the transmission / reception unit 81.

<Minutes search process>
Subsequently, the process of searching the minutes in the shared system 1 will be described with reference to FIGS. 59 to 62.

FIG. 59 is a sequence diagram showing an example of the flow of the search process of the minutes of the shared system according to the embodiment. FIG. 60 is a diagram showing a minutes search screen displayed on the PC according to the embodiment. FIG. 61 is a diagram showing a minutes search screen including search results displayed on the PC according to the embodiment. FIG. 62 is a diagram showing a minutes search screen including search results displayed on the PC according to the embodiment.

First, when the user operates the keyboard 511 or the like of the PC 5, the display control unit 54 of the PC 5 performs the minutes search screen 590 on the display 508 as shown in FIG. 60. Is displayed (step S241). The minutes search screen 590 includes a period start input field 591a, a period end input field 591b, a calendar button 592a, a calendar button 592b, a keyword input field 593, a speaker input field 594, and a search start button 595. , Search result display area 596, and the like. The period start input field 591a is a field for inputting the start date (period information) of the holding period of the meeting in which the minutes to be searched are generated. The period end input field 591b is a field for inputting the end date (period information) of the holding period of the meeting in which the minutes to be searched are generated. The calendar button 592a is a button for displaying a calendar for assisting the input of the start date in the period start input field 591a. The calendar button 592b is a button for displaying a calendar for assisting the input of the end date in the period end input field 591b. The keyword input field 593 is a field for inputting a keyword for searching the text information of the minutes. The speaker input field 594 is a field for inputting a speaker for the text information of the minutes to be searched. The search start button 595 is a button for executing a search for minutes. The search result display area 596 is an area for displaying the search results of the minutes. At least one of the information input in the period start input field 591a, the period end input field 591b, the keyword input field 593, and the speaker input field 594 is a search condition for executing the search of the minutes.

Next, the user inputs the above information into at least one of the information input in the period start input field 591a, the period end input field 591b, the keyword input field 593, and the speaker input field 594, and sets the search condition. When the designation is made and the search start button 595 is pressed, the reception unit 52 accepts the request for the search of the minutes (step S242). Here, it is assumed that the start date and end date (period information), keywords, and speaker are entered in the period start input field 591a, the period end input field 591b, the keyword input field 593, and the speaker input field 594, respectively. To do.

Then, the transmission / reception unit 51 of the PC 5 transmits the search request information indicating the request for the search of the minutes to the schedule management server 8 (step S243). The search request information includes the information (keyword, speaker, period information) input in step S242. As a result, the transmission / reception unit 81 of the schedule management server 8 receives the search request information.

Next, the search unit 84 of the schedule management server 8 searches the user management DB 8002 using the speaker (user name) included in the search request information received in step S243 as a search key to search the corresponding user. Read the ID (step S244). Next, the search unit 84 responds to the start date and time and end date and time included in the period information by searching the execution event history management DB 8008 using the period information included in the search request information received in step S243 as a search key. The content processing ID to be performed and the execution event ID corresponding to the content processing ID are read out (step S245).

Next, the search unit 84 reads the event name, start date and time, and end date and time corresponding to the execution event ID by searching the execution event management DB 8009 using the execution event ID read in step S245 as a search key (step S246). ). Then, the search unit 84 uses the execution event ID and content processing ID read in step S245, the user ID read in step S244, and the keywords included in the search request information received in step S243 as search keys to text. Text data is read out by searching the information management DB 8012 (step S247). Specifically, the search unit 84 refers to the text information management DB 8012, and reads out the text data including the keyword among the text data corresponding to the execution event ID, the content processing ID, and the user ID.

Next, the transmission / reception unit 81 of the schedule management server 8 makes a combination of the text data read by the search unit 84 and the speaker, event name, start date / time, and end date / time corresponding to the text data for the PC 5. The list information is transmitted (step S248). As a result, the transmission / reception unit 51 of the PC 5 receives the information of the list of the combinations. Then, as shown in FIG. 62, the display control unit 54 causes the search result display area 596 to display the minutes search screen 590 displaying the information of the received list (step S249). The minutes search screen 590 shown in FIG. 62 has "2019/01/01" in the period start input field 591a, "2019/06/01" in the period end input field 591b, "bar" in the keyword input field 593, and the speaker. This is a display example of the search result when the search is executed by inputting "Satoko Ueda" in the input field 594. In the example shown in FIG. 62, the minutes summary display area 596r to 596t showing the summary of each minutes is displayed as the search result in the search result display area 596. Since the search is performed by designating the speaker "Satoko Ueda" in the speaker input field 594, all the keywords in the list included in the search results are based on the voice "Satoko Ueda" uttered. As shown in FIG. 62, of the list displayed in the search result display area 596, as for the text data, only a part of the text data including the keyword may be displayed. As a result, the list can be displayed while securing a space in the search result display area 596, and the user can quickly find the target minutes by checking the words around the keyword.

Further, FIG. 61 shows an example of the list display on the minutes search screen 590 when the search is executed by designating only the start date, the end date, and the keyword. In the example shown in FIG. 61, since the search condition is set so that the speaker is not specified in the speaker input field 594, it is shown that a huge amount of text data corresponding to the specified start date, end date, and keyword has been searched. In the example shown in FIG. 61, in the search result display area 596, minutes summary display areas 596a to 596e showing an outline of each minutes are displayed as a part of the search result. On the other hand, as shown in FIG. 62, by further designating the speaker and searching, it is possible to search based on the remarks of someone in the meeting, and the search results can be narrowed down.

Then, when the user selects any minutes from the list displayed in the search result display area 596 of the minutes search screen 590 by operating the mouse 512 or the like of the PC 5, the reception unit 52 receives the minutes. The selection operation is accepted (step S250). Then, the display control unit 54 displays the selected minutes (step S251). For example, the display control unit 54 displays the entire text data and the speaker, event name, start date and time, and end date and time corresponding to the text data as minutes selected from the list in a dialog box. The display control unit 54 is not limited to displaying a dialog box for displaying the selected minutes. For example, FIG. 45 includes the text data and an image of the speaker in the text data display area 582. The execution event history screen 580 shown in the above may be displayed. As a result, not only the text data including the keyword of the specified speaker but also other text data and snapshots generated in the conference corresponding to the text data can be confirmed, and the user who executed the search can check the text data. It becomes easy to finally judge whether or not the minutes are the target minutes.

In the example of the minutes search screen 590 shown in FIGS. 60 to 62, the minutes are searched by the start date, the end date, the speaker, and the keyword, but the minutes are not limited to this. Instead of the speaker's name, the speaker's e-mail address or the like may be specified for search. In addition to or instead of the start date, end date, and speaker, for example, the union name is used when the union is known, and the event name is used as the search condition when the event name is known. It may be included.

Further, in the above description, the minutes search process is performed on the PC 5, but the present invention is not limited to this, and the minutes search process is performed on the electronic blackboard 2, the video conference terminal 3, or the car navigation device 4. It may be the one to be called. Further, in the case of the video conferencing terminal 3, the searched minutes may be displayed on the display 320 connected to the video conferencing terminal 3.

(Main effects of the embodiment)
As described above, in the shared system 1 according to the present embodiment, the speaker is identified from the voice spoken at the meeting, and the identified speaker is associated with the text data corresponding to the voice to form a part of the minutes. It is supposed to be remembered as. Then, when searching the minutes, at least by specifying the target speaker and keyword as search conditions and executing the search, the minutes corresponding to the speaker and keyword are searched and displayed in a list. It is supposed to be. This makes it possible to search based on someone's remarks during the meeting, narrow down the search results, and efficiently search the minutes of the target meeting.

Further, in the list displayed as the search result, as for the text data, only a part of the text data including the keyword may be displayed. As a result, the list can be displayed while securing a space in the display area of the search result, and the user can quickly find the target minutes by checking the words around the keyword.

(Supplement)
It should be noted that each function of the embodiment described above can be realized by one or a plurality of processing circuits. Here, the "processing circuit" in the present specification is a processor programmed to execute each function by software such as a processor implemented by an electronic circuit, or a processor designed to execute each function described above. ASIC (Application Special Integrated Circuit), DSP (Digital Signal Processor), FPGA (Field Program Gate Array), SoC (System on a Chip), SoC (System on a Chip), GPU, etc.

Further, in the above-described embodiment, when at least one of the functional parts of the electronic blackboard 2, the PC 5, the shared support server 6, the schedule management server 8 and the voice-text conversion server 9 is realized by executing the program, the program is , ROM, etc. are provided in advance. Further, the programs executed by the electronic blackboard 2, PC5, shared support server 6, schedule management server 8 and voice-text conversion server 9 according to the above-described embodiment are CD-CDs in an installable format or an executable format file. Even if it is configured to be recorded and provided on a computer-readable recording medium such as a ROM (Compact Disk Read Only Memory), a flexible disk (FD), a CD-R (Compact Disk-Recordable), a DVD or an SD card, etc. Good. Further, the programs executed by the electronic blackboard 2, PC5, shared support server 6, schedule management server 8 and voice-text conversion server 9 according to the above-described embodiment are stored on a computer connected to a network such as the Internet. It may be configured to be provided by downloading it via a network. Further, the programs executed by the electronic blackboard 2, the PC 5, the shared support server 6, the schedule management server 8, and the voice-text conversion server 9 according to the above-described embodiment are configured to be provided or distributed via a network such as the Internet. You may. Further, the program executed by the electronic blackboard 2, the PC 5, the shared support server 6, the schedule management server 8 and the voice text conversion server 9 according to the above-described embodiment has a module configuration including at least one of the above-mentioned functional units. As the actual hardware, the CPU reads a program from the above-mentioned storage device and executes it, so that each of the above-mentioned functional units is loaded on the main storage device and generated.

1 Shared system 2 Electronic blackboard 3 Video conferencing terminal 4 Car navigation device 5 PC
6 Shared support server 8 Schedule management server 9 Voice text conversion server 10 Communication network 21 Transmission / reception unit 22 Reception unit 23-1 Image processing unit 23-2 Sound processing unit 24 Display control unit 25 Judgment unit 26 Sound control unit 28 Acquisition / provision unit 29 Storage / reading processing unit 51 Transmission / reception unit 52 Reception unit 54 Display control unit 56 Generation unit 58 Sound control unit 59 Storage / reading processing unit 61 Transmission / reception unit 62 Authentication unit 63 Creation unit 64 Generation unit 65 Judgment unit 69 Storage / reading processing unit 81 Transmission / reception unit 82 Authentication unit 83 Generation unit 84 Search unit 89 Storage / reading processing unit 91 Transmission / reception unit 93 Conversion unit 95 Speaker identification unit 97 Generation unit 99 Storage / reading processing unit 2000 Storage unit 5000 Storage unit 6000 Storage unit 6001 User Authentication management DB
6002 Access management DB
6003 Schedule management DB
6005 Content management DB
6007 Speaker information management DB
8000 Storage 8001 User authentication management DB
8002 User management DB
8003 Union management DB
8004 Union reservation management DB
8005 Event management DB
8006 Server authentication management DB
8008 Execution event history management DB
8009 Execution event management DB
8010 Related information management DB
8012 Text information management DB
9000 Storage unit 9001 User voice information management DB

Japanese Unexamined Patent Publication No. 2014-146066

Claims (10)

A communication system having a management server that manages text data converted by a conversion server for sound data recorded in a video conference, and an information processing device.
The management server
A storage processing unit is provided which associates the text data with the speaker who has emitted the sound of the sound data corresponding to the text data and stores the text data in the storage unit as at least a part of the minutes.
The information processing device
It is provided with a first transmission unit that transmits at least a keyword and a speaker specified as search conditions according to an input operation received by the reception unit to the management server.
The management server
From the minutes stored in the storage unit, a search unit that includes a keyword transmitted from the first transmission unit and searches for minutes corresponding to the speaker transmitted from the first transmission unit, and a search unit.
Communication system equipped with. The management server further includes a second transmission unit that transmits the minutes searched by the search unit to the information processing apparatus.
The communication system according to claim 1, wherein the information processing device further includes a display control unit for displaying a list of minutes transmitted from the second transmission unit on the display unit. The communication system according to claim 2, wherein the display control unit displays a part of the text data included in the minutes to be displayed in a list, including the keyword. The first transmission unit transmits the holding period of the video conference specified as the search condition to the management server according to the input operation received by the reception unit.
The search unit includes a keyword transmitted by the first transmission unit from the minutes stored in the storage unit, corresponds to the speaker transmitted from the first transmission unit, and is the first transmission unit. The communication system according to any one of claims 1 to 3, which searches the minutes of a video conference included in the holding period transmitted by. The communication system according to claim 2 or 3, wherein the display control unit displays the entire text data included in the minutes selected by the operation received by the reception unit among the minutes displayed in the list. It also has the conversion server
The conversion server includes a speaker identification unit that identifies the speaker of the sound data by comparing the sound data recorded in the video conference with the user voice data registered in advance.
Any one of claims 1 to 5, wherein the storage processing unit associates the text data with the speaker specified by the speaker identification unit for the sound data corresponding to the text data and stores the text data in the storage unit. The communication system according to paragraph 1. At least the keyword and the speaker specified as the search conditions according to the input operation received by the reception unit are transmitted to the management server that manages the text data converted by the conversion server for the sound data recorded in the video conference. With the transmitter
It is a storage unit in the management server, and is stored in the storage unit that stores the text data as at least a part of the minutes by associating the text data with the speaker who uttered the sound data corresponding to the text data. A display control unit that displays a list of minutes that include the keywords transmitted by the transmission unit searched by the management server and that corresponds to the speaker transmitted by the transmission unit. When,
Information processing device equipped with. It is an information processing method of a communication system having a management server that manages text data converted by a conversion server for sound data recorded in a video conference and an information processing device.
A storage processing step of associating the text data with the speaker who emitted the sound of the sound data corresponding to the text data and storing the text data in the storage unit as at least a part of the minutes.
A transmission step of transmitting at least a keyword and a speaker specified as search conditions according to an input operation received by the reception unit to the management server, and a transmission step.
A search step for searching the minutes stored in the storage unit, including the transmitted keyword, and corresponding to the transmitted speaker, and
Information processing method having. At least the keyword and the speaker specified as the search conditions according to the input operation received by the reception unit are transmitted to the management server that manages the text data converted by the conversion server for the sound data recorded in the video conference. Send step and
It is a storage unit in the management server, and is stored in the storage unit that stores the text data as at least a part of the minutes by associating the text data with the speaker who emitted the voice of the sound data corresponding to the text data. A display control step for displaying a list of minutes, including the transmitted keyword and corresponding to the transmitted speaker, searched by the management server from the minutes.
Information processing method having. Computer,
At least the keyword and the speaker specified as the search conditions according to the input operation received by the reception unit are transmitted to the management server that manages the text data converted by the conversion server for the sound data recorded in the video conference. With the transmitter
It is a storage unit in the management server, and is stored in the storage unit that stores the text data as at least a part of the minutes by associating the text data with the speaker who uttered the sound data corresponding to the text data. A display control unit that displays a list of minutes corresponding to the speaker transmitted from the transmission unit, including the keyword transmitted by the transmission unit searched by the management server, from the minutes. When,
A program to make it work.

Images