JP7347124B2 - Programs, information processing methods, information processing devices, and communication systems - Google Patents

Description

The present invention relates to a program, an information processing method, an information processing device, and a communication system.

BACKGROUND ART There is a known technology that can convert audio data recorded in a video conference into text using a cloud service, automatically generate minutes as minutes, and then modify or delete unnecessary text.

As such a video conference system, a system has been disclosed that converts input voice into text, uses the uttered text as minutes, and searches for highly relevant minutes (for example, Patent Document 1).

However, in conventional video conferencing systems, even if the spoken text is edited and deleted using hidden characters, etc., the audio section corresponding to the deleted spoken text will be invalidated (for example, silence, self-limited sound), etc. There is a problem that it is not possible to perform processing such as replacement, etc. In this case, if audio is recorded in the video conference system, there remains the problem that audio that is not desired to be recorded as minutes is shared without being invalidated.

The present invention has been made in view of the above, and provides a program that, when text that is not desired to be recorded as minutes is replaced with blank text, can invalidate the audio section corresponding to the text that has been blanked out. The purpose is to provide an information processing method, an information processing device, and a communication system.

In order to solve the above-mentioned problems and achieve the objects, the present invention uses a computer to edit text data converted by a conversion server with respect to audio data recorded in a video conference, in an information processing device. an acquisition unit that acquires the sound data and the text data corresponding to the sound data; a first editing unit that edits the text data acquired by the acquisition unit according to an input operation accepted by the reception unit; a first transmitting unit that transmits the text data edited by the first editing unit to a management server; and a determining unit that determines whether or not the text data edited by the first editing unit includes blank characters. , a second step of transmitting to the management server an audio editing request for disabling the sound of the audio section in the sound data corresponding to the text data including the hidden characters, if the determining unit determines that the hidden characters are included; It is characterized in that it functions as a transmitter.

According to the present invention, when text that is not desired to be recorded as minutes is replaced with blank text, the audio section corresponding to the text that is blanked out can be invalidated.

FIG. 1 is a schematic diagram of a shared system according to an embodiment. FIG. 2 is a diagram illustrating an example of the hardware configuration of the electronic whiteboard according to the embodiment. FIG. 3 is a diagram illustrating an example of the hardware configuration of the video conference terminal according to the embodiment. FIG. 4 is a diagram showing an example of the hardware configuration of the car navigation device according to the embodiment. FIG. 5 is a diagram illustrating an example of the hardware configuration of a PC and a server according to the embodiment. FIG. 6 is a diagram illustrating an example of the software configuration of the electronic whiteboard according to the embodiment. FIG. 7 is a diagram illustrating an example of the software configuration of the PC according to the embodiment. FIG. 8 is a functional block diagram of the shared system according to the embodiment. FIG. 9 is a conceptual diagram of the user authentication management table, the access management table, and the schedule management table. FIG. 10 is a conceptual diagram showing a content management table. FIG. 11 is a conceptual diagram showing a user authentication management table, a user management table, and a union management table. FIG. 12 is a conceptual diagram showing a union reservation management table and an event management table. FIG. 13 is a conceptual diagram showing a server authentication management table and an execution event history management table. FIG. 14 is a conceptual diagram showing an execution event management table and a related information management table. FIG. 15 is a conceptual diagram showing the text information management table. FIG. 16 is a sequence diagram illustrating an example of the flow of schedule registration processing in the shared system according to the embodiment. FIG. 17 is a diagram showing the sign-in screen. FIG. 18 is a diagram showing an example of the initial screen of the PC. FIG. 19 is a diagram showing a schedule input screen. FIG. 20 is a sequence diagram illustrating an example of the flow of event initiation processing in the shared system according to the embodiment. FIG. 21 is a diagram showing a sign-in screen displayed on the electronic blackboard. FIG. 22 is a diagram showing a union reservation list screen. FIG. 23 is a sequence diagram illustrating an example of the flow of event initiation processing in the shared system according to the embodiment. FIG. 24 is a diagram showing a detailed event information screen. FIG. 25 is a diagram showing a display screen displayed on the electronic blackboard at the start of an event. FIG. 26 is an image diagram showing how a user uses the electronic blackboard. FIG. 27 is a sequence diagram illustrating an example of the flow of execution event history registration processing in the shared system according to the embodiment. FIG. 28 is a flowchart illustrating an example of the flow of speech-to-text conversion processing in the shared system according to the embodiment. FIG. 29 is a sequence diagram illustrating an example of the flow of speech-to-text conversion processing in the shared system according to the embodiment. FIG. 30 is a diagram showing an example of a data file including text data. FIG. 31 is a sequence diagram illustrating an example of the flow of execution event history registration processing in the shared system according to the embodiment. FIG. 32 is a sequence diagram illustrating an example of the flow of event termination processing in the shared system according to the embodiment. FIG. 33 is a sequence diagram illustrating an example of the flow of event termination processing in the shared system according to the embodiment. FIG. 34 is a diagram showing an event end screen displayed on the electronic blackboard. FIG. 35 is a diagram showing a file data upload screen displayed on the electronic blackboard. FIG. 36 is a diagram showing a file data upload completion screen displayed on the electronic blackboard. FIG. 37 is a sequence diagram illustrating an example of the flow of execution event history viewing processing in the shared system according to the embodiment. FIG. 38 is a sequence diagram illustrating an example of the flow of execution event history viewing processing in the shared system according to the embodiment. FIG. 39 is a diagram showing an execution event list screen displayed on the PC according to the embodiment. FIG. 40 is a diagram showing an execution event history screen displayed on the PC according to the embodiment. FIG. 41 is a sequence diagram illustrating an example of the flow of execution event history selection processing in the shared system according to the embodiment. FIG. 42 is a diagram showing an execution event history screen displayed on the PC according to the embodiment. FIG. 43 is a diagram showing an execution event history screen displayed on the PC according to the embodiment. FIG. 44 is a sequence diagram illustrating an example of the flow of execution event history selection processing in the shared system according to the embodiment. FIG. 45 is a sequence diagram illustrating an example of the flow of execution event history selection processing in the shared system according to the embodiment. FIG. 46 is a sequence diagram illustrating an example of the flow of text data editing processing in the shared system according to the embodiment. FIG. 47 is a diagram showing an execution event history screen displayed on the PC according to the embodiment. FIG. 48 is a diagram showing a text editing screen displayed on the PC according to the embodiment. FIG. 49 is a conceptual diagram showing the updated text information management table. FIG. 50 is a diagram illustrating an overview of the flow of editing processing of sound data in a shared system according to an embodiment. FIG. 51 is a sequence diagram illustrating an example of the flow of sound data editing processing in the shared system according to the embodiment. FIG. 52 is a diagram illustrating an example of a text editing screen displayed on the PC according to the embodiment when editing is done to hide text. FIG. 53 is a diagram showing an execution event history screen displayed on the PC according to the embodiment. FIG. 54 is a sequence diagram illustrating an example of the flow of editing history viewing processing in the shared system according to the embodiment. FIG. 55 is a diagram showing an edit history screen displayed on the PC according to the embodiment.

The shared system 1 according to this embodiment will be described in detail below using the drawings. Note that in this embodiment, "file" refers to "electronic file."

(Summary of system configuration)
FIG. 1 is a schematic diagram of a shared system according to an embodiment. An overview of the configuration of the shared system 1 will be described with reference to FIG. 1.

As shown in FIG. 1, the shared system 1 of this embodiment includes an electronic whiteboard 2, a video conference terminal 3, a car navigation device 4, and a PC (Personal Computer) 5 (an example of an information processing device). , a shared support server 6, a schedule management server 8, and a speech-to-text conversion server 9 (conversion server).

Furthermore, the electronic blackboard 2, video conference terminal 3, car navigation device 4, PC 5, shared support server 6, schedule management server 8, and speech-to-text conversion server 9 can communicate via the communication network 10. The communication network 10 is constructed using the Internet, a mobile communication network, a LAN (Local Area Network), and the like. The communication network 10 may include not only wired communication but also wireless communication networks such as 3G (3rd Generation), WiMAX (Worldwide Interoperability for Microwave Access), or LTE (Long Term Evolution). Good.

Further, the electronic blackboard 2 is used in the conference room X. Video conference terminal 3 is used in conference room Y. Furthermore, the union is subject to reservation by each user. The car navigation device 4 is used in the vehicle α. The vehicle α in this case is a vehicle to be car-shared. Vehicles include cars, motorcycles, bicycles, wheelchairs, etc.

"Common entity" refers to goods, services, spaces (rooms), places, or information that are used jointly by multiple people or groups. Conference room X, conference room Y, and vehicle α are examples of a shared entity shared by a plurality of users. An example of information given to a union is an account. For example, in a particular service provided on the Web, the number of accounts that can be used is limited to one.

The electronic whiteboard 2, video conference terminal 3, and car navigation device 4 are examples of communication terminals. The "communication terminal" is, for example, a terminal that becomes usable upon sign-in by the user (see S32 described below). Note that the communication terminal used in the vehicle α includes not only the car navigation device 4 but also a smartphone or a smart watch on which a car navigation application is installed.

The PC 5 is an example of a display terminal, and is an example of a registration device that registers reservations for use of each union and registers events scheduled to be executed by each user with respect to the schedule management server 8. The event is, for example, a conference, meeting, gathering, getting together, consultation, meeting, driving, riding, or moving.

The shared support server 6 is a computer, and provides support for each communication terminal to remotely share a common body.

The schedule management server 8 is a computer that manages reservations for each union and schedules for each user.

The speech-to-text conversion server 9 is a computer, and converts sound (voice) data received from an external computer (for example, the shared support server 6) into text data. Note that the speech-to-text conversion server 9 may be connected by WebSocekt and realized by an external service (for example, Watson or Bing Speech) that converts speech into text. In this case, when the external service outputs the converted text, it also outputs information on which section of the input audio the converted text corresponds to.

Here, the shared support server 6, schedule management server 8, and speech-to-text conversion server 9 are referred to as a management system. The management system may be, for example, a computer in which all or part of the functions of the shared support server 6, schedule management server 8, and speech-to-text conversion server 9 are integrated. Furthermore, each of the shared support server 6, schedule management server 8, and speech-to-text conversion server 9 may have a configuration in which each function is distributed to a plurality of computers. Furthermore, although the shared support server 6, schedule management server 8, and speech-to-text conversion server 9 will be described as server computers existing in a cloud environment, the shared support server 6 and the speech-to-text conversion server 9 may include electronic blackboards 2, etc. The server may be a server that operates on a communication terminal, or may be a server that exists in an on-premises environment. Further, the schedule management server 8 may be a server existing in an on-premises environment.

(Hardware configuration)
Next, the hardware configuration of the devices or terminals that construct the shared system 1 will be explained using FIGS. 2 to 5.

<Hardware configuration of electronic whiteboard>
FIG. 2 is a diagram illustrating an example of the hardware configuration of the electronic whiteboard according to the embodiment. As shown in FIG. 2, the electronic whiteboard 2 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, and an SSD (Solid State Drive). ive) 204 and , a network I/F 205, and an external device connection I/F (Interface) 206.

Among these, the CPU 201 controls the entire operation of the electronic whiteboard 2 . The ROM 202 stores programs used to drive the CPU 201, such as the CPU 201 and an IPL (Initial Program Loader). RAM 203 is used as a work area for CPU 201. The SSD 204 stores various data such as electronic whiteboard programs. Network I/F 205 controls communication with communication network 10 . External device connection I/F 206 controls communication with a USB (Universal Serial Bus) memory 2600, PC 2700, and external devices (microphone 2200, speaker 2300, camera 2400).

The electronic whiteboard 2 also includes a capture device 211, a GPU 212, a display controller 213, a contact sensor 214, a sensor controller 215, an electronic pen controller 216, a short-range communication circuit 219, an antenna 219a of the short-range communication circuit 219, and a power switch 222. We are prepared.

Of these, the capture device 211 acquires image data displayed on a display 220, which is an example of a display unit (display means), via the display controller 213, and stores it in the RAM 203 or the like. A GPU (Graphics Processing Unit) 212 is a semiconductor chip that specializes in graphics. The display controller 213 controls and manages screen display in order to output the output image from the capture device 211 or the GPU 212 to the display 220 or the like. The contact sensor 214 detects when the electronic pen 2500, the user's hand H, or the like comes into contact with the display 220. Sensor controller 215 controls processing of contact sensor 214. The contact sensor 214 performs coordinate input and coordinate detection using an infrared cutoff method. This method of inputting and detecting coordinates is such that two light receiving and emitting devices installed at both ends of the upper side of the display 220 emit a plurality of infrared rays in parallel to the display 220. This is a method of receiving light that is reflected by a reflecting member and returns along the same optical path as the light emitted by the light receiving element. The contact sensor 214 outputs the ID of the infrared rays emitted by the two light receiving and emitting devices blocked by the object to the sensor controller 215, and the sensor controller 215 identifies the coordinate position that is the contact position of the object. The electronic pen controller 216 communicates with the electronic pen 2500 to determine whether or not the display 220 is touched with the tip of the pen or the end of the pen. The near field communication circuit 219 is a communication circuit such as NFC (Near Field Communication) or Bluetooth (registered trademark). The power switch 222 is a switch for switching the power of the electronic whiteboard 2 ON/OFF.

Further, the electronic whiteboard 2 includes a bus line 210. The bus line 210 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 201 shown in FIG. 2.

Note that the contact sensor 214 is not limited to the infrared cutoff type, but can also be a capacitive type touch panel that identifies the touch position by detecting changes in capacitance, or a touch sensor that identifies the touch position by voltage changes between two opposing resistive films. Various detection means may be used, such as a resistive film type touch panel that detects the electromagnetic induction generated when a touch object contacts the display section, and an electromagnetic induction type touch panel that specifies the contact position. Furthermore, the electronic pen controller 216 may determine whether or not a portion of the electronic pen 2500 that the user grasps or other portions of the electronic pen is touched, in addition to the pen tip and the pen tail of the electronic pen 2500.

<Hardware configuration of video conference terminal>
FIG. 3 is a diagram illustrating an example of the hardware configuration of the video conference terminal according to the embodiment. As shown in FIG. 3, the video conference terminal 3 includes a CPU 301, a ROM 302, a RAM 303, a flash memory 304, an SSD 305, a media I/F 307, an operation button 308, a power switch 309, and a bus. A line 310, a network I/F 311, a CMOS (complementary metal oxide semiconductor) sensor 312, an image sensor I/F 313, a microphone 314, a speaker 315, a sound input/output I/F 316, a display I/F 317, It includes an external device connection I/F 318, a short-range communication circuit 319, and an antenna 319a of the short-range communication circuit 319. Among these, the CPU 301 controls the entire operation of the video conference terminal 3. The ROM 302 stores programs used to drive the CPU 301, such as IPL. RAM 303 is used as a work area for CPU 301. Flash memory 304 stores various data such as communication programs, image data, and sound data. The SSD 305 controls reading and writing of various data to the flash memory 304 under the control of the CPU 301. Note that an HDD may be used instead of the SSD. A media I/F 307 controls reading or writing (storage) of data to a recording medium 306 such as a flash memory. The operation button 308 is a button that is operated when selecting the destination of the video conference terminal 3, for example. The power switch 309 is a switch for switching the power of the video conference terminal 3 ON/OFF.

Further, the network I/F 311 is an interface for data communication using the communication network 10 such as the Internet. The CMOS sensor 312 is a type of built-in imaging means that images a subject and obtains image data under the control of the CPU 301. The image sensor I/F 313 is a circuit that controls driving of the CMOS sensor 312. The microphone 314 is a type of built-in sound collecting means for inputting sound. The sound input/output I/F 316 is a circuit that processes input/output of sound signals between the microphone 314 and the speaker 315 under the control of the CPU 301. Display I/F 317 is a circuit that transmits image data to external display 320 under the control of CPU 301. External device connection I/F 318 is an interface for connecting various external devices. The short-range communication circuit 319 is a communication circuit such as NFC or Bluetooth.

Further, the bus line 310 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 301 shown in FIG. 3.

The display 320 is a type of display unit (display means) configured with a liquid crystal or organic EL that displays an image of a subject, operation icons, and the like. Further, the display 320 is connected to the display I/F 317 via a cable 320c. This cable 320c may be a cable for analog RGB (VGA) signals, a cable for component video, DisplayPort, HDMI (registered trademark) (High-Definition Multimedia Interface), or DVI ( It may also be a cable for Digital Video Interactive) signals.

Note that instead of the CMOS sensor 312, an image sensor such as a CCD (Charge Coupled Device) sensor may be used. External devices such as an external camera, an external microphone, and an external speaker can be connected to the external device connection I/F 318 using a USB cable or the like. When an external camera is connected, the external camera is driven with priority over the built-in CMOS sensor 312 under the control of the CPU 301. Similarly, when an external microphone or external speaker is connected, the external microphone or speaker is given priority over the built-in microphone 314 or the built-in speaker 315, respectively, according to the control of the CPU 301. Driven by external speakers.

Furthermore, the recording medium 306 is configured to be detachable from the video conference terminal 3. Furthermore, as long as it is a nonvolatile memory that reads or writes data under the control of the CPU 301, it is not limited to the flash memory 304, and an EEPROM or the like may be used.

<Hardware configuration of car navigation device>
FIG. 4 is a diagram showing an example of the hardware configuration of the car navigation device according to the embodiment. As shown in FIG. 4, the car navigation device 4 includes a CPU 401, a ROM 402, a RAM 403, an EEPROM 404, a power switch 405, an acceleration/direction sensor 406, a media I/F 408, and a GPS receiver 409. It is equipped with.

Among these, the CPU 401 controls the operation of the car navigation device 4 as a whole. The ROM 402 stores programs used to drive the CPU 401, such as IPL. RAM 403 is used as a work area for CPU 401. The EEPROM 404 reads or writes various data such as programs for the car navigation device 4 under the control of the CPU 401 . The power switch 405 is a switch for switching the power of the car navigation device 4 ON/OFF. The acceleration/direction sensor 406 is a variety of sensors such as an electronic magnetic compass, a gyro compass, and an acceleration sensor that detect geomagnetism. A media I/F 408 controls reading or writing (storage) of data to a recording medium 407 such as a flash memory. GPS receiving section 409 receives GPS signals from GPS satellites.

The car navigation device 4 also includes a long-distance communication circuit 411, an antenna 411a of the long-distance communication circuit 411, a CMOS sensor 412, an image sensor I/F 413, a microphone 414, a speaker 415, and a sound input/output I/F. F416, a display 417, a display I/F 418, an external device connection I/F 419, a short-range communication circuit 420, and an antenna 420a of the short-range communication circuit 420.

Of these, the long-distance communication circuit 411 receives traffic congestion information, road construction information, traffic accident information, etc. provided from infrastructure outside the vehicle, and conversely transmits information such as the vehicle's location and emergency life-saving signals to the outside. This is a circuit for The external infrastructure is, for example, a road information guidance system such as a VICS (registered trademark) (Vehicle Information and Communication System) system. The CMOS sensor 412 is a type of built-in imaging means that images a subject and obtains image data under the control of the CPU 401. The image sensor I/F 413 is a circuit that controls driving of the CMOS sensor 412. The microphone 414 is a type of built-in sound collection means for inputting audio. The sound input/output I/F 416 is a circuit that processes input/output of sound signals between the microphone 414 and the speaker 415 under the control of the CPU 401 . The display 417 is a type of display unit (display means) such as a liquid crystal or organic EL display that displays an image of a subject, various icons, etc., and has a touch panel function. The touch panel is a type of input means for a user to operate the car navigation device 4. Display I/F 418 is a circuit that causes display 417 to display an image. The external device connection I/F 419 is an interface for connecting various external devices. The short-range communication circuit 420 is a communication circuit such as NFC or Bluetooth. The car navigation device 4 also includes a bus line 410. The bus line 410 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 401 shown in FIG. 4.

<Hardware configuration of PC and server>
FIG. 5 is a diagram illustrating an example of the hardware configuration of a PC and a server according to the embodiment. The PC 5 is constructed by a computer, and as shown in FIG. (display device), a network I/F 509, a keyboard 511, a mouse 512, a CD-RW drive 514, a speaker 515, and a bus line 510.

Among these, the CPU 501 controls the operation of the entire PC 5. The ROM 502 stores programs used to drive the CPU 501 such as IPL. RAM 503 is used as a work area for CPU 501. The HD 504 stores various data such as programs. The HDD controller 505 controls reading and writing of various data to the HD 504 under the control of the CPU 501. A media I/F 507 controls reading or writing (storage) of data to a recording medium 506 such as a flash memory. The display 508 displays various information such as a cursor, menu, window, characters, or images. Display 508 is an example of a display unit (display means). The network I/F 509 is an interface for data communication using the communication network 10. The keyboard 511 is a type of input means that includes a plurality of keys for inputting characters, numbers, various instructions, and the like. The mouse 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like. A CD-RW drive 514 controls reading and writing of various data to and from a CD-RW (Compact Disc-ReWritable) 513, which is an example of a removable recording medium. The speaker 515 outputs a sound signal under the control of the CPU 501.

Further, the PC 5 includes a bus line 510. The bus line 510 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 501 shown in FIG. 5.

Further, the shared support server 6 is constructed by a computer, and as shown in FIG. F607, display 608, network I/F 609, keyboard 611, mouse 612, CD-RW drive 614, and bus line 610. These are the CPU 501, ROM 502, RAM 503, HD 504, HDD controller 505, recording medium 506, media I/F 507, display 508, network I/F 509, keyboard 511, mouse 512, CD-RW drive 514, and bus line 510 in the PC 5, respectively. It has the same configuration as .

Furthermore, the schedule management server 8 is constructed by a computer, and as shown in FIG. It includes an F807, a display 808, a network I/F 809, a keyboard 811, a mouse 812, a CD-RW drive 814, and a bus line 810. These are the CPU 501, ROM 502, RAM 503, HD 504, HDD controller 505, recording medium 506, media I/F 507, display 508, network I/F 509, keyboard 511, mouse 512, CD-RW drive 514, and bus line 510 in the PC 5, respectively. It has the same configuration as .

The speech-to-text conversion server 9 is constructed by a computer, and as shown in FIG. /F 907, a display 908, a network I/F 909, a keyboard 911, a mouse 912, a CD-RW drive 914, and a bus line 910. These are the CPU 501, ROM 502, RAM 503, HD 504, HDD controller 505, recording medium 506, media I/F 507, display 508, network I/F 509, keyboard 511, mouse 512, CD-RW drive 514, and bus line 510 in the PC 5, respectively. It has the same configuration as .

Note that each of the above programs may be an installable or executable file and may be recorded on a computer-readable recording medium and distributed. Examples of recording media include CD-R (Compact Disc Recordable), DVD (Digital Versatile Disk), Blu-ray Disc, SD card, and the like. Furthermore, the recording medium can be provided domestically or internationally as a program product.

Furthermore, the shared support server 6 may be constructed by a single computer, or may be constructed by a plurality of computers in which each section (function, means, or storage section) is divided and arbitrarily assigned. The same applies to the schedule management server 8 and speech-to-text conversion server 9.

(Software configuration of electronic whiteboard)
Next, computer software installed on the electronic whiteboard 2 will be explained using FIG. 6. Here, computer software (hereinafter referred to as software) refers to a program related to the operation of a computer and other information used for processing by a computer, which is equivalent to a program. A program is a set of instructions to a computer that are combined to produce a single result. Also, what is said to be similar to a program refers to something that cannot be called a program because it is not a direct command to a computer, but has properties similar to a program in that it specifies computer processing. For example, a data structure (a logical structure of data expressed by mutual relationships between data elements) corresponds to a program.

Further, an application is a general term for software used to perform a specific process among software categories. On the other hand, an operating system (hereinafter referred to as OS) is software that controls a computer and allows applications and the like to use computer resources. The OS performs basic management and control of the computer, such as input/output control, hardware management such as memory and hard disks, and process management. Applications operate using functions provided by the OS.

FIG. 6 is a diagram illustrating an example of the software configuration of the electronic whiteboard according to the embodiment. As shown in FIG. 6, the OS 101, Launcher 102, schedule viewer 103a, file viewer 103b, and browser application 103c operate on the work area 15 of the RAM 203. The OS 101 is basic software that provides the basic functions of the electronic whiteboard 2 and manages the entire electronic whiteboard 2.

Launcher 102 is a launcher application that runs on OS 101. The Launcher 102 manages, for example, the start and end of an event such as a meeting executed on the electronic whiteboard 2, or manages external applications such as the schedule viewer 103a, file viewer 103b, and browser application 103c used during the execution of the event.

The schedule viewer 103a, the file viewer 103b, and the browser application 103c are external applications (hereinafter referred to as external applications 103 unless there is a particular need to distinguish them) that operate on the Launcher 102. The external application 103 is processed independently of the Launcher 102 and executes a service or function provided on the OS 101. Note that although FIG. 6 shows an example in which three external applications, the schedule viewer 103a, the file viewer 103b, and the browser application 103c, are installed on the electronic whiteboard 2, the number of external applications is not limited to this.

(PC software configuration)
FIG. 7 is a diagram illustrating an example of the software configuration of the PC according to the embodiment. The computer software installed on the PC 5 will be explained using FIG. 7. As shown in FIG. 7, the OS 5501, the minutes application 5502a, and the browser 5502b operate on the work area 5500 of the RAM 503. The OS 5501 is basic software that provides the basic functions of the PC 5 and manages the entire PC 5.

The minutes application 5502a and the browser 5502b are applications that use various data sent from the schedule management server 8 to generate and display an execution event history screen that is the minutes of events executed by the electronic blackboard 2. be. Note that although FIG. 7 shows an example in which two external applications, the minutes application 5502a and the browser 5502b, are installed on the PC 5, the number of external applications is not limited to this.

(Functional configuration of shared system)
Next, the functional configuration of this embodiment will be explained using FIGS. 8 to 15.

FIG. 8 is a functional block diagram of the shared system according to the embodiment. Note that, in FIG. 8, among the terminals, devices, and servers shown in FIG. 1, those related to the processing or operation described below are shown.

<Functional configuration of electronic blackboard>
As shown in FIG. 8, the electronic blackboard 2 includes a transmitting/receiving section 21, a receiving section 22, an image/sound processing section 23, a display control section 24, a determining section 25, and an acquisition/providing section 28. , a storage/read processing section 29. Each of these units is a function or a means of functioning that is realized when any of the components shown in FIG. be. Further, the electronic whiteboard 2 has a storage unit 2000 constructed by the RAM 203, SSD 204, or USB memory 2600 shown in FIG.

<<Functional configuration of electronic blackboard>>
Next, each component of the electronic whiteboard 2 will be explained. The transmitting/receiving unit 21 is realized by commands from the CPU 201 shown in FIG. or information).

The receiving unit 22 is mainly realized by commands from the CPU 201 shown in FIG. 2, as well as the contact sensor 214 and the electronic pen controller 216, and receives various inputs from the user.

The image/sound processing unit 23 is realized by the command from the CPU 201 and the capture device 211 shown in FIG. 2, and stores image data displayed on the display 220. Further, the image/sound processing unit 23 is realized by the instructions from the CPU 201 and the GPU 212 shown in FIG. 2, and performs image processing for displaying an image on the display 220. Furthermore, the image/sound processing unit 23 performs image processing on image data obtained by imaging a subject with the camera 2400. Further, after the user's voice is converted into an audio signal by the microphone 2200, the image/sound processing unit 23 performs audio processing on sound data related to this audio signal. Further, the image/sound processing unit 23 outputs an audio signal related to the sound data to the speaker 2300, and causes the speaker 2300 to output audio. The image/sound processing unit 23 also performs a process of converting drawn image data obtained by drawing on the display 220 with the electronic pen 2500 or hand H by the user into coordinate data. For example, when an electronic blackboard (2a) at an arbitrary base sends coordinate data to an electronic blackboard (2b) at another base, the electronic blackboard (2b) sends coordinate data to the electronic blackboard (2b) on the side of the other electronic blackboard (2b) based on the coordinate data. A drawing of the same content is displayed on the display 220.

The display control unit 24 is realized by the commands from the CPU 201 shown in FIG. 2 and the display controller 213 shown in FIG. It accesses the server 6 and displays various screen data. Specifically, the display control unit 24 starts and executes the Launcher 102 running on the OS 101 shown in FIG. A screen is displayed on the display 220.

The determining unit 25 is realized by instructions from the CPU 201 shown in FIG. 2, and performs various determinations.

The acquisition/providing unit 28 is executed by the command from the CPU 201 shown in FIG. 2, as well as the short-range communication circuit 219 and the antenna 219a, and communicates with a proprietary terminal such as an IC card or a smartphone by short-range communication. Obtain and provide data.

The storage/read processing unit 29 is executed by the instructions from the CPU 201 and the SSD 204 shown in FIG. I do. Furthermore, the storage unit 2000 overwrites and stores image data and sound data that are received when communicating with other electronic whiteboards or video conference terminals each time they are received. Among these, an image is displayed on the display 220 using the image data before being overwritten, and sound is output from the speaker 2300 using the sound data before being overwritten.

Note that the video conference terminal 3 and the car navigation device 4 have the same functions as the electronic whiteboard 2, so the description thereof will be omitted here.

Note that the transmitting/receiving section 21, receiving section 22, image/sound processing section 23, display control section 24, judgment section 25, acquisition/providing section 28, and storage/reading processing section 29 of the electronic blackboard 2 shown in FIG. This is shown conceptually and is not limited to this configuration. For example, a plurality of functional blocks illustrated as independent functional blocks in the electronic whiteboard 2 shown in FIG. 8 may be configured as one functional block. On the other hand, in the electronic whiteboard 2 shown in FIG. 8, the functions of one functional block may be divided into a plurality of functions, and the electronic whiteboard 2 may be configured as a plurality of functional blocks.

<Functional configuration of PC>
As shown in FIG. 8, the PC 5 includes a transmitting/receiving section 51 (obtaining section, first transmitting section, second transmitting section), a receiving section 52, a display control section 54, a generating section 56, and a determining section. 57, a sound control section 58, and a storage/read processing section 59. Each of these units is a function or functioning means that is realized when any of the components shown in FIG. . Furthermore, the PC 5 has a storage unit 5000 constructed by the HD 504 shown in FIG.

<<Functional configuration of PC>>
Next, each component of the PC 5 will be explained. The transmitting/receiving unit 51 is realized by commands from the CPU 501 and network I/F 509 shown in FIG. 5, and transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 10.

The receiving unit 52 is mainly realized by commands from the CPU 501 shown in FIG. 5, as well as a keyboard 511 and a mouse 512, and receives various inputs from the user.

The display control unit 54 is realized by instructions from the CPU 501 shown in FIG. 5, and displays images on the display 508, accesses the shared support server 6 using a web browser, and displays various screen data. do. Specifically, the display control unit 54 accesses the shared support server 6 or the schedule management server 8 by, for example, starting and running the minutes application 5502a or the browser 5502b running on the OS 5501 shown in FIG. . Then, the display control unit 54 downloads a Web APP (Web Application) that includes at least HTML (HyperText Markup Language), CSS (Cascading Style Sheets), JavaScript (registered trademark), etc., and downloads the WebA Various types generated by PP The image data is displayed on the display 508. The display control unit 54 is configured to display HT data including data in, for example, XML (Extensible Markup Language), JSON (JavaScript Object Notation), or SOAP (Simple Object Access Protocol) format. The image data generated by ML5 is displayed on the display 508.

The generation unit 56 is realized by instructions from the CPU 501 shown in FIG. 5, and has a function of generating various image data to be displayed on the display 508. The generating unit 56 generates various image data using the content data received by the transmitting/receiving unit 51. For example, the generation unit 56 generates image data related to the text data, which is content image data, in order to render text data, which is content data, and display the rendered data. Rendering is a process of interpreting data written in a web page description language (HTML, CSS, XML, etc.) and calculating the arrangement of characters, image data, etc. that are actually displayed on the screen. The generating unit 56 is an example of a generating means.

The determining unit 57 has a function of determining, when specific text data shown in FIG. 5 has been edited, whether or not there is a portion of the text data that has been edited in bold. Note that examples of hidden characters include symbols such as "", "○", "x", "*", and "=", but it is sufficient to decide in advance the symbols to be recognized as hidden characters. Further, the hidden characters may be set such that a specific symbol can be hidden.

The sound control unit 58 is realized by instructions from the CPU 501 shown in FIG. 5, and has a function of outputting a sound signal from the speaker 515. The sound control unit 58 sets sound data to be output from the speaker 515, and reproduces the sound data by outputting a sound signal related to the set sound data from the speaker 515. The sound control section 58 is an example of a reproduction means.

The storage/read processing unit 59 is executed by the instructions from the CPU 501 and the HDD controller 505 shown in FIG. 5, and stores various data in the storage unit 5000 and reads various data from the storage unit 5000 Perform processing.

The functions of the transmitting/receiving section 51, receiving section 52, display control section 54, generation section 56, judgment section 57, sound control section 58, and storage/reading processing section 59 of the PC 5 shown in FIG. 8 are conceptually shown. However, the configuration is not limited to this. For example, a plurality of functional blocks illustrated as independent functional blocks in the PC 5 shown in FIG. 8 may be configured as one functional block. On the other hand, in the PC 5 shown in FIG. 8, the functions of one function block may be divided into a plurality of functions, and the functions may be configured as a plurality of function blocks.

<Functional configuration of shared support server>
The shared support server 6 includes a transmitting/receiving section 61 , an authentication section 62 , a creating section 63 , a generating section 64 , a determining section 65 , and a storage/reading processing section 69 . Each of these units has a function or functions that is realized when any of the components shown in FIG. It is a means. Further, the shared support server 6 has a storage unit 6000 constructed from the HD 604 shown in FIG. 5 and the like.

<<User authentication management table>>
FIG. 9(A) is a conceptual diagram showing a user authentication management table. In the storage unit 6000, a user authentication management DB 6001 is constructed, which is composed of a user authentication management table as shown in FIG. 9(A). In this user authentication management table, a user ID for identifying a user, a user name, an organization ID for identifying an organization to which the user belongs, and a password are managed in association with each other. Note that the organization ID also includes a domain name representing a group or organization for managing multiple computers on a communication network.

<<Access management table>>
FIG. 9(B) is a conceptual diagram showing the access management table. In the storage unit 6000, an access management DB 6002 is constructed, which includes an access management table as shown in FIG. 9(B). In this access management table, organization IDs, access IDs and access passwords necessary for authentication when accessing the schedule management server 8 are managed in association with each other. The access ID and access password are provided by the shared support server 6 using the HTTP (Hypertext Transfer Protocol) or HTTPS (Hypertext Transfer Protocol Secure) protocol via a Web API or the like, using a service provided by the schedule management server 8 ( functions). The schedule management server 8 manages a plurality of schedulers, and different organizations may use different schedulers, so management using an access management table is required.

<< Schedule management table >>
FIG. 9(C) is a conceptual diagram showing the schedule management table. In the storage unit 6000, a schedule management DB 6003 is constructed, which includes a schedule management table as shown in FIG. 9(C). In this schedule management table, for each scheduled event ID and execution event ID, the organization ID, the user ID of the person making the reservation, whether or not this person is participating, the name of the person making the reservation, the scheduled start time, the scheduled end time, the event name, etc. The user ID of the participant, presence or absence of participation by other participants, names of other participants, and file data are managed in association with each other.

Among these, the scheduled event ID is identification information for identifying a reserved event. The scheduled event ID is an example of scheduled event identification information for identifying an event scheduled to be executed. The execution event ID is identification information for identifying an event that has actually been executed or an event that is being executed among the reserved events. The execution event ID is an example of execution event identification information for identifying an event that has been executed or an event that is being executed. The reservation person's name is the name of the person who reserved the shared entity, and if the shared entity is a conference room, it is, for example, the name of the organizer, and if the shared entity is a vehicle, it is, for example, the name of the driver. The scheduled start time indicates the scheduled time to start using the union. The scheduled end time indicates the scheduled end time of use of the union. The event name indicates the name of the event scheduled to be executed by the reservation person. The user IDs of other participants are identification information for identifying participants other than the reservation person. Other participant names are names of participants other than the reservation person, and include unions. In other words, users in this case include not only the person making the reservation and other participants, but also the union. The file data is file data of a material file used in the event corresponding to the scheduled event ID, which is registered by user A using a schedule input screen 550 (see FIG. 19), which will be described later. Further, the file data is data in a predetermined file format created using various applications. The file format of the file data is, for example, a PowerPoint file, an Excel file, or the like.

＜＜Content management table＞＞
FIG. 10 is a conceptual diagram showing a content management table. In the storage unit 6000, a content management DB 6005 is constructed, which is composed of a content management table as shown in FIG. In this content management table, a content processing ID, content processing type, content content, and content processing start date and time and end date and time are associated and managed for each execution event ID. Here, the content is the content of an executed event that occurs at an event such as a conference, or materials used at the event. Types of content processing include recording, snapshots, speech-to-text conversion, action item generation, and document delivery. Further, the content processing ID is identification information for identifying the content processing that occurred in each event.

Here, the content includes history information indicating the execution details of the event and action items caused by the executed event. The history information is data such as recorded data, snapshots, audio text, or materials. A snapshot is a process of acquiring a display screen at a certain point in time of an event being executed as image data. Note that a snapshot is also called, for example, capture or image recognition.

When the content processing type is "recording", the content includes a URL indicating the storage location of the recorded sound data. Further, when the type of content processing is "snapshot", the content includes a URL indicating the storage location of the image data of the screen acquired by the snapshot (capture). Capturing means saving an image (still image, moving image) displayed on the display 220 as image data. Further, when the type of content processing is "speech-to-text conversion", the content includes a URL indicating a storage location of text data of the received speech text.

Here, an action item indicates the content of an action that occurs at an event such as a meeting and that a person related to the event should respond to. When the content processing type is "Action item generation", the content includes the user ID of the person executing the action item, the deadline for completing the action item, and the URL indicating the storage location of the image data representing the action item. including.

<<Functional configuration of the shared support server>>
Next, each functional configuration of the shared support server 6 will be explained in detail. In the following, when explaining each functional configuration of the shared support server 6, the main components for realizing each functional configuration of the shared support server 6 among the components shown in FIG. Also explain the relationship.

The transmitting/receiving unit 61 of the shared support server 6 shown in FIG. 8 is realized by the command from the CPU 601 shown in FIG. 5 and the network I/F 609 shown in FIG. to send and receive various data (or information) to and from other terminals, devices, or systems.

The authentication unit 62 is realized by instructions from the CPU 601 shown in FIG. Authentication is performed by determining whether the information is authentic.

The creation unit 63 is realized by instructions from the CPU 601 shown in FIG. 5, and creates the information as shown in FIG. A reservation list screen 230 is created.

The generation unit 64 is realized by instructions from the CPU 601 shown in FIG. 5, and generates an execution event ID, a content processing ID, and a storage destination URL.

The determination unit 65 is realized by instructions from the CPU 601 shown in FIG. 5, and performs various determinations. This determination will be described later.

The storage/read processing unit 69 is realized by instructions from the CPU 601 shown in FIG. 5 and the HDD controller 605 shown in FIG. Performs processing such as reading out various stored data.

Note that the functions of the transmitting/receiving section 61, authentication section 62, creation section 63, generation section 64, judgment section 65, and storage/reading processing section 69 of the shared support server 6 shown in FIG. 8 are shown conceptually. , but is not limited to this configuration. For example, a plurality of functional blocks illustrated as independent functional blocks in the shared support server 6 shown in FIG. 8 may be configured as one functional block. On the other hand, in the shared support server 6 shown in FIG. 8, the functions of one function block may be divided into a plurality of functions and configured as a plurality of function blocks.

<Functional configuration of schedule management server>
The schedule management server 8 includes a transmitting/receiving section 81, an authentication section 82, a generating section 83, a sound data editing section 84 (specific section, second editing section), a storage/reading processing section 89 (storage processing section), have. Each of these units has a function or function that is realized when any of the components shown in FIG. It is a means. Furthermore, the schedule management server 8 has a storage unit 8000 constructed by the HD 804 shown in FIG.

<<User authentication management table>>
FIG. 11(A) is a conceptual diagram showing a user authentication management table. In the storage unit 8000, a user authentication management DB 8001 is constructed, which is composed of a user authentication management table as shown in FIG. 11(A). In this user authentication management table, a user ID for identifying a user is managed in association with an organization ID and password for identifying an organization to which the user belongs.

<<User management table>>
FIG. 11(B) is a conceptual diagram showing the user management table. In the storage unit 8000, a user management DB 8002 is constructed, which includes a user management table as shown in FIG. 11(B). In this user management table, a user ID and a user name (user name) indicated by the user ID are associated and managed for each organization ID.

<<Union management table>>
FIG. 11C is a conceptual diagram showing the union management table. In the storage unit 8000, a union management DB 8003 is constructed, which includes a union management table as shown in FIG. 11(C). In this union management table, a union ID for identifying a union and a name of the union (union name) are managed in association with each organization ID.

<<Union reservation management table>>
FIG. 12A is a conceptual diagram showing a union reservation management table. In the storage unit 8000, a union reservation management DB 8004 is constructed, which is composed of a union reservation management table as shown in FIG. 12(A). In this union reservation management table, reservation information in which each piece of information is associated is managed. The reservation information includes, for each organization ID, a union ID, a union name, a user ID of the person making the reservation, a scheduled start date and time of use, a scheduled end date and time of use, and an event name. Among these, the scheduled start date and time of use indicates the scheduled start date and time of use of the union. The scheduled end date and time of use indicates the scheduled end date and time of use of the union. Each date and time indicates year, month, day, hour, minute, second, and time zone, but in FIG. 12(A), due to space limitations, year, month, day, hour, and minute are also indicated.

<<Event management table>>
FIG. 12(B) is a conceptual diagram showing the event management table. In the storage unit 8000, an event management DB 8005 is constructed, which includes an event management table as shown in FIG. 12(B). This event management table manages schedule information in which each piece of information is associated. In the schedule information, an organization ID, user ID, user name, scheduled event start date and time, scheduled event end date and time, and event name are associated and managed for each scheduled event ID. Among these, the scheduled event start date and time indicates the scheduled start date and time when the event is to be executed. The scheduled event end date and time indicates the scheduled end date and time when the event is executed. Each date and time indicates year, month, day, hour, minute, second, and time zone, but in FIG. 12(B), due to space limitations, year, month, day, hour, and minute are also indicated. Further, in the event management table, file data of material files used in the event indicated in the schedule information is managed in association with the schedule event ID.

<<Server authentication management table>>
FIG. 13(A) is a conceptual diagram showing a server authentication management table. In the storage unit 8000, a server authentication management DB 8006 is constructed, which is composed of a server authentication management table as shown in FIG. 13(A). In this server authentication management table, access IDs and access passwords are managed in association with each other. The access ID and access password have the same concept as the access ID and access password managed by the access management DB 6002 of the shared support server 6.

<<Execution event history management table>>
FIG. 13(B) is a conceptual diagram showing the execution event history management table. In the storage unit 8000, an execution event history management DB 8008 is constructed, which is composed of an execution event history management table as shown in FIG. 13(B). In this execution event history management table, a content processing ID, content processing type, content content, and start date and time and end date and time of the content processing are managed in association with each execution event ID. This execution event history management DB 8008 manages some of the same data as the content management DB 6005. The same data includes an execution event ID, a content processing ID, a content processing type, a start date and time, and an end date and time of the content processing. The contents of the contents are the same except for the notation method (http:// or c://) of the storage destination of the content data.

<<Execution event management table>>
FIG. 14(A) is a conceptual diagram showing an execution event management table. In the storage unit 8000, an execution event management DB 8009 is constructed, which is composed of an execution event management table as shown in FIG. 14(A). In this execution event management table, an execution event ID is managed in association with an event name, an event start date and time, and an event end date and time. This execution event management DB 8009 manages information regarding actually executed events among the schedule information managed in the event management DB 8005.

<<Related information management table>>
FIG. 14(B) is a conceptual diagram showing the related information management table. A related information management DB 8010 configured by a related information management table as shown in FIG. 14(B) is constructed in the storage unit 8000. In this related information management table, related information in which each piece of information (data) is associated is managed for each execution event ID. The related information is managed in association with the content generation time period, sound data, audio text data, and screen data. The content occurrence time zone indicates the elapsed time from the start date and time of the event at the time when the content occurred in the executed event. Here, the content occurrence time period is generated by the generation unit 83 based on the start date and time of the event stored in the event management DB 8005 and the start date and time and end date and time of the content processing stored in the execution event history management DB 8008. Ru. The time period in which content occurs is an example of time information. The sound data also includes a content processing ID and a content type. The audio text data and screen data include a content processing ID, content type, and sequence number. The sequence number indicates the chronological order in which content processing occurred for each of audio text data and screen data.

<<Text information management table>>
FIG. 15 is a conceptual diagram showing the text information management table. In the storage unit 8000, a text information management DB 8012 is constructed, which is composed of a text information management table as shown in FIG. In this text information management table, text information including voice text data generated in an executed event is managed for each execution event ID. This text information is associated with a content processing ID, a text ID for identifying the text data, a transcript indicating the content of the text data, and status information indicating the status of the text data. Among these, the content processing ID identifies content processing whose type of content processing is "speech-to-text conversion." Further, transcript (text data) is text data indicating the content of the content associated in the execution event history management DB 8008 with the same content processing ID as the associated content processing ID as text information. Further, the status information is information indicating whether or not the text data has been edited. If the associated text data has not been edited from the text data generated by the speech-to-text conversion server 9, the status information indicates "Original" information indicating that the text data has not been edited.

<<Functional configuration of schedule management server>>
Next, each functional configuration of the schedule management server 8 will be explained in detail. In the following, when explaining each functional configuration of the schedule management server 8, the main components for realizing each functional configuration of the schedule management server 8 among the components shown in FIG. Also explain the relationship.

The transmission/reception unit 81 of the schedule management server 8 shown in FIG. 8 is realized by the command from the CPU 801 shown in FIG. 5 and the network I/F 809 shown in FIG. to send and receive various data (or information) to and from other terminals, devices, or systems.

The authentication unit 82 is realized by instructions from the CPU 801 shown in FIG. Authentication is performed by determining whether the information is authentic. Further, the authentication unit 82 performs authentication by determining whether the information (access ID and access password) sent from the shared support server 6 is information registered in advance in the server authentication management DB 8006.

The generation unit 83 is realized by instructions from the CPU 801 shown in FIG. 5, and has a function of generating related information to be registered in the related information management DB 8010.

The sound data editing unit 84 is realized by a command from the CPU 801 shown in FIG. This is a function that can be edited and disabled. Examples of methods for invalidating the sound include processing such as silencing the sound or replacing it with a self-limiting sound.

The storage/read processing unit 89 is realized by instructions from the CPU 801 shown in FIG. 5 and the HDD controller 805 shown in FIG. Performs processing such as reading out various stored data.

The functions of the transmitting/receiving section 81, authentication section 82, generation section 83, sound data editing section 84, and storage/reading processing section 89 of the schedule management server 8 shown in FIG. 8 are conceptually shown. The present invention is not limited to such a configuration. For example, a plurality of functional blocks illustrated as independent functional blocks in the schedule management server 8 shown in FIG. 8 may be configured as one functional block. On the other hand, in the schedule management server 8 shown in FIG. 8, the functions of one functional block may be divided into a plurality of functions, and the functions may be configured as a plurality of functional blocks.

<Functional configuration of speech-to-text conversion server>
The speech-to-text conversion server 9 includes a transmitting/receiving section 91, a converting section 93, and a storage/reading processing section 99. Each of these units is a function or functioning means that is realized when any of the components shown in FIG. . The speech-to-text conversion server 9 also has a storage unit 9000 constructed by the HD 904 shown in FIG.

<<Functional configuration of speech-to-text conversion server>>
Next, each functional configuration of the speech-to-text conversion server 9 will be explained in detail. In the following, when explaining each functional configuration of the speech-to-text conversion server 9, the main components for realizing each functional configuration of the speech-to-text conversion server 9 among the components shown in FIG. Also explain the relationship.

The transmission/reception unit 91 of the speech-to-text conversion server 9 shown in FIG. 8 is realized by the command from the CPU 901 shown in FIG. 5 and the network I/F 909 shown in FIG. Send and receive various types of data (or information) to and from other terminals, devices, or systems.

The converting unit 93 is realized by a command from the CPU 901 shown in FIG. 5, and converts the sound data received by the transmitting/receiving unit 91 via the communication network 10 into text data.

The storage/read processing unit 99 is realized by the commands from the CPU 901 shown in FIG. 5 and the HDD controller 905 shown in FIG. Performs processing such as reading out various stored data.

Note that each of the above IDs is an example of identification information. Further, the organization ID includes a company name, a business office name, a department name, a region name, and the like. User IDs include employee numbers, driver's license numbers, My Numbers in Japan's social security and tax number system, etc.

Further, the functions of the transmitting/receiving section 91, the converting section 93, and the storage/reading processing section 99 of the speech-to-text conversion server 9 shown in FIG. 8 are conceptually shown, and the configuration is not limited to this. do not have. For example, a plurality of functional blocks illustrated as independent functional blocks in the speech-to-text conversion server 9 shown in FIG. 8 may be configured as one functional block. On the other hand, the functions of one function block in the speech-to-text conversion server 9 shown in FIG. 8 may be divided into a plurality of functions, and configured as a plurality of function blocks.

(Processing or operation of embodiment)
Hereinafter, the processing or operation of each embodiment will be described.

<Schedule registration process>
FIG. 16 is a sequence diagram illustrating an example of the flow of schedule registration processing in the shared system according to the embodiment. FIG. 17 is a diagram showing the sign-in screen. FIG. 18 is a diagram showing an example of the initial screen of the PC. FIG. 19 is a diagram showing a schedule input screen. The process by which reservationer A (Rikotaro) registers his own schedule from the PC 5 to the schedule management server 8 will be explained using FIGS. 16 to 19.

First, when the user A operates the keyboard 511 or the like of the PC 5, the display control unit 54 of the PC 5 displays a sign-in screen 530 for signing in on the display 508, as shown in FIG. Display it (step S11). This sign-in screen 530 includes an input field 531 for entering the user ID and organization ID of the user, an input field 532 for entering the password, a sign-in button 538 to be pressed when signing in, and a sign-in button 538 for signing in. A cancel button 539 is included to be pressed when canceling. Here, the user ID and organization ID are user A's e-mail address. The user name part of the e-mail address is the user ID, and the domain name part is the organization ID. Note that the input field 531 may be an input field for inputting a user ID and an organization ID separately, instead of an e-mail address.

Next, when User A enters his or her user ID and organization ID in the input field 531, enters his or her password in the input field 532, and presses the sign-in button 538, the reception unit 52 prompts the user to sign in. The request is accepted (step S12). Then, the transmitting/receiving unit 51 of the PC 5 transmits sign-in request information indicating a request for sign-in to the schedule management server 8 (step S13). This sign-in request information includes the information (user ID, organization ID, and password) accepted in step S12. Thereby, the transmitting/receiving unit 81 of the schedule management server 8 receives the sign-in request information.

Next, the authentication unit 82 of the schedule management server 8 authenticates the user A using the user ID, organization ID, and password (step S14). Specifically, in the user authentication management DB 8001 (see FIG. 11(A)), the storage/read processing unit 89 stores the user information corresponding to the set of user ID, organization ID, and password received in step S13. Search for a set of ID, organization ID, and password. If there is a corresponding pair, the authentication unit 82 determines that the requesting user A is an authorized user. If there is no corresponding pair, the authentication unit 82 determines that user A is an unauthorized (unauthorized) user. If it is not valid, the transmitting/receiving unit 81 notifies the PC 5 that it is not valid, but here we will continue to explain the case where it is valid.

Next, the transmitter/receiver 81 transmits the authentication result to the PC 5 (step S15). Thereby, the transmitting/receiving unit 51 of the PC 5 receives the authentication result.

Next, when the generation unit 56 of the PC 5 receives the authentic authentication result in step S15, it generates an initial screen 540 as shown in FIG. 18 (step S16). Then, the display control unit 54 of the PC 5 displays an initial screen 540 as shown in FIG. 18 on the display 508 (step S17). This initial screen 540 includes a "schedule registration" button 541 that is pressed to register a schedule, a "view execution event history" button 543 that is pressed to view the execution event history, and a button for closing the initial screen. A "Close" button 549 is included. Here, when the user presses the "schedule registration" button 541, the reception unit 52 accepts schedule registration (step S18). Then, the transmitting/receiving unit 51 transmits a schedule registration request to the schedule management server 8 (step S19). Thereby, the transmitting/receiving unit 81 of the schedule management server 8 receives the schedule registration request.

Next, the storage/read processing unit 89 of the schedule management server 8 searches the user management DB 8002 (see FIG. 11(B)) using the organization ID received in step S13 as a search key, thereby searching all the corresponding The user ID and all user names are read (step S20). Then, the transmitting/receiving unit 81 transmits schedule input screen information to the PC 5 (step S21). This schedule input screen information includes all user IDs and all user names read out in step S20. All the user names also include the name of the reservation person who is the user A who made the input for sign-in in step S12. Thereby, the transmitting/receiving unit 51 of the PC 5 receives the schedule input screen information.

Next, in the PC 5, the generation unit 56 generates a schedule input screen 550 using the schedule input screen information received in step S21. (Step S22). Then, the display control unit 24 of the PC 5 displays a schedule input screen 550 as shown in FIG. 19 on the display 508 (step S23).

On this schedule input screen 550, an input field 551 for inputting an event name, an input field 552 for inputting a union ID or a union name, and a scheduled start date and time of event execution (union use) are input. an input field 553 for inputting the scheduled end date and time of the event execution (use of the union), an input field 555 for inputting notes such as an agenda, and a display area for displaying the reservation person's name. 556, a selection menu 557 for selecting the participant name of a participant other than the person making the reservation, an "OK" button 558 that is pressed when registering a reservation, and a button 558 that is pressed when registering a reservation, and when canceling the entered contents. It includes a "CANCEL" button 559 that can be pressed. The reservation user name is the name of the user who made the input to sign in to the PC 5 in step S12. A mouse pointer p1 is also displayed.

Note that an e-mail address may be entered in the input field 552. Furthermore, when a union name is selected in the selection menu 557, the union is also added as another participant.

Next, user A enters predetermined items in input fields 551 to 555, uses mouse pointer p1 to select the name of the user who wants to participate in the event (user name) from selection menu 557, When the "OK" button 558 is pressed, the receiving unit 52 receives input of schedule information (step S24). Then, the transmitting/receiving unit 51 transmits the schedule information to the schedule management server 8 (step S25). This schedule information includes the event name, union ID (or union name), scheduled start date and time, scheduled end date and time, user ID of each participant, and memo. In addition, on the schedule input screen 550, if a union ID is input in the input field 552, this union ID is transmitted, and if a union name is input in the input field 552, this union name is transmitted. is sent. Further, on the schedule input screen 550, a user name is selected in the selection menu 557, but since the user ID has also been received in step S21, the user ID corresponding to the user name is transmitted. Thereby, the transmitting/receiving unit 81 of the schedule management server 8 receives the schedule information.

Next, the schedule management server 8 searches the union management DB 8003 (see FIG. 11(C)) using the union ID (or union name) received in step S25 as a search key to find the corresponding union. name (or union ID) is read out (step S26).

Next, the storage/read processing unit 89 stores the reservation information in the union reservation management DB 8004 (see FIG. 12(A)) (step S27). In this case, the storage/read processing unit 89 adds one record of reservation information to the union reservation management table of the union reservation management DB 8004 managed by the scheduler registered in advance. The reservation information is configured based on the schedule information received in step S25 and the union name (or union ID) read out in step S26. Note that the scheduled usage start date and time in the union reservation management DB 8004 corresponds to the scheduled start date and time in the schedule information. Further, the scheduled usage end date and time in the union reservation management DB 8004 corresponds to the scheduled end date and time in the schedule information.

Further, the storage/read processing unit 89 stores the schedule information in the event management DB 8005 (see FIG. 12(B)) (step S28). In this case, the storage/read processing unit 89 adds one record of schedule information to the event management table of the event management DB 8005 managed by the scheduler registered in advance. The schedule information is configured based on the schedule information received in step S25. Note that the scheduled event start date and time in the event management DB 8005 corresponds to the scheduled start date and time in the schedule information. Furthermore, the scheduled end date and time of the event in the event management DB 8005 corresponds to the scheduled end date and time in the schedule information.

As described above, user A can register his or her own schedule in the schedule management server 8. Note that although FIGS. 16 to 19 describe the case in which the schedule is registered using the PC 5, the schedule may be registered using the electronic whiteboard 2, the video conference terminal 3, or the car navigation device 4 through similar processing.

<Event start processing>
20 and 23 are sequence diagrams illustrating an example of the flow of event initiation processing in the shared system according to the embodiment. FIG. 21 is a diagram showing a sign-in screen displayed on the electronic blackboard. FIG. 22 is a diagram showing a union reservation list screen. FIG. 24 is a diagram showing a detailed event information screen. FIG. 25 is a diagram showing a display screen displayed on the electronic blackboard at the start of an event. FIG. 26 is an image diagram showing how a user uses the electronic blackboard. With reference to FIGS. 20 to 26, a process for holding a conference with other participants using the electronic blackboard 2 in a conference room X reserved by a reservation person A (Rikotaro) will be described.

First, when user A presses the power switch 222 of the electronic whiteboard 2, the reception unit 22 of the electronic whiteboard 2 receives a power ON request from the user (step S31). When the reception unit 22 accepts the power ON request, the Launcher 102 shown in FIG. 6 starts up. Then, the display control unit 24 of the electronic whiteboard 2 displays the sign-in screen 110 for signing in on the display 220, as shown in FIG. 21 (step S32). This sign-in screen 110 includes a selection icon 111 that is pressed when user A signs in using his or her own IC card, and a selection icon 111 that user A presses to sign in using his or her own IC card. It includes a selection icon 113 that is pressed when signing in by entering a password, and a power icon 115 that is pressed when turning off the power without signing in.

Next, the user A presses the selection icon 111 to perform communication with the short-range communication circuit 219 such as an IC card reader, or the user A presses the selection icon 113 and the user A By inputting the user's e-mail address and password, the receiving unit 22 receives the sign-in request (step S33). Then, the transmitting/receiving unit 21 transmits sign-in request information indicating a sign-in request to the shared support server 6 (step S34). This sign-in request information includes the information accepted in step S33 (user ID, organization ID, and password), time zone information of the country or region where the electronic blackboard 2 is installed, and the communication terminal (here, the electronic blackboard 2) includes the user ID, organization ID, and password. Thereby, the transmitting/receiving unit 61 of the shared support server 6 receives the sign-in request information.

Next, the authentication unit 62 of the shared support server 6 authenticates the user A using the user ID, organization ID, and password of the user A received in step S34 (step S35). Specifically, the storage/reading processing unit 69 uses the user ID, organization ID, and password of user A received in step S34 as search keys in the user authentication management DB 6001 (see FIG. 9(A)). , searches for a set of user ID, organization ID, and password that corresponds to the above-received set of user ID, organization ID, and password of user A. If there is a corresponding pair, the authentication unit 62 determines that the requesting user A is an authorized user. If there is no corresponding pair, the authentication unit 62 determines that the requesting user A is an unauthorized (unauthorized) user. If it is not valid, the transmitting/receiving unit 61 notifies the electronic blackboard 2 that it is not valid, but here we will continue to explain the case where it is valid.

Next, the storage/read processing unit 69 of the shared support server 6 searches the access management DB 6002 (see FIG. 9(B)) using the organization ID of user A received in step S34 as a search key. The access ID and access password are read out (step S36).

Next, the transmitting/receiving unit 61 transmits reservation request information indicating a request for reservation information of the union and schedule request information indicating a request for user schedule information to the schedule management server 8 (step S37). The reservation request information and schedule request information include the time zone information, the user ID and organization ID of the communication terminal received in step S34, and the access ID and access password read out in step S36. . Thereby, the transmitting/receiving unit 81 of the schedule management server 8 receives the reservation request information and the schedule request information.

Next, the authentication unit 82 of the schedule management server 8 authenticates the shared support server 6 using the access ID and the access password (step S38). Specifically, the storage/read processing unit 89 stores the set of access ID and access password corresponding to the set of access ID and access password received in step S37 in the server authentication management DB 8006 (see FIG. 13(A)). Search for. If there is a corresponding pair, the authentication unit 82 determines that the requesting shared support server 6 is an authorized accesser. If there is no corresponding pair, the authentication unit 82 determines that the requesting shared support server 6 is an unauthorized (unauthorized) accessor. If it is not valid, the transmitting/receiving unit 81 notifies the shared support server 6 that it is not valid, but here we will continue to explain the case where it is valid.

The storage/read processing unit 89 of the schedule management server 8 uses the user ID of the communication terminal received in step S37 as a search key to search the union reservation management DB 8004 (see FIG. 12(A)) managed by the scheduler. By searching, the corresponding reservation information is read (step S39). In this case, the storage/reading processing unit 89 reads the reservation information whose usage start date and time is today.

Furthermore, the storage/read processing unit 89 uses the user ID of the communication terminal received in step S37 as a search key to search the event management DB 8005 (see FIG. 12(B)) managed by the scheduler. The corresponding schedule information is read (step S40). In this case, the storage/reading processing unit 89 reads out schedule information in which the scheduled event start date and time is today. If the schedule management server 8 is located in a different country or region from the communication terminal such as the electronic whiteboard 2, the time zone is set to match the country or region where the communication terminal is installed based on the time zone information. I'm making adjustments.

Next, the transmitting/receiving unit 81 transmits the reservation information read out in step S39 and the schedule information read out in step S40 to the shared support server 6 (step S41). Thereby, the transmitting/receiving unit 61 of the shared support server 6 receives the reservation information and schedule information.

Next, the creation unit 63 of the shared support server 6 creates a reservation list based on the reservation information and schedule information received in step S41 (step S42). Then, the transmitting/receiving unit 61 transmits reservation list information indicating the contents of the reservation list to the electronic blackboard 2 (step S43). Thereby, the transmitting/receiving section 21 of the electronic whiteboard 2 receives the reservation list information.

Next, in the electronic whiteboard 2, the display control unit 24 displays a reservation list screen 230 as shown in FIG. 22 on the display 220 (step S44). This reservation list screen 230 includes a display area 231 that displays a union name (here, a place name) and a display area 232 that displays today's date and time. Further, the reservation list screen 230 displays event information 235, 236, 237, etc. indicating today's event using the main community (here, conference room X). Each event information includes, for each event, the scheduled start time and end time of use of the union, the event name, and the name of the person who reserved the union (reserver name). The event information includes start buttons 235s, 236s, 237s, etc. that are pressed when the user specifies an event to start.

Next, in FIG. 23, when the user A presses the start button 235s with, for example, the electronic pen 2500, the reception unit 22 accepts the selection of the event indicated by the event information 235 (step S51). Next, the transmitting/receiving unit 21 of the electronic whiteboard 2 transmits the scheduled event ID indicating the scheduled event selected in step S51 above to the shared support server 6 (step S52). The process in step S52 is a process for requesting transmission of execution event identification information. Thereby, the transmitting/receiving unit 61 of the shared support server 6 receives the selected scheduled event ID.

Next, in the shared support server 6, the generation unit 64 generates a unique execution event ID (step S53). Then, the storage/reading processing unit 69 manages the execution event ID generated in step S53, the reservation event ID received in step S52, the user ID and organization ID of the reservation person, and the event information in association with each other (step S54). Note that the user ID and organization ID of the reservation person, and the event information are IDs and information based on the reservation information and schedule information received in step S41. Note that at this point, there is no entry in the participation/non-participation column in the reservation management table (see FIG. 9(C)).

Next, in the shared support server 6, the transmitting/receiving unit 61 transmits file data transmission request information indicating a transmission request for file data registered in the schedule management server 8 to the schedule management server 8 (step S55). The file data transmission request information includes the scheduled event ID received in step S52, the user ID and organization ID of the communication terminal received in step S34, and the access ID and access password read out in step S36. ing. Thereby, the transmitting/receiving unit 81 of the schedule management server 8 receives the file data transmission request information.

Next, the storage/read processing unit 89 of the schedule management server 8 searches the event management DB 8005 (see FIG. 12(B)) using the scheduled event ID received in step S55 as a search key. The associated file data is read (step S56). Then, the transmitting/receiving unit 81 transmits the file data read in step S56 to the shared support server 6 (step S57). Thereby, the transmitting/receiving unit 61 of the shared support server 6 receives the file data.

Next, the storage/read processing unit 69 of the shared support server 6 associates the scheduled event ID received in step S52 with the execution event ID generated in step S53, and stores the file data received in step S57. It is stored and managed in the schedule management DB 6003 (see FIG. 9(C)) (step S58).

Then, the transmitting/receiving unit 61 transmits the execution event ID generated in step S53 and the file data received in step S57 to the electronic whiteboard 2 (step S59). Thereby, the transmitting/receiving unit 21 of the electronic whiteboard 2 receives the execution event ID and the file data.

Next, in the electronic whiteboard 2, the storage/read processing unit 29 stores the execution event ID and file data in the storage unit 2000 (step S60). Here, the file data transmitted from the shared support server 6 is stored in a specific storage area of the storage unit 2000. Thereby, in the electronic whiteboard 2, by accessing the specific storage area during the execution of the event, the display control unit 24 causes the display 220 to display the file data stored in the specific storage area. Here, the specific storage area is a temporary storage destination for data provided for each event being executed, and is a storage area specified by an arbitrary path (character string) indicating the location within the storage unit 2000. It is. Note that the specific storage area is not limited to the configuration provided inside the electronic whiteboard 2, but may be an external storage device connected to the electronic whiteboard 2, or a local server that exists in an on-premises environment and can communicate with the electronic whiteboard 2. The structure may be provided.

Next, the display control unit 24 displays a detailed information screen 250 of the selected event on the display 220, as shown in FIG. 24 (step S61). The event detailed information screen 250 includes a display area 251 for the event name, a display area 252 for the scheduled event time (scheduled start time and scheduled end time), and a display area 253 for the reservation person's name. Further, the event detailed information screen 250 includes a display area 256 for displaying the contents of the memo, a display area 257 for displaying the names of prospective participants, and files stored in a specific storage area of the storage unit 2000. A display area 258 is displayed for displaying identification information (eg, file name) for identifying data. In the display area 257, the name of the person making the reservation shown in FIG. 19 and the names of other selected participants are displayed, and each person's name is checked to see who is actually participating in the conference. A check box is displayed. In addition, the display area 258 displays the file name of the file data stored in a specific storage area of the storage unit 2000, that is, the file data downloaded from the shared support server 6. The file name of the file data inside is also displayed. Further, the lower right side of the event detailed information screen 250 includes a "close" button 259 for closing the detailed information screen 250.

Next, when the user checks the checkbox of the user who is actually participating among the names of the planned participants and presses the "Close" button 259, the reception unit 22 accepts the selection of the participant. (Step S62). Then, the transmitting/receiving unit 21 transmits the user ID of each prospective participant and information indicating whether or not they will participate to the shared support server 6 (step S63). Thereby, the transmitting/receiving unit 61 of the shared support server 6 receives the user ID of each prospective participant and information indicating whether or not they will participate.

Next, in the shared support server 6, in the schedule management DB 6003, the presence or absence of participation is stored and managed in the participation presence/absence column that has not been entered (step S64).

As described above, user A starts an event (here, a policy decision meeting) using the common body (here, conference room X) and the communication terminal (here, electronic blackboard 2). When the event starts, the display control unit 24 of the electronic blackboard 2 causes the display 220 to display a display screen 100a as shown in FIG. 25. FIG. 25 is a diagram showing a display screen displayed on the electronic blackboard at the start of an event. The display screen 100a shown in FIG. 25 includes a menu bar 121, time information 124 indicating the elapsed time from the start of the event or the remaining time until the end of the event, and a power icon 117 that is pressed when turning off the power to the electronic blackboard 2. It is included. The menu bar 121 is an example of an operation display screen (window) of the Launcher 102. This menu bar 121 includes a plurality of operation icons 125 (125a to 125j) that are selected (pressed) when performing various processes during event execution. The operation icon 125a is selected (pressed) to view detailed information of the event being executed. The operation icon 125b is selected (pressed) when starting various external applications 103. The operation icon 125c is selected (pressed) when viewing file data stored in a specific storage area of the storage unit 2000. The operation icon 125d is selected (pressed) when switching the display of the application display screen of the external application 103 that is currently running. The operation icon 125e is selected (pressed) when changing the screen size of the application display screen of the external application 103. The operation icon 125f is selected (pressed) when performing various operations related to the event being executed. The operation icon 125g is selected (pressed) when capturing the display screen 100a displayed on the display 220. The operation icon 125h is selected (pressed) when ending an event in progress. The operation icon 125i is selected (pressed) when starting the browser application 103c to perform a search using the browser. The operation icon 125j is selected (pressed) when inputting characters, numbers, or the like.

Here, various icons included in the display screen displayed on the electronic blackboard 2 are an example of a "reception area." The reception area may include not only images such as icons and buttons, but also characters such as "change" or a combination of images and characters. The images here may be not only symbols and figures, but also illustrations, patterns, and other images that can be visually recognized by the user. Further, selection (pressing) of various icons is an example of an operation on various icons. Operations on various icons include, for example, an input operation on the display 220 using the electronic pen 2500, a double click or single click with a mouse, which is an example of an input means included in the PC 2700, and a keyboard, which is an example of an input means included in the PC 2700. Examples include the input operations used.

Then, as shown in FIG. 26, user A can hold a meeting in conference room X using electronic blackboard 2. The display control unit 24 of the electronic blackboard 2 causes the display 220 to display, as the event execution screen R, the menu bar 120 in which the menu bar 121 included in the display screen 100a is reduced and the file data r4 of the document file. For example, when the user A of the electronic blackboard 2 presses the operation icon 125c, the reception unit 22 receives the selection of the operation icon 125c, and the display control unit 24 displays the information stored in a specific storage area of the storage unit 2000. Display the file data r4 of the file. Note that the display control unit 24 of the electronic whiteboard 2 displays not only the file data received in step S59 but also the file data stored in advance in the storage unit 2000 or the file data newly generated in the event that is being started. It may be displayed. In this case, the storage/read processing unit 29 of the electronic whiteboard 2 stores file data generated or updated in the event that has been started in a specific storage area of the storage unit 2000.

<Execution event history registration process>
Next, the execution event history registration process will be explained using FIGS. 27 to 31.

27 and 31 are sequence diagrams illustrating an example of the flow of execution event history registration processing in the shared system according to the embodiment. FIG. 28 is a flowchart illustrating an example of the flow of speech-to-text conversion processing in the shared system according to the embodiment.

First, the determination unit 25 of the electronic whiteboard 2 determines the type of content processing in the event that is being started (step S71). Specifically, when the content is sound data generated by recording by the image/sound processing unit 23, the determining unit 25 determines that the type of content processing is “recording”. If the content is image data acquired by a snapshot (capture) by the image/sound processing unit 23, the determining unit 25 determines that the content processing type is “snapshot”. If the content is material file data transmitted by the transmitting/receiving section 21, the determining section 25 determines that the content processing type is "material transmission".

Next, the transmitting/receiving unit 21 transmits registration request information indicating the generated content registration request to the shared support server 6 (step S72). In this case, the transmitter/receiver 21 automatically transmits registration request information every time content occurs. This registration request information includes an execution event ID, a content transmission source user ID, content data, and content processing type information. Thereby, the transmitting/receiving unit 61 of the shared support server 6 receives the registration request information.

Next, the generation unit 64 generates a unique content processing ID for identifying the content processing that occurred due to the event (step S73).

Next, the determining unit 65 of the shared support server 6 determines the type of content processing received based on the type information included in the registration request information received by the transmitting/receiving unit 61 (step S74). Then, when the determining unit 65 determines that the type of content processing is "recording", the transmitting/receiving unit 61 of the shared support server 6 transmits sound data, which is content data, to the voice-to-text conversion server 9 (step S75). Thereby, the transmitter/receiver section 91 of the speech-to-text conversion server 9 receives the sound data. Note that if the type of content processing is other than "recording", the shared support server 6 moves to the process of step S78 without performing the processes of steps S75 to S77.

Next, the converter 93 of the speech-to-text conversion server 9 converts the sound data received by the transmitter/receiver 91 into text data (step S76). Here, the speech-to-text conversion process in the speech-to-text conversion server 9 will be explained using FIG. 28. First, the converter 93 acquires information indicating the date and time when the sound data was received by the transmitter/receiver 91 (step S76-1). Note that the information acquired in step S76-1 may be information indicating the date and time when the shared support server 6 received the sound data or the date and time when the shared support server 6 transmitted the sound data. In this case, the transmitting/receiving unit 91 of the speech-to-text conversion server 9 receives the sound data and the information indicating the date and time transmitted from the shared support server 6 in step S75.

Next, the conversion unit 93 executes text conversion processing of the sound data received by the transmission/reception unit 91 (step S76-2). Then, when the text conversion process of the sound data is completed (YES in step S76-3), the conversion unit 93 moves the process to step S76-4. On the other hand, the conversion unit 93 repeats the process of step S76-2 until the audio data text conversion process is completed. Here, in step S76-3, if a predetermined amount of text conversion has been performed based on the sound data received by the transmitter/receiver 91, the converter 93 determines that the text conversion process of the sound data is completed. do. For example, when the text conversion for one sentence has been performed, the conversion unit 93 determines that the text conversion process of the sound data is completed. The conversion unit 93 then generates text data converted from the sound data (step S76-4). Thereby, the speech-to-text conversion server 9 converts the sound data sent from the shared support server 6 into text data. Since the speech-to-text conversion server 9 receives sound data transmitted from the shared support server 6 at any time, it repeatedly executes the process shown in FIG. 28.

Here, steps S75 to S77 in FIG. 27 and the speech-to-text conversion process described in FIG. 28 will be described in more detail using FIG. 29. FIG. 29 is a sequence diagram illustrating an example of the flow of speech-to-text conversion processing in the shared system according to the embodiment.

As shown in step S75 in FIG. 27, the transmitting/receiving unit 21 of the electronic whiteboard 2 transmits sound data generated by the event being executed to the shared support server 6 at any time (steps Sa1, step Sa2). Thereby, the transmitting/receiving section 61 of the shared support server 6 receives sound data transmitted from the electronic whiteboard 2 at any time. Then, the transmitting/receiving unit 61 of the shared support server 6 transmits the sound data transmitted from the electronic blackboard 2 to the speech-to-text conversion server 9 at any time (step Sb1, step Sb2). Thereby, the transmitting/receiving unit 91 of the speech-to-text conversion server 9 receives the sound data transmitted from the shared support server 6 at any time.

Here, the sound data transmitted from the electronic blackboard 2 to the shared support server 6 in step Sa1 corresponds to the sound data transmitted from the shared support server 6 to the speech text conversion server 9 in step Sb1, and The sound data transmitted from 2 to the shared support server 6 corresponds to the sound data transmitted from the shared support server 6 to the schedule management server 8 in step Sb2. In this way, the sound data collected and processed by the image/sound processing unit 23 of the electronic whiteboard 2 is transmitted to the speech-to-text conversion server 9 at any time.

The converter 93 of the speech-to-text conversion server 9 converts the sound data received by the transmitter/receiver 91 into text data. Then, the transmitting/receiving unit 91 of the speech-to-text conversion server 9 transmits the text data converted by the converting unit 93 to the shared support server 6 (step Sc1). In this case, the text data transmitted in step Sc1 is text data obtained by converting the sound data received in steps Sb1 and Sb2 into text.

Here, the text data transmitted from the speech-to-text conversion server 9 to the shared support server 6 in step Sc1 is unconfirmed text data. When a predetermined amount of text conversion has been performed, the conversion unit 93 of the speech-to-text conversion server 9 determines that the text conversion of the sound data is completed and the text data is finalized. For example, when text conversion for one sentence has been performed, the conversion unit 93 determines that the text conversion of the sound data is completed and the text data is finalized. In step Sc1, since the text data is unconfirmed, the transmitting/receiving unit 91 converts the text data (“In this financial year, more”) converted by the converting unit 93 into the unconfirmed text data (unconfirmed). Sends an unconfirmed flag indicating that.

Then, the transmitting/receiving unit 61 of the shared support server 6 transmits the text data (unconfirmed text data) transmitted from the speech-to-text conversion server 9 to the electronic blackboard 2 (step Sc2). Thereby, the transmitting/receiving section 21 of the electronic whiteboard 2 receives the text data (unconfirmed text data) transmitted from the shared support server 6. Then, the display control unit 24 of the electronic whiteboard 2 causes the display 508 to display the text data received by the transmitting/receiving unit 21. The electronic blackboard 2 can display text data as subtitles for sound data generated by the event being executed on the event execution screen R as shown in FIG. Furthermore, by receiving and displaying even unconfirmed text data, the electronic blackboard 2 informs the user of the electronic blackboard 2 that the voice-to-text conversion process by the voice-to-text conversion server 9 is being performed at any time. You can confirm it.

By repeating the process shown above (steps Sa3 to Sa7, Sb3 to Sb7, Sc3 to Sc4), the shared system 1 collects and processes the sound collected and processed by the image/sound processing unit 23 of the electronic whiteboard 2. Repeatedly convert data to text. Note that the number of times (frequency) at which undefined text data is transmitted by the speech-to-text conversion server 9 can be set as appropriate.

When the conversion unit 93 determines that the text conversion of the sound data is completed and the text data is finalized, the transmission/reception unit 91 of the speech-to-text conversion server 9 transmits the finalized text data to the shared support server 6. (Step Sc5). In this case, the transmitter/receiver 91 transmits the text data converted by the converter 93 (“In this fiscal year, more than twice the results of last year.”) as well as a confirmation flag indicating that the text data has been confirmed. . Thereby, the transmitting/receiving unit 61 of the shared support server 6 receives the text data (confirmed text data) transmitted from the speech-to-text conversion server 9.

Here, the text data received in step Sc5 is stored and managed in the content management DB 6005 through the process in step S79 of FIG. 27. In this case, the date and time related to the processing of the text data received in step Sc5 is from the date and time when the sound data was transmitted in step Sb1 to the date and time when the text data was received in step Sc5. That is, the start date and time of content processing stored in the content management DB 6005 is the date and time when the sound data was transmitted in step Sb1, and the end date and time of content processing is the date and time when the text data was received in step Sc5. In this way, the shared support server 6 records the text data (confirmed text data) sent from the speech-to-text conversion server 9 as well as the date and time when the transmission of sound data corresponding to the text data to the speech-to-text conversion server 9 was started. It is possible to manage the date and time when the text data determined from the content processing is received as the start and end date and time of content processing.

Then, the transmitting/receiving unit 61 of the shared support server 6 transmits the text data (confirmed text data) transmitted from the speech-to-text conversion server 9 to the electronic blackboard 2 (step Sc6). Thereby, the transmitting/receiving section 21 of the electronic whiteboard 2 receives the text data transmitted from the shared support server 6. Then, the transmitting/receiving section 21 of the electronic blackboard 2 causes the display 220 to display the text data transmitted from the shared support server 6.

In this way, the shared system 1 is able to convert sound data generated by an event being executed into text data using the speech-to-text conversion server 9, as well as information regarding the time of occurrence of the text data (content processing (start date and time and end date and time) can be stored and managed by the shared support server 6 and the schedule management server 8.

Returning to FIG. 27, the description of the execution event history registration process will be continued. The transmitter/receiver 91 transmits the text data converted by the converter 93 to the shared support server 6 (step S77). Here, the transmitting/receiving unit 91 transmits information indicating the date and time acquired in step S76-1 to the shared support server 6 along with the text data. Thereby, the transmitting/receiving unit 61 of the shared support server 6 receives the text data.

Next, the generation unit 64 generates a URL of content data indicating the contents of the content (step S78). Then, the storage/read processing unit 69 stores the type of content processing, the start date and time of the content processing, and the end date and time of the content processing in step S73 for each execution event ID received in step S72 for the content management DB 6005 (see FIG. 10). The content processing ID generated in step S78 and the URL indicating the storage location of the content generated in step S78 are managed in association with each other (step S79).

Here, if the content processing type is "speech-to-text conversion", the start date and time and end date and time of the content processing are the dates and times when the sound data was converted to text data. Here, the date and time when the sound data was converted into text data refers to the date and time when the sound data was transmitted by the transmitting/receiving unit 61 of the shared support server 6 and the transmitting/receiving unit of the shared support server 6, as shown in FIG. This is the date and time when the text data was received by 61. Note that the date and time when the sound data was converted to text data is the date and time when the sound data was received by the transmitting and receiving unit 91 of the voice-to-text conversion server 9 and the date and time when the text data was transmitted by the transmitting and receiving unit 91 of the voice-to-text conversion server 9. There may be. Furthermore, if the content processing type is "speech-to-text conversion", the start date and time and end date and time of the content processing may be the same as the start date and time and end date and time of the content processing related to the audio data to be converted into text. .

In addition, when the type of content processing is "recording", "snapshot", and "material transmission", the start date and time and end date and time of content processing are determined by the content data (sound data, This is the date and time when the image data, file data) was received. In addition, when the content processing type is "recording", "snapshot", and "material transmission", the start date and time of content processing and the end date and time are the date and time when the content data was transmitted by the transmitting/receiving unit 21 of the electronic blackboard 2. It may be. Furthermore, when the type of content processing is "recording", the start date and time and end date and time of recording by the image/sound processing unit 23 may be used. Further, when the type of content processing is "snapshot", the start date and time and end date and time of content processing may be the date and time when a snapshot (capture) was performed by the image/sound processing unit 23.

Next, in FIG. 31, the storage/read processing unit 69 of the shared support server 6 searches the user authentication management DB 6001 (see FIG. 9(A)) using the user ID received in step S72 as a search key. The corresponding organization ID is read out (step S91).

Next, the storage/read processing unit 69 reads out the corresponding access ID and access password by searching the access management DB 6002 (see FIG. 9(B)) using the organization ID read in step S91 as a search key. (Step S92).

Next, the transmitter/receiver 61 transmits execution event history registration request information indicating an execution event history registration request to the schedule management server 8 (step S93). This execution event history registration request information includes the execution event ID received in step S72, the sender's user ID, the content data, the content processing ID generated in step S73, and the content data generated in step S78. The URL, the access ID and access password read in step S92, and the start date and time and end date and time of content processing are included. Thereby, the transmitting/receiving unit 81 of the schedule management server 8 receives the execution event history registration request information.

Next, in the schedule management server 8, the authentication unit 82 authenticates the shared support server 6 using the access ID and the access password (step S94). This authentication is the same process as step S38, so the explanation will be omitted. Here, we will continue to explain cases where it is legal.

Next, the storage/read processing unit 89 stores and manages the various data (information) received in step S93 in the execution event history management DB 8008 (see FIG. 13(B)) (step S95). The storage/read processing unit 89 stores various data (information) in the execution event history management DB 8008 in association with the execution event ID received in step S93. Thereby, the schedule management server 8 manages the same data as the shared support server 6 side.

Furthermore, the generation unit 83 of the schedule management server 8 generates related information that associates the content data received in step S93 for each content generation time slot (step S96). Here, the occurrence time period of the content included in the related information is generated using the start date and time of the event stored in the event management DB 8005 and the start date and time and end date and time of the content processing stored in the execution event history management DB 8008. Ru. That is, the content occurrence time zone indicates the elapsed time from the start date and time of the event at the time when the content occurred in the executed event. Then, the storage/reading processing unit 89 of the schedule management server 8 generates the information generated by the generation unit 83 in association with the execution event ID received in step S93 in the related information management DB 8010 (see FIG. 14(B)). The associated information is stored and managed (step S97). Thereby, the schedule management server 8 manages content data of different types of content processing in association with each content generation time.

Then, the storage/read processing unit 89 of the schedule management server 8 stores the text data received in step S93 in association with the execution event ID received in step S93 in the text information management DB 8012 (see FIG. 15). The included text information is stored and managed (step S98). Specifically, the generation unit 83 generates text information including the text data and content processing ID received in step S93, a text ID that identifies the text data received in step S93, and status information. Then, the storage/read processing unit 89 stores the text information generated by the generation unit 83 in the text information management DB 8012 in association with the execution event ID received in step S93. In this case, the status information included in the text information is given "Original" information indicating that the associated text data has not been edited.

Here, the data format of the text data stored in the shared support server 6 and the schedule management server 8 will be explained using FIG. 30. FIG. 30 is a diagram showing an example of a data file including text data. The data file d1 shown in FIG. 30 is stored and managed in the URL of the content storage location shown in the content management DB 6005 and the execution event history management DB 8008.

As shown in FIG. 30, the data file d1 includes 10 pieces of text data (transcript). In addition, each text data (0 to 9) includes a sequence number (seq_num) indicating the order in which content processing occurred, text data (text), content processing start date and time (start time), and content processing end date and time (end_time). ) and content processing ID (id). These pieces of information are the same as the information corresponding to each item in the content management DB 6005 and the execution event history management DB 8006. Note that the number of text data included in the data file d1 is not limited to this.

In this way, the shared support server 6 and the schedule management server 8 associate text data with time information related to the processing of the text data (start date and time and end date and time of content processing), and perform operations as shown in FIG. It can be managed using the data file d1.

As described above, the electronic whiteboard 2 can transmit the execution event ID of the event being executed and the content generated by the corresponding event to the schedule management server 8. Furthermore, the schedule management server 8 can store the received content in the execution event history management DB 8008 for each execution event ID. This allows the shared system 1 to store content generated by executed events for each event.

<Event end processing>
Next, a process for terminating an event in progress will be described using FIGS. 32 to 36.

32 and 33 are sequence diagrams illustrating an example of the flow of event termination processing in the shared system according to the embodiment. FIG. 34 is a diagram showing an event end screen displayed on the electronic blackboard. FIG. 35 is a diagram showing a file data upload screen displayed on the electronic blackboard. FIG. 36 is a diagram showing a file data upload completion screen displayed on the electronic blackboard.

In FIG. 32, when user A closes the screen displayed on display 220, reception unit 22 accepts the end of the event being executed (step S301).

Then, the transmitting/receiving unit 21 transmits execution event start/end information indicating the start date and time and end date and time of the execution event, and file data registration request information indicating a file data registration request to the shared support server 6 (step S302). This execution event start and end information includes an execution event ID, an event name, a start date and time, and an end date and time of the event. Further, this file data registration request information includes an execution event ID, a sender user ID, and file data. Thereby, the transmitting/receiving unit 61 of the shared support server 6 receives the execution event start/end information and the file data registration request information.

Next, the generation unit 64 of the shared support server 6 generates a unique content processing ID for identifying the content processing that occurred due to the event (step S303). Furthermore, the generation unit 64 generates a URL of content data indicating the contents of the content (step S304). Then, the storage/read processing unit 69 stores the type of content processing, the start date and time of the content processing, and the end date and time of the content processing in step S303 for each execution event ID received in step S302 for the content management DB 6005 (see FIG. 10). The content processing ID generated in step S304 and the URL indicating the storage location of the content generated in step S304 are managed in association with each other (step 305).

Next, the storage/read processing unit 69 of the shared support server 6 searches the user authentication management DB 6001 (see FIG. 9(A)) using the user ID received in step S72 as a search key. The organization ID is read (step S306).

Next, the storage/read processing unit 69 reads out the corresponding access ID and access password by searching the access management DB 6002 (see FIG. 9(B)) using the organization ID read in step S91 as a search key. (Step S307).

Next, in FIG. 33, the transmitting/receiving unit 61 transmits the execution event start/end information received in step S302 and file data registration request information indicating a file data registration request to the schedule management server 8 ( Step S308). This file data registration request information includes the execution event ID received in step S302, the sender's user ID and file data, the content processing ID generated in step S303, and the URL of the file data generated in step S304. , the access ID and access password read in step S307, and the start date and time and end date and time of content processing. Thereby, the transmitting/receiving unit 81 of the schedule management server 8 receives the execution event start/end information and the file data registration request information.

Next, in the schedule management server 8, the authentication unit 82 authenticates the shared support server 6 using the access ID and the access password (step S309). This authentication is the same process as step S38, so the explanation will be omitted. Here, we will continue to explain cases where it is legal.

Next, the storage/read processing unit 89 of the schedule management server 8 stores the execution event start/end information received in step S308 in the execution event management DB 8009 (see FIG. 14(A)) (step S310). . In this case, the storage/read processing unit 89 adds one record of execution event start/end information to the execution event management table of the execution event management DB 8009.

Next, the storage/read processing unit 89 stores and manages the various data (information) received in step S308 in the execution event history management DB 8008 (step S311). The storage/read processing unit 89 stores various data (information) including file data in the execution event history management DB 8008 (see FIG. 13(B)) in association with the execution event ID received in step S308. Thereby, the schedule management server 8 manages the same file data as the shared support server 6 side.

Next, the transmitting/receiving unit 81 transmits file data registration information indicating that file data has been registered to the shared support server 6 (step S312). Thereby, the transmitting/receiving unit 61 of the shared support server 6 receives the file data registration information.

Then, the transmitting/receiving unit 61 of the shared support server 6 transmits the file data registration information transmitted from the schedule management server 8 to the electronic blackboard 2 (step S313). Thereby, the transmitting/receiving section 21 of the electronic whiteboard 2 receives the file data registration information.

Next, when the file data registration information is received by the transmitter/receiver 21, the storage/read processing unit 29 of the electronic whiteboard 2 deletes the file data stored in a specific storage area of the storage unit 2000 (step S314). . Thereby, the electronic blackboard 2 can reduce the risk of leakage of meeting information, etc. by deleting file data that has already been sent to the shared support server 6.

Here, the screen transition of the screen displayed on the electronic blackboard 2 when the event in progress is ended will be explained. First, in step S301, when the reception unit 22 accepts the termination of the event being executed, the display control unit 24 displays an event termination screen 270 as shown in FIG. 34 on the display 220. . This event end screen 270 includes a file display area 272, a file upload selection area 273, a "Close" button 278 that is pressed when proceeding with the end processing of the currently running event, and a "Close" button 278 that is pressed to proceed with the end process of the currently running event. includes a "CANCEL" button 279 that is pressed. In the file display area 272, file data images 272a, 272b, and 272c for identifying file data of material files stored in a specific storage area of the storage unit 2000 are displayed. The file upload selection area 273 includes a check box that accepts a selection as to whether or not to upload the file data corresponding to the file data image displayed in the file display area 272 to the shared support server 6.

When the reception unit 22 receives a selection of the “Close” button 278 with the file upload selection area 273 selected, the display control unit 24 displays a file upload screen 280a as shown in FIG. 35 on the display 220. Display. This file upload screen 280a is displayed on the display 220 when file data stored in a specific storage area of the storage unit 2000 is being uploaded to the shared support server 6. The file upload screen 280a includes an event name 281 of the event to end, an end date and time 282 of the event, a file data upload progress display area 283, and a "CANCEL" button 288 that is pressed to interrupt (cancel) the file data upload. include. The upload progress display area 283 displays the number of file data to be uploaded (for example, "3" shown in FIG. 35) and the number of file data that have been uploaded (for example, "0" shown in FIG. 35). ) is displayed.

Then, when the file data upload is completed, the display control unit 24 displays an upload completion screen 280b as shown in FIG. 36. The upload completion screen 280b includes a "Close" button 288 that is pressed to end the event. When the upload completion screen 280b is displayed on the display 220, the storage/read processing unit 29 of the electronic blackboard 2 stores the uploaded file data in a specific storage area of the storage unit 2000, as shown in step S314. Delete from.

On the other hand, if uploading of file data fails while the file upload screen 280a is displayed on the display 220, the display control unit 24 displays information for identifying the file data for which the upload has failed (for example, the file name). As a result, for example, when a failure occurs in the communication network 10, event participants can print the file data created or edited at the event, or store the file data in the USB memory 2600 connected to the electronic whiteboard 2. By doing this, you can bring back the file data that failed to upload.

In addition, if file data remains in a specific storage area of the storage unit 2000 even after the event ends, the storage/read processing unit 29 of the electronic blackboard 2 starts the next event using the electronic blackboard 2, for example. Sometimes, file data stored in a specific storage area is deleted. Thereby, the electronic whiteboard 2 can reduce the risk of leakage of meeting information and the like.

<Viewing execution event history>
37 and 38 are sequence diagrams illustrating an example of the flow of the execution event history viewing process of the shared system according to the embodiment. FIG. 39 is a diagram showing an execution event list screen displayed on the PC according to the embodiment. FIG. 40 is a diagram showing an execution event history screen displayed on the PC according to the embodiment. The execution event history viewing process will be described with reference to FIGS. 37 to 40. Note that the processing in steps S111 to S117 shown in FIG. 37 is similar to the processing in steps S11 to S17 shown in FIG. 16, respectively.

When the user presses the "view execution event history" button 543 on the initial screen 540 shown in FIG. 18, the reception unit 52 accepts viewing of the execution event history (step S118).

Then, the transmitting/receiving unit 51 transmits execution event history viewing request information indicating an execution event history viewing request to the schedule management server 8 (step S119). Thereby, the transmitting/receiving unit 81 of the schedule management server 8 receives the execution event history viewing request information.

Next, in the schedule management server 8, the storage/read processing unit 89 reads out the execution event start/end information stored in the execution event management DB 8009 (step S120). This execution event start and end information includes an execution event ID, an event name, a start date and time, and an end date and time.

Then, the transmitting/receiving unit 81 transmits the execution event ID, event name, start date and time, and end date and time read out in step S120 to the PC 5 (step S121). Thereby, the transmitting/receiving unit 51 of the PC 5 receives the execution event ID, event name, start date and time, and end date and time.

Next, the generation unit 56 of the PC 5 generates an execution event list screen 570 as shown in FIG. 39 using the data (information) received in step S121 (step S122). Then, in the PC 5, the display control unit 54 causes the execution event list screen 570 generated by the generation unit 56 to be displayed on the display 508 (step S123). As shown in FIG. 39, the execution event list screen 570 includes information 571 to 573 indicating execution events. For example, the information 571 to 573 indicating the execution event displays the name of the execution event, the start date and time, and the end date and time of the event. Information 571 to 573 indicating the executed event is an example of a history information selection area that accepts a selection for displaying history information indicating the execution content of the event. Further, the lower right portion of the execution event list screen 570 includes a “close” button 575 for closing the execution event list screen 570.

Next, the reception unit 52 of the PC 5 receives the selection of an execution event included in the execution event list screen 570 (step S124). Specifically, when the user selects any of the information 571 to 573 indicating an execution event included in the execution event list screen 570, the reception unit 52 accepts the selection of the information indicating an execution event. Then, the transmitting/receiving unit 51 of the PC 5 transmits the execution event ID of the execution event selected by the accepting unit 52 to the schedule management server 8 (step S125). Thereby, the transmitting/receiving unit 81 of the schedule management server 8 receives the execution event ID.

Next, the storage/read processing unit 89 of the schedule management server 8 reads out the corresponding execution event history information by searching the execution event history management DB 8008 using the execution event ID received in step S125 as a search key (step S126). This execution event history information includes a content process ID, a type of content process, a start date and time, and an end date and time of the content process.

Furthermore, the storage/read processing unit 89 of the schedule management server 8 reads the corresponding related information by searching the related information management DB 8010 using the execution event ID received in step S125 as a search key (step S127). This related information includes the content generation time period, content processing ID, content processing type, and sequence number. The time period of content occurrence included in the related information is an example of time information.

Then, the storage/reading processing unit 89 reads content data indicating the contents of the execution event history from the storage location of the content data indicating the contents of the execution event history read out in step S126 (step S128). Then, the transmitting/receiving unit 81 transmits the read related information, content data, and execution event history information to the PC 5 (step S129). Thereby, the transmitting/receiving unit 51 of the PC 5 receives the related information, content data, and execution event history information.

Next, the sound control unit 58 of the PC 5 sets the playback location of the sound data, which is the content data received in step S129 (step S130). In this case, the sound control unit 58 sets a portion of the received related information that corresponds to the sound data associated with the content generation time slot “00:00” as a playback portion of the sound data.

Next, the generation unit 56 of the PC 5 generates an execution event history screen 580 as shown in FIG. 40 using the related information and content data received in step S129 (step S131). As shown in FIG. 40, the execution event history screen 580 shows content data generated by executed events for each type of content processing. The execution event history screen 580 includes a playback part display area 581 that displays the playback part of recorded data, a text data display area 582 that displays audio text, a snapshot display area 583 that displays snapshots, and materials used in the event. It includes a "material file" button 585 that is pressed when viewing. More specifically, the generation unit 56 generates an execution event to display images related to text data, which is received content data, in the text data display area 582 in the order of sequence numbers included in the received related information. A history screen 580 is generated. The generation unit 56 also generates the execution event history screen 580 so as to display images related to the screen data, which is the received content data, in the snapshot display area 583 in the order of the sequence numbers included in the received related information. generate. Furthermore, the generation unit 56 displays the execution event history screen so that the playback point 581p included in the playback point display area 581 is displayed at the position corresponding to the playback point set in step S130 during the entire playback time of the sound data. Generate 580.

Then, the display control unit 54 of the PC 5 causes the execution event history screen 580 generated by the generation unit 56 to be displayed on the display 508 (step S132). Furthermore, the sound control unit 58 of the PC 5 plays back the sound data at the playback location set in step S130.

Here, the playback point display area 581 includes a playback point 581p indicating the playback point in the total playback time, a slider 581b indicating the playback history (playback process) of the recorded data, and a playback time display image indicating the playback time in the total playback time. 581t is included. Further, the playback point 581p and the slider 581b constitute a seek bar. The seek bar is an operation area that displays the playback location of the recorded data and also specifies the playback location of the recorded data. The user can visually grasp which part of the recorded data is being played back from the beginning to the end by the position of the playback point 581p and the slider 581b. Furthermore, the user can move the playback point 581p using an input means such as the mouse 507, thereby playing back the recorded data from any playback location. The playback point 581p is an example of a playback location specific image. Further, a seek bar including a playback point 581p and a slider 581b is an example of a playback history display image.

Further, in the text data display area 582, images 582a, 582b, 582c, 582d, and 582e indicating text data are displayed in chronological order from top to bottom. Similarly, in the snapshot display area 583, snapshot (captured) images 583a, 583b, and 583c are displayed in chronological order from top to bottom. In this way, the text data display area 582 and the snapshot display area 583 are displayed in parallel in chronological order. As a result, by viewing the image 583b, for example, the user can easily grasp the time period in which the pie chart was explained at the meeting. 582c and 582d can be easily found. Here, images 583a, 583b, and 583c are snapshot images of the entire screen displayed on display 220.

Thereby, the shared system 1 can easily share the execution event history caused by the executed event among the members (participants) who participated in the event. Note that the shared system 1 may be configured such that the user can edit the reference authority for viewing the executed event history by registering or changing the user after the event is executed.

<Execution event history selection process>
41, 44, and 45 are sequence diagrams showing an example of the flow of execution event history selection processing in the shared system according to the embodiment. 42 and 43 are diagrams showing execution event history screens displayed on the PC according to the embodiment. The execution event history selection process will be explained using FIGS. 41 to 45.

First, the accepting unit 52 of the PC 5 accepts the selection of a specific image included in the text data display area 582 of the execution event history screen 580 (step S151). Specifically, for example, as shown in FIG. 42, the user points the mouse pointer p3 at an image (for example, image 582t) indicating specific text data included in the text data display area 582 of the execution event history screen 580. When the image is selected using the text data display area 582, the reception unit 52 accepts the selection of a specific image included in the text data display area 582.

The transmitting/receiving unit 51 of the PC 5 transmits the selection information related to the selection accepted by the receiving unit 52 to the schedule management server 8 (step S152). Here, the selection information includes a content processing ID corresponding to a specific image (for example, image 582t) for which selection is accepted. Thereby, the transmitting/receiving unit 81 of the schedule management server 8 receives the selection information transmitted from the PC 5.

Next, the storage/read processing unit 89 of the schedule management server 8 reads the corresponding related information by searching the related information management DB 8010 using the content processing ID included in the selection information received in step S152 as a search key. (Step S153). More specifically, the storage/read processing unit 89 reads related information associated with the content processing ID included in the selection information from the related information management DB 8010. This related information includes the content generation time period, content processing ID, content processing type, and sequence number. The time period of content occurrence included in the related information is an example of time information. Thereby, the schedule management server 8 can identify the content that occurred in the same time period as the image for which the user's selection was accepted from the data (information) related to various contents included in the related information.

Furthermore, the storage/read processing unit 89 of the schedule management server 8 searches the execution event history management DB 8008 using the content processing ID included in the related information read in step S153 as a search key, thereby searching for the corresponding execution event history. Information is read out (step S154). This execution event history information includes content processing ID, content processing type, and content content.

Then, the storage/reading processing unit 89 reads content data indicating the content of the execution event history from the storage location of the content data indicating the content included in the execution event history information read in step S154 (step S155 ). Then, the transmitting/receiving unit 81 transmits the read related information and content data to the PC 5 (step S156). Thereby, the transmitting/receiving unit 51 of the PC 5 receives the related information and content data.

Next, the sound control unit 58 of the PC 5 sets the playback location corresponding to the sound data that is the content data received in step S156 as the playback location (start time of the sound data) of the sound data (step S157).

Next, the generation unit 56 of the PC 5 generates an execution event history screen 580 based on the content data received in step S156 (step S158). Specifically, the generation unit 56 generates the execution event history screen 580 so that an image related to the text data that is the received content data is displayed in the text data display area 582. Further, the generation unit 56 generates the execution event history screen 580 so as to display an image related to the screen data, which is the received content data, in the snapshot display area 583.

Furthermore, the generation unit 56 generates the execution event history screen 580 so that the playback point 581p included in the playback location display area 581 is displayed at the position related to the sound data set in step S157 during the entire playback time of the sound data. generate.

The display control unit 54 causes the display 508 to display the execution event history screen 580 generated in step S158 (step S159). FIG. 43 shows an execution event history screen 580 displayed on the display 508 in step S159. In the execution event history screen 580 shown in FIG. 43, the position of the reproduction point 581p included in the reproduction part display area 581 has been moved to the time position corresponding to the image 582t selected by the user. In this way, when a specific image is selected from among images related to text data obtained by converting sound data into text, the PC 5 selects a specific image at a time position indicated by the generation time zone of the content related to the selected specific image. Move the playback point 581p. Thereby, the PC 5 can reproduce the sound data of the reproduction location corresponding to the image by simply selecting the image related to the text data for which the user wants to check the sound. Furthermore, by moving the playback point 581p in accordance with the movement of the image related to the text data, the PC 5 allows the user to easily understand which playback point of the total playback time the sound data is being played. can be done.

Further, the processes shown in FIGS. 44 and 45 are processes when content data (execution event history) of a different type from the text data (execution event history) shown in FIG. 41 is selected. FIG. 44 shows processing when an image related to screen data displayed in the snapshot display area 583 is selected. First, the accepting unit 52 of the PC 5 accepts the selection of a specific image included in the snapshot display area 583 of the execution event history screen 580 (step S161). Specifically, for example, when the user selects a specific image (for example, snapshot image 583r) included in the snapshot display area 583 of the execution event history screen 580 using the mouse pointer p3, the reception unit 52 accepts selection of a specific image included in snapshot display area 583. Note that the processing from step S162 to step S169 is similar to the processing from step S152 to step S159 shown in FIG. 41, respectively.

Further, FIG. 45 shows the process when the playback point 581p displayed in the playback part display area 581 is selected. First, the receiving unit 52 of the PC 5 receives a movement of the playback point 581p included in the playback point display area 581 of the execution event history screen 580 (step S181). Specifically, for example, when the user moves the playback point 581p included in the playback part display area 581 of the execution event history screen 580 using the mouse pointer p3, the reception unit 52 moves the playback point 581p included in the playback part display area 581 of the execution event history screen 580. The movement of the included playback point 581p is accepted.

The transmitting/receiving unit 51 of the PC 5 transmits the selection information regarding the movement accepted by the receiving unit 52 to the schedule management server 8 (step S182). Here, the selection information includes a content processing ID related to the sound data of the accepted reproduction location of the destination. Thereby, the transmitting/receiving unit 81 of the schedule management server 8 receives the selection information transmitted from the PC 5. Note that the processing from step S182 to step S189 is the same as the processing from step S152 to step S159 shown in FIG. 41, respectively, so a description thereof will be omitted.

<Text data editing process>
Next, the process of editing each text data displayed on the execution event history screen will be explained using FIGS. 46 to 55.

FIG. 46 is a sequence diagram illustrating an example of the flow of text data editing processing in the shared system according to the embodiment. 47 and 53 are diagrams showing execution event history screens displayed on the PC according to the embodiment. FIG. 48 is a diagram showing a text editing screen displayed on the PC according to the embodiment. FIG. 49 is a conceptual diagram showing the updated text information management table. Note that the description will be made assuming that an execution event history screen 580 as shown in FIG. 40 is displayed on the display 508 of the PC 5.

First, when the user presses the text data display area 582 using the mouse pointer p3, the display control unit 54 of the PC 5 displays the execution event history screen 580a shown in FIG. 47 (step S201). Specifically, when the user uses the mouse pointer p3 to press an image (for example, image 582c) indicating specific text data included in the text data display area 582, the reception unit 52 selects the specific text data. accept. Then, the display control unit 54 displays an execution event history screen 580a that includes an "edit" button 620 displayed at a position corresponding to the specific text data whose selection has been accepted. This "edit" button 620 is pressed to request editing of specific selected text data. The "edit" button 620 is an example of an edit request image.

Next, when the user presses the "edit" button 620 using the mouse pointer p3, the reception unit 52 accepts the selection of the "edit" button 620 and accepts a request to edit text data (step S202). Next, the display control unit 54 displays a text editing screen 630 as shown in FIG. 48 on the execution event history screen 580a displayed on the display 508 (step S203). The text editing screen 630 is an editing screen that accepts editing of the contents of selected specific text data (for example, the text data shown in the image 582c). The text editing screen 630 includes an input area 631 for editing specific text data, a "save" button 633 that is pressed to save specific text data edited in the input area 631, and text data editing processing. It includes a "cancel" button 635 that is pressed to interrupt (stop) the editing process, and a "play" button 637 that is pressed to play the sound data corresponding to the specific text data to be edited.

Next, when the user makes an input to the input area 631, the reception unit 52 accepts editing of specific text data (step S204). Thereby, the user can edit the selected specific text data by inputting into the input area 631. In this case, the function of the PC 5 (for example, the display control unit 54) that edits text data according to the reception unit 52 corresponds to the “first editing unit”. Further, when the reception unit 52 accepts the selection of the “play” button 637, the sound control unit 58 reproduces the sound data corresponding to the specific text data to be edited. Thereby, the user can edit the text data while listening to the sound data corresponding to the specific text data to be edited.

Next, when the reception unit 52 accepts the selection of the “Save” button 633, the transmission/reception unit 51 of the PC 5 transmits text information update request information indicating a text information update request to the schedule management server 8 ( Step S205). The text data edited in step S204, the content processing ID received in step S156, and the user ID accepted in step S112 are transmitted to this text information update request information. The content processing ID in this case is the content processing ID corresponding to the edited text data. Thereby, the transmitting/receiving unit 81 of the schedule management server 8 receives the text information update request information.

The storage/read processing unit 89 of the schedule management server 8 stores the text data received in step S205 in the text information management DB 8012 (see FIG. 15) (step S206). In this case, the text information management DB 8012 stores a text information management table as shown in FIG. 49. FIG. 49 is a conceptual diagram of the updated text information management table. In the text information management table of FIG. 49, text information including the edited text data received in step S205 is added to the text information stored in the text information management table shown in FIG. This text information includes a content processing ID similar to the text data before editing, a text ID (for example, text12.2) whose version has been updated from the text ID of the text data before editing (for example, text12), and the text data is associated with status information indicating that it has been edited. Of these, the status information includes an editor name indicating the editor who edited the text data and an editing date and time indicating the date and time when the text data was edited.

The transmitting/receiving unit 81 of the schedule management server 8 transmits the text data stored in step S206 to the PC 5 (step S207). Thereby, the transmitting/receiving unit 51 of the PC 5 receives the text data transmitted from the schedule management server 8.

The generation unit 56 of the PC 5 generates the execution event history screen 580b shown in FIG. 53 using the text data received in step S207 (step S208). More specifically, the generation unit 56 generates an execution event history screen 580b in which the unedited text data included in the execution event history screen 580a is changed to the edited text data received in step S207.

Then, the display control unit 54 of the PC 5 displays the execution event history screen 580b generated by the generation unit 56 on the display 508 (step S209). As shown in FIG. 53, an image 582c showing the edited text data is displayed in the text data display area 582 included in the execution event history screen 580b. The execution event history screen 580b also includes an "edited" button 625 indicating that the text data shown in the image 582c has been edited. The "edited" button 625 is an example of an edited image. As a result, the user can check whether the text data displayed in the text data display area 582 has been edited by checking the "Edited" button 625 included in the execution event history screen 580b. can do.

As explained above, the PC 5 allows the user to edit the text data displayed on the execution event history screen 580, and displays the edited text data in place of the unedited text data. Text data can be displayed as minutes. Furthermore, even if the accuracy of conversion from sound data to text data is low, the PC 5 allows the user to edit the displayed text data, so that the user can display the executed event history screen 580 as the minutes of the event. It can be used effectively. Furthermore, by displaying the edited text data, the PC 5 allows the user to view the edited text data and reproduce the sound data of the part the user wants to hear.

<<Editing process of sound data accompanying editing of text data>>
Next, with reference to FIGS. 50 to 52, when editing the text data, if the edited text data includes a predetermined blank character, the audio section corresponding to the blank character in the sound data is The operation of editing to make it invalid will be explained.

FIG. 50 is a diagram illustrating an overview of the flow of editing processing of sound data in a shared system according to an embodiment. An overview of the flow of sound data editing processing accompanying text data editing will be explained using FIG.

As described above, the sound data recorded by the electronic whiteboard 2, the image data acquired by snapshot (capture), etc. are transmitted to the shared support server 6 as content data. Then, the shared support server 6 transmits the sound data of the content data to the voice-to-text conversion server 9, and receives the text data converted from the sound data by the voice-to-text conversion server 9. Content data and the like are then transmitted from the shared support server 6, and the execution event history information, related information, and text information are stored and managed in the schedule management server 8.

Then, in the PC 5, when the text data managed as text information by the schedule management server 8 is edited by the process shown in FIG. 46, the edited text data is sent to the schedule management server 8. ((1) Edited text data). The edited text data sent to the schedule management server 8 is then stored in the text information management DB 8012 ((2) Edited text data storage).

Next, the PC 5 determines whether or not the edited text data sent to the schedule management server 8 includes any portions that have been edited with predetermined blanks so as not to be recorded as minutes ((3) Presence of blanks) judgment). Then, when the PC 5 confirms that there is a part edited with blank characters in the edited text data, it identifies the section of sound data corresponding to the text data edited with blank characters, and information about the corresponding part. Audio editing request information including (hidden section information) is transmitted to the schedule management server 8 ((4) audio editing request).

Upon receiving the audio editing request information, the schedule management server 8 acquires the access ID and access password from the shared support server 6 based on the user ID of the PC 5, and determines whether the information is registered in the server authentication management DB 8006. By doing so, the authentication process is performed ((5) Authentication). If the authentication is successful, the schedule management server 8 edits the sound data corresponding to the text data edited with hidden characters so that the sound in the section indicated by the hidden character section information is invalidated ((6) Sound data editing) .

Then, the schedule management server 8 stores the edited sound data in the storage location indicated by the execution event history information ((7) Saves in the storage location indicated by the URL).

FIG. 51 is a sequence diagram illustrating an example of the flow of sound data editing processing in the shared system according to the embodiment. FIG. 52 is a diagram illustrating an example of a text editing screen displayed on the PC according to the embodiment when editing is done to hide text. Next, with reference to FIGS. 51 and 52, details of the flow of sound data editing processing accompanying text data editing will be described.

First, on the text editing screen 630 displayed on the execution event history screen 580a displayed on the display 508, in the input area 631 for editing text data, the user selects a text part (for example, “ Assume that the file ``our organization'') is edited and saved with blank characters (for example, ``'') as shown in FIG. 52. The edited text data is stored in the text information management DB 8012 of the schedule management server 8, as described above with reference to FIG.

Then, the determination unit 57 of the PC 5 determines whether or not there is a portion edited with blank characters in the edited text data sent to the schedule management server 8 (step S221). If the determining unit 57 determines that the edited text data includes a portion edited with hidden characters, the transmitting/receiving unit 51 of the PC 5 transmits the user ID and the sound data corresponding to the edited text data. , the audio editing request information including section information (hidden section information) defined by the start date and time and end date and time of the text data included in the execution event history information received in step S129 of FIG. 38 is scheduled. It is transmitted to the server 8 (step S222). Thereby, the transmitting/receiving unit 81 of the schedule management server 8 receives the audio editing request information. Note that the edited text data is associated with the sound data based on the related information received in step S129 of FIG. 38. Further, as for text data, since the start date and time and the end date and time are associated with each other as described above, it is possible to specify the audio section corresponding to the text data of the corresponding sound data as blank section information.

Upon receiving the audio editing request information, the transmitting/receiving unit 81 of the schedule management server 8 transmits the user ID received in step S222 to the shared support server 6 (step S223). Thereby, the transmitting/receiving unit 61 of the shared support server 6 receives the user ID. Then, the storage/read processing unit 69 of the shared support server 6 searches the user authentication management DB 6001 (see FIG. 9(A)) using the user ID received in step S223 as a search key to find the corresponding organization. The ID is read (step S224).

Next, the storage/read processing unit 69 reads out the corresponding access ID and access password by searching the access management DB 6002 (see FIG. 9(B)) using the organization ID read out in step S224 as a search key (step S224). S225).

Next, the transmitting/receiving unit 61 transmits the read access ID and access password to the schedule management server 8 (step S226). Thereby, the transmitting/receiving unit 81 of the schedule management server 8 receives the access ID and the access password.

Next, in the schedule management server 8, the authentication unit 82 performs authentication using the access ID and the access password (step S227). This authentication is the same process as step S38, so the explanation will be omitted. In this way, by passing authentication based on the access ID and access password that correspond to the organization ID of the organization to which the user of PC5 belongs, a mechanism is adopted to permit editing of sound data, which will be described later, so that unauthorized editing of sound data can be prevented. It is possible to suppress the occurrence of a situation in which this happens. Here, we will continue to explain cases where it is legal.

Next, the sound data editing unit 84 of the schedule management server 8 identifies the sound section corresponding to the edited text data in the sound data defined by the blank section information received in step S222. Then, the sound data editing unit 84 edits and invalidates the sound of the specified sound section in the sound data (step S228). Then, the storage/read processing unit 89 stores (overwrites) the sound data edited by the sound data editing unit 84 in the storage location indicated by the execution event history information read in S126 (step S229).

As mentioned above, when you edit a specific part of the text data that you do not want to keep in the minutes with blank letters, you can not only edit the text data with blank letters, but also correspond to the corresponding text data in the audio data. It is possible to disable the audio in the audio section.

In addition, in the above-mentioned FIGS. 50 to 52, the process of editing the text into hidden characters and then disabling the sound of the corresponding audio section of the sound data (silence, etc.) is explained. Conversely, the process of specifying the audio section of the sound data It is also possible to make the part of the corresponding text data that corresponds to the audio section in red. For example, if the section from 30 seconds to 60 seconds of 10 minutes of sound data is invalidated, the text of the section can be specified if the start date and time of the meeting is known.

<< Edit history viewing process >>
FIG. 54 is a sequence diagram illustrating an example of the flow of editing history viewing processing in the shared system according to the embodiment. FIG. 55 is a diagram showing an edit history screen displayed on the PC according to the embodiment. The process of viewing the editing history of text data displayed on the execution event history screen 580 will be described with reference to FIGS. 54 and 55. Note that the description will be made assuming that an execution event history screen 580b as shown in FIG. 53 is displayed on the display 508 of the PC 5.

First, when the user presses the "edited" button 625, the reception unit 52 of the PC 5 accepts the selection of the "edited" button 625 and accepts a request to view the editing history (step S211). Next, the transmitter/receiver 51 transmits edit history transmission request information indicating a request to transmit the edit history of the text data to the schedule management server 8 (step S212). This edit history transmission request information includes a content processing ID corresponding to the text data that is the subject of the viewing request. Thereby, the transmitting/receiving unit 81 of the schedule management server 8 receives the edit history transmission request information.

Next, the storage/read processing unit 89 of the schedule management server 8 reads the corresponding text information by searching the text information management DB 8012 using the content processing ID received in step S212 as a search key (step S213). In this case, the storage/read processing unit 89 reads out the text information included in the updated text information management table as shown in FIG. For example, when the content processing ID received in step S212 is "c269", the storage/reading processing unit 89 reads text information that includes the text IDs "text12" and "text12.1". The transmitter/receiver 81 transmits the text information read in step S213 to the PC 5 (step S214). This text information includes a content processing ID, text ID, text data, and status information. Thereby, the transmitting/receiving unit 51 of the PC 5 receives the text information transmitted from the schedule management server 8.

Then, the display control unit 54 of the PC 5 displays an edit history screen 640 as shown in FIG. 55 on the execution event history screen 580a displayed on the display 508 (step S215). This editing history screen 640 includes an original data display area 641 where text data before editing is displayed, an editing history display area 643 where text data after editing is displayed, and a "Close" button for closing this editing history screen 640. ” button 649 is included. Of these, the original data display area 641 includes text data associated with status information indicating "Original" among the text information received in step S214. Furthermore, the editing history display area 643 includes text data that is associated with status information indicating the editor's name and editing date and time, among the text information received in step S214. In this edit history display area 643, if the corresponding text data has been edited multiple times, an edited text data display area (644a, 644b) that displays multiple pieces of edited text data is included in the edit date and time indicated in the status information. are displayed in chronological order based on In addition, in the edit history display area 643, based on the status information associated with each text data, the text data is displayed at a position corresponding to the edited text data display area (644a, 644b) that displays each text data. Editor information (645a, 645b) indicating the editor who edited the text data and editing date/time information (646a, 645b) indicating the time when the text data was edited are displayed.

Thereby, the PC 5 can display the execution event history screen 580b on the display 508 to allow the user to confirm that the text data has been edited, and also display the editing history of the edited text data on the editing history screen 640. You can confirm with the user using . In addition, the PC 5 displays the information of the editor who edited the text data and the date and time of editing, together with the content of the text data, so that the user can know the authenticity of the edited text data. can be confirmed.

Although FIGS. 37 to 55 describe the case where the executed event history is viewed using the PC 5, the executed event history can be viewed on the electronic blackboard 2 through similar processing. Furthermore, the processes and screen examples shown in FIGS. 37 to 55 are similar processes and screen displays performed not only on the PC 5 but also on the electronic blackboard 2, video conference terminal 3, or car navigation device 4. Good too.

Furthermore, in FIGS. 37 to 55, the case where the image data related to each screen to be displayed on the PC 5 is generated by the generation unit 56 of the PC 5 has been described, but this image data may also be generated by the schedule management server 8. . In this case, in the schedule management server 8, the generation unit 83 generates image data, and the transmission/reception unit 81 transmits the generated image data to the PC 5.

(Main effects of the embodiment)
As described above, in the shared system 1 according to the present embodiment, when a specific part of text data that you do not want to keep as minutes is edited in blank letters, the text data is not only edited in blank letters, but also , it is possible to invalidate the audio in the audio section corresponding to the text data in the audio data. Furthermore, this can prevent information that was intended to be erased from being shared among saved information. Further, simply by editing the text data, the audio of the corresponding sound data section is automatically edited, so the user's effort can be reduced.

(supplement)
Note that each function of the embodiment described above can be realized by one or more processing circuits. Here, the term "processing circuit" as used herein refers to a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, or a processor designed to execute each function explained above. ASIC (Application Specific Integrated Circuit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), SoC (Sys This includes devices such as GPUs (Graphics Processing Units), GPUs (Graphics Processing Units), and conventional circuit modules. .

Further, in the above-described embodiment, if at least one of the functional units of the electronic whiteboard 2, the PC 5, the shared support server 6, the schedule management server 8, and the speech-to-text conversion server 9 is realized by executing a program, the program is , ROM, etc., and is provided in advance. Furthermore, the programs executed on the electronic whiteboard 2, PC 5, shared support server 6, schedule management server 8, and speech-to-text conversion server 9 according to the above-described embodiments are stored in installable or executable format files on a CD-ROM. The information may be provided by being recorded on a computer-readable recording medium such as a ROM (Compact Disc Read Only Memory), a flexible disk (FD), a CD-R (Compact Disk-Recordable), a DVD, or an SD card. good. Furthermore, the programs executed by the electronic whiteboard 2, PC 5, shared support server 6, schedule management server 8, and speech-to-text conversion server 9 according to the above-described embodiments are stored on a computer connected to a network such as the Internet, The information may be provided by being downloaded via a network. Furthermore, the programs executed by the electronic whiteboard 2, PC 5, shared support server 6, schedule management server 8, and speech-to-text conversion server 9 according to the above-described embodiments are configured to be provided or distributed via a network such as the Internet. It's okay. Furthermore, the programs executed by the electronic whiteboard 2, the PC 5, the shared support server 6, the schedule management server 8, and the speech-to-text conversion server 9 according to the embodiment described above have a module configuration that includes at least one of the functional units described above. In actual hardware, each of the functional units described above is loaded onto the main storage device and generated by the CPU reading out and executing a program from the storage device described above.

1 Shared system 2 Electronic whiteboard 3 Video conference terminal 4 Car navigation device 5 PC
6 Shared support server 8 Schedule management server 9 Speech-to-text conversion server 15 Work area 21 Transmission/reception unit 22 Reception unit 23 Image/sound processing unit 24 Display control unit 25 Judgment unit 28 Acquisition/Provision unit 29 Storage/reading processing unit 51 Transmission/reception unit 52 Reception section 54 Display control section 56 Generation section 57 Judgment section 58 Sound control section 59 Storage/readout processing section 61 Transmission/reception section 62 Authentication section 63 Creation section 64 Generation section 65 Judgment section 69 Storage/readout processing section 81 Transmission/reception section 82 Authentication section 83 Generation section 84 Sound data editing section 89 Storage/reading processing section 91 Transmission/reception section 93 Conversion section 99 Storage/reading processing section 5000 Storage section 5500 Work area 5501 OS
5502a Minutes application 5502b Browser 6000 Storage unit 6001 User authentication management DB
6002 Access management DB
6003 Schedule management DB
6005 Content management DB
8000 Storage unit 8001 User authentication management DB
8002 User management DB
8003 Union management DB
8004 Union reservation management DB
8005 Event management DB
8006 Server authentication management DB
8008 Execution event history management DB
8009 Execution event management DB
8010 Related information management DB
8012 Text information management DB
9000 Storage section

Japanese Patent Application Publication No. 2014-146066

Claims (10)

computer,
an acquisition unit that acquires the sound data and the text data corresponding to the sound data in order to edit the text data converted by the conversion server with respect to the sound data recorded in the video conference in an information processing device;
a first editing unit that edits the text data acquired by the acquisition unit according to an input operation accepted by the reception unit;
a first transmitting unit that transmits the text data edited by the first editing unit to a management server;
a determination unit that determines whether or not the text data edited by the first editing unit includes blank characters;
a second transmission of transmitting, to the management server, an audio editing request for disabling the sound of the audio section in the sound data corresponding to the text data including the hidden characters, if the determination unit determines that the hidden characters are included; Department and
A program to make it work. The second transmitting unit converts hidden section information defined by a start date and time and an end date and time of the sound data output when the text data acquired by the acquisition unit is converted by the conversion server into the audio edit. The program according to claim 1, wherein the program is included in a request and transmitted to the management server. further comprising a display control unit that causes a display device to display the text data acquired by the acquisition unit,
The program according to claim 1 or 2, wherein the first editing section edits the text data displayed on the display device according to the input operation. 4. The display control section displays, when the text data is edited by the first editing section, an edited image indicating that the text data has been edited near the display area of the text data. program. 5. The program according to claim 1, wherein the audio editing request includes identification information of a user of the information processing device for authentication processing when editing the audio data in the management server. an acquisition step of acquiring the sound data and the text data corresponding to the sound data in order to edit the text data converted by the conversion server with respect to the sound data recorded in the video conference in an information processing device;
an editing step of editing the acquired text data according to the input operation accepted by the reception unit;
a first sending step of sending the edited text data to a management server;
a determination step of determining whether or not the edited text data includes hidden characters;
a second sending step of transmitting, to the management server, an audio editing request for disabling audio in a voice section in the sound data corresponding to the text data including the hidden characters, if it is determined that the hidden characters are included;
An information processing method having An information processing device capable of editing text data converted by a conversion server regarding sound data recorded in a video conference,
an acquisition unit that acquires the sound data of the video conference and the text data corresponding to the sound data;
a first editing unit that edits the text data acquired by the acquisition unit according to an input operation accepted by the reception unit;
a first transmitting unit that transmits the text data edited by the first editing unit to a management server;
a determination unit that determines whether or not the text data edited by the first editing unit includes blank characters;
a second transmission of transmitting, to the management server, an audio editing request for disabling the sound of the audio section in the sound data corresponding to the text data including the hidden characters, if the determination unit determines that the hidden characters are included; Department and
An information processing device equipped with A communication system comprising: an information processing device capable of editing text data converted by a conversion server with respect to sound data recorded in a video conference; and a management server that manages the sound data and the text data,
The information processing device includes:
an acquisition unit that acquires the sound data of the video conference and the text data corresponding to the sound data;
a first editing unit that edits the text data acquired by the acquisition unit according to an input operation accepted by the reception unit;
a first transmitting unit that transmits the text data edited by the first editing unit to a management server;
a determination unit that determines whether or not the text data edited by the first editing unit includes blank characters;
a second transmission of transmitting, to the management server, an audio editing request for disabling the sound of the audio section in the sound data corresponding to the text data including the hidden characters, if the determination unit determines that the hidden characters are included; Department and
Equipped with
The management server is
a specifying unit that specifies the audio section corresponding to the text data in the audio data indicated by the audio editing request;
a second editing unit that disables the audio in the audio section specified by the identification unit in the sound data;
communication system with. The audio editing request includes identification information of a user of the information processing device for authentication processing when editing the audio data in the management server,
The management server further includes an authentication unit that performs the authentication process using identification information of the user,
9. The communication system according to claim 8, wherein the second editing unit performs editing to invalidate the audio in the audio section in the audio data when the authentication processing by the authentication unit is successful. 9. The management server further comprises a storage processing unit that saves the sound data edited by the second editing unit by overwriting the sound data before being edited by the second editing unit. 9. The communication system according to 9.

Images