JP2021177321A - Program, displaying method and terminal - Google Patents

Abstract

To facilitate an input of alphabetic information on the basis of sound information into a chat room.SOLUTION: A program that is executed by a user terminal which displays a chat room involving a user and a user of a first terminal different from the user causes the terminal to execute: obtaining sound information by an obtaining unit of the terminal; and causing alphabetic information on the basis of the sound information obtained by the obtaining unit to be input into a selected chat room, and causing the chat room containing the alphabetic information to be displayed on a display unit of the terminal.SELECTED DRAWING: Figure 1-1

Description

The present disclosure relates to programs, display methods, and terminals.

An information processing system related to sound recognition (for example, voice recognition) is known (for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2014-206961

According to the first aspect of the present invention, a program executed by a user's terminal displaying a chat room including a user and a user of a first terminal different from the user acquires sound information by a terminal acquisition unit. That is, the character information based on the sound information acquired by the acquisition unit is input to the selected chat room, and the chat room including the character information is displayed on the display unit of the terminal, which is executed by the terminal.
According to the second aspect of the present invention, the display method of the terminal of the user who displays the chat room including the user and the user of the first terminal different from the user is that the sound information is acquired by the acquisition unit of the terminal. , Character information based on the sound information acquired by the acquisition unit is input to the selected chat room, and the chat room including the character information is displayed on the display unit of the terminal.
According to the third aspect of the present invention, the terminal of the user who displays the chat room including the user and the user of the first terminal different from the user is acquired by the acquisition unit for acquiring the sound information and the acquisition unit. Character information based on sound information is input to the selected chat room, and a display unit for displaying the chat room including the character information is provided.

The figure which shows an example of the structure of the communication system which concerns on embodiment. The flowchart which shows an example of the flow of the process executed by each apparatus which concerns on 1st Embodiment. The figure which shows an example of the information stored in the storage part of the server which concerns on 2nd Example. The figure which shows an example of the messaging application user registration data which concerns on 2nd Example. The figure which shows an example of the group management database which concerns on 2nd Example. The figure which shows an example of the calendar information management database which concerns on 2nd Example. The figure which shows an example of the information stored in the storage part of the terminal which concerns on 2nd Example. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 2nd Example. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 2nd Example. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 2nd Example. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 2nd Example. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 2nd Example. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 2nd Example. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 2nd Example. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 2nd Example. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 2nd Example. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 2nd Example. The flowchart which shows an example of the flow of the process executed by each apparatus which concerns on 2nd Example. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 3rd Example. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 3rd Example. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 3rd Example. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 3rd Example. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 3rd Example. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 3rd Example. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 3rd Example. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 3rd Example. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 3rd Example. The flowchart which shows an example of the flow of the process executed by each apparatus which concerns on 3rd Example. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 4th Embodiment. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 4th Embodiment. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 4th Embodiment. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 4th Embodiment. FIG. 5 is a flowchart showing an example of a flow of processing executed by each device according to the fourth embodiment. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 5th Embodiment. The figure which shows an example of the display format table which concerns on 6th Example. The figure which shows an example of the calendar information management database which concerns on 7th Example. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 7th Embodiment. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 7th Embodiment. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 7th Embodiment. FIG. 5 is a flowchart showing an example of a flow of processing executed by each device according to the fourth embodiment. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 7th modification. The figure which shows an example of the screen displayed on the display part of the terminal which concerns on 7th modification. FIG. 5 is a flowchart showing an example of a flow of processing executed by each device according to the seventh modification.

<Compliance with legal matters>
It should be noted that the disclosures described herein are premised on compliance with the legal matters of the implementing country necessary for the implementation of this disclosure, such as secrecy of communications.

An embodiment for implementing the program or the like according to the present disclosure will be described with reference to the drawings.

In the present specification, the expression "by communication I / F" is used as appropriate. This indicates that the device transmits and receives various information and data via the communication I / F (via the communication unit) based on the control of the control unit (processor or the like), for example, without limitation. ..

<System configuration>
FIG. 1-1 is a diagram showing an example of the system configuration of the communication system 1 in this embodiment.
In the communication system 1, the server 10 and a plurality of terminals 20 (terminals 20A, terminals 20B, terminals 20C, ...) Are connected to each other via a network 30 as an example, not limited to the above.

The server 10 has a function of providing a messaging service to a terminal 20 owned by a user via a network 30. The server 10 can also be expressed as a messaging server, a messaging service server, or the like.

The terminal 20 (terminal 20A, terminal 20B, terminal 20C, ...) Can be any terminal as long as it is an information processing terminal capable of realizing the functions described in each embodiment. The terminal 20 is not limited but, for example, a smartphone, a mobile phone (feature phone), a computer (not limited, for example, a desktop, a laptop, a tablet, etc.), a media computer platform (not limited, for example, a cable, a satellite set). Top boxes, digital video recorders), handheld computer devices (not limited to personal digital assistants, e-mail clients, etc.), wearable terminals (glasses-type devices, clock-type devices, etc.), VR (Virtual Reality) Includes terminals, smart speakers (devices for voice recognition), or other types of computers, or communication platforms. Further, the terminal 20 may be expressed as an information processing terminal.

The configurations of the terminal 20A, the terminal 20B and the terminal 20C are basically the same. Further, if necessary, the terminal used by the user X may be expressed as the terminal 20X, and the user information in the predetermined service associated with the user X or the terminal 20X may be expressed as the user information X.
The user information is user information associated with an account used by the user in a predetermined service. User information is not limited, but as an example, input by the user or given by a predetermined service, the user's name, the user's icon image, the user's age, the user's gender, the user's address, and the user's hobbies. It includes information associated with the user, such as preference, user's identifier, and may or may not be any one or combination of these.

The network 30 plays a role of connecting one or more terminals 20 and one or more servers 10. That is, the network 30 means a communication network that provides a connection route so that data can be transmitted and received after the above-mentioned various devices are connected.
The number of servers 10 and the number of terminals 20 connected to the network 30 are not limited.

One or more parts of the network 30 may or may not be a wired or wireless network. The network 30 is not limited, but by example, an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), and a wireless network. LAN (wireless LAN: WLAN), wide area network (WAN), wireless WAN (wireless WAN: WWAN), metropolitan area network (MAN), part of the Internet, public exchange telephone network (Public) Switched Telephone Network: Part of PSTN, mobile network, ISDN (integrated service digital networks), wireless LAN, LTE (long term evolution), CDMA (code division multiple access), Bluetooth (Bluetooth (registered trademark)), satellite It can include communications, etc., or a combination of two or more of these. The network 30 may include one or more networks 30.

The server 10 (not limited to an example of a server, an information processing device, and an information management device) has a function of providing a predetermined service (messaging service in this embodiment) to the terminal 20. The server 10 may be any device as long as it is an information processing device capable of realizing the functions described in each embodiment. The server 10 is not limited, but by example, a server device, a computer (not limited, by example, a desktop, a laptop, a tablet, etc.), a media computer platform (not limited, by example, a cable, a satellite set-top box, a digital video recorder). ), Handheld computer devices (for example, but not limited to, PDA, e-mail client, etc.), or other types of computers, or communication platforms. Further, the server 10 may be expressed as an information processing device. When it is not necessary to distinguish between the server 10 and the terminal 20, the server 10 and the terminal 20 may or may not be expressed as information processing devices, respectively.

[Hardware (HW) configuration of each device]
The HW configuration of each device included in the communication system 1 will be described.

(1) HW configuration of the terminal FIG. 1-1 shows an example of the HW configuration of the terminal 20.
The terminal 20 includes a control unit 21 (CPU: central processing unit), a storage unit 28, a communication I / F 22 (interface), an input / output unit 23, a clock unit 29A, and a position calculation information detection unit 29B. .. Each component of the HW of the terminal 20 is connected to each other via bus B as an example, but not a limitation. It is not essential that the HW configuration of the terminal 20 includes all the components. As an example, but not a limitation, the terminal 20 may or may not have a configuration in which individual components or a plurality of components are removed.

The communication I / F 22 transmits and receives various data via the network 30. Communication may be executed by wire or wirelessly, and any communication protocol may be used as long as mutual communication can be executed. The communication I / F 22 has a function of executing communication with various devices such as a server 10 via the network 30. The communication I / F 22 transmits various data to various devices such as the server 10 according to an instruction from the control unit 21. Further, the communication I / F 22 receives various data transmitted from various devices such as the server 10 and transmits the various data to the control unit 21. Further, the communication I / F 22 may be simply expressed as a communication unit. Further, when the communication I / F 22 is composed of a physically structured circuit, it may be expressed as a communication circuit.

The input / output unit 23 includes a device for inputting various operations to the terminal 20 and a device for outputting the processing result processed by the terminal 20. The input / output unit 23 may or may not be integrated with the input unit and the output unit, or may be separated into the input unit and the output unit.

The input unit is realized by any or a combination of all kinds of devices capable of receiving input from the user and transmitting information related to the input to the control unit 21. The input unit includes, but is not limited to, hardware keys such as a touch panel, a touch display, and a keyboard, a pointing device such as a mouse, a camera (operation input via a moving image), and a microphone (operation input by voice).

The output unit is realized by any one or a combination of all kinds of devices capable of outputting the processing result processed by the control unit 21. The output unit is not limited and includes, as an example, a touch panel, a touch display, a speaker (audio output), a lens (not limited, as an example, 3D (three dimensions) output or hologram output), a printer, and the like.

As one embodiment, the input / output unit 23 includes, but is not limited to, a display unit 24, a sound input unit 25, a sound output unit 26, and an imaging unit 27 as an example.

The display unit 24 is realized by any or a combination of all kinds of devices capable of displaying according to the display data written in the frame buffer. The display unit 24 is not limited but is an example of a touch panel, a touch display, a monitor (not limited but an example, a liquid crystal display or an Electroluminescence display (OELD)), a head mounted display (HDM), a projection mapping, and a hologram. , Includes a device capable of displaying images, text information, etc. in air (which may or may not be vacuum). It should be noted that these display units 24 may or may not be able to display display data in 3D.

The sound input unit 25 is used for inputting sound data (including voice data; the same applies hereinafter). The sound input unit 25 includes a microphone and the like.
The sound output unit 26 is used for outputting sound data. The sound output unit 26 includes a speaker and the like.
The imaging unit 27 is used to acquire moving image data. The imaging unit 27 includes a camera and the like.

When the input / output unit 23 is a touch panel, the input / output unit 23 and the display unit 24 may be arranged so as to face each other with substantially the same size and shape.

The clock unit 29A is a built-in clock of the terminal 20 and outputs time information (timekeeping information). The clock unit 29A is configured to include, for example, a clock using a crystal oscillator, and the like, without limitation. The clock unit 29A can be expressed as a time measuring unit or a time information detecting unit as an example without limitation.

The clock unit 29A may or may not have a clock to which the NITZ (Network Identity and Time Zone) standard or the like is applied.

The position calculation information detection unit 29B has a function of detecting (measuring) information necessary for the control unit 21 to calculate (measure) the position of its own terminal 20 (hereinafter, referred to as "position calculation information"). It is a department. The position calculation information detection unit 29B can be expressed as a position calculation sensor unit as an example without limitation.

The position calculation information detection unit 29B is not limited, but as an example, a satellite positioning sensor (satellite positioning) which is a sensor or unit for calculating the position of the terminal 20 using a satellite positioning system such as GPS (Global Positioning System). Unit), a sensor for calculating the position of the terminal 20 using an inertial navigation system, an inertial measurement unit (IMU (Inertial Measurement Unit)), and a UWB (Ultra Wide). A UWB positioning sensor (UWB positioning unit), which is a sensor and a unit for calculating the position of the terminal 20 using the Band), and the like are included.

The satellite positioning unit is not limited to, for example, an RF receiving circuit that converts an RF (Radio Frequency) signal including a positioning satellite signal transmitted from a positioning satellite received by an antenna (not shown) into a digital signal. Correlation calculation processing is performed on the digital signal output from the RF reception circuit to capture the positioning satellite signal, and information such as satellite orbit data and time data extracted from the positioning satellite signal is used as position calculation information. It has a baseband processing circuit to output.

The inertial measurement unit has an inertial sensor which is a sensor that detects information necessary for calculating the position of the terminal 20 by inertial navigation calculation. The inertial sensor includes, for example, a three-axis acceleration sensor and a three-axis gyro sensor, and the acceleration detected by the acceleration sensor and the angular velocity detected by the gyro sensor are used as position calculation information. Output.

The UWB positioning unit is not limited, but as an example, is an ultra-wideband RF (Radio Frequency) signal that converts an ultra-wideband RF (Radio Frequency) signal including a positioning ultra-wideband pulse signal transmitted from a positioning beacon received by an antenna (not shown) into a digital signal. It has a wideband RF receiving circuit, a relative position calculation processing circuit that calculates the relative position between the terminal 20 and the positioning beacon based on a digital signal output from the ultra-wideband RF receiving circuit, and the like.
As an example, the UWB positioning unit may function as a positioning beacon by transmitting an ultra-wideband RF signal including a positioning ultra-wideband pulse signal from an antenna (not shown). You don't have to.

The control unit 21 calculates the position of its own terminal 20 at a periodic timing or a specific timing based on the position calculation information detected by the position calculation information detection unit 29B, as an example but not a limitation. The position of the terminal is referred to as "terminal position", and the calculated terminal position is referred to as "calculated terminal position". Then, the control unit 21 associates the calculated terminal position with the calculated date and time and stores the calculated terminal position in the storage unit 28 as the calculated terminal position history data.

The control unit 21 has a physically structured circuit for executing a function realized by a code or an instruction contained in the program, and is not limited, but as an example, a data processing device built in hardware. Is realized by. Therefore, the control unit 21 may or may not be expressed as a control circuit.

The control unit 21 is not limited, but as an example, a central processing unit (CPU), a microprocessor (microprocessor), a processor core (processor core), a multiprocessor (multiprocessor), an ASIC (application-specific integrated circuit), and an FPGA (field programmable). gate array) is included.

The storage unit 28 has a function of storing various programs and various data required for the terminal 20 to operate. The storage unit 28 includes various storage media such as an HDD (hard disk drive), an SSD (solid state drive), a flash memory, a RAM (random access memory), and a ROM (read only memory). Further, the storage unit 28 may or may not be expressed as a memory.

The terminal 20 stores the program P in the storage unit 28, and by executing the program P, the control unit 21 executes the processing as each unit included in the control unit 21. That is, the program P stored in the storage unit 28 causes the terminal 20 to realize each function executed by the control unit 21. Further, this program P may or may not be expressed as a program module.

(2) HW configuration of the server FIG. 1-1 shows an example of the HW configuration of the server 10.
The server 10 includes a control unit 11 (CPU), a storage unit 15, a communication I / F 14 (interface), an input / output unit 12, and a clock unit 19. Each component of the HW of the server 10 is connected to each other via the bus B as an example, but not a limitation. It is not essential that the HW of the server 10 includes all the components as the configuration of the HW of the server 10. As an example, but not a limitation, the HW of the server 10 may or may not be configured to remove individual components or a plurality of components.

The control unit 11 has a physically structured circuit for executing a function realized by a code or an instruction contained in the program, and is not limited, but as an example, a data processing device built in hardware. Is realized by.

The control unit 11 is typically a central processing unit (CPU), and may or may not be a microprocessor, processor core, multiprocessor, ASIC, FPGA, or the like. In the present disclosure, the control unit 11 is not limited to these.

The storage unit 15 has a function of storing various programs and various data required for the server 10 to operate. The storage unit 15 is realized by various storage media such as HDD, SSD, and flash memory. However, in the present disclosure, the storage unit 15 is not limited to these. Further, the storage unit 15 may or may not be expressed as a memory.

The communication I / F 14 transmits / receives various data via the network 30. Communication may be executed by wire or wirelessly, and any communication protocol may be used as long as mutual communication can be executed. The communication I / F 14 has a function of executing communication with various devices such as a terminal 20 via the network 30. The communication I / F 14 transmits various data to various devices such as a terminal 20 according to an instruction from the control unit 11. Further, the communication I / F 14 receives various data transmitted from various devices such as the terminal 20 and transmits the various data to the control unit 11. Further, the communication I / F 14 may be simply expressed as a communication unit. Further, when the communication I / F 14 is composed of a physically structured circuit, it may be expressed as a communication circuit.

The input / output unit 12 is realized by a device that inputs various operations to the server 10.
The input unit is realized by any one or a combination of all kinds of devices capable of receiving an input from a user and transmitting information related to the input to the control unit 11. The input unit is typically realized by a hardware key typified by a keyboard or the like, or a pointing device such as a mouse. The input unit may or may not include a touch panel, a camera (operation input via a moving image), and a microphone (operation input by voice) as an example without limitation.

The output unit is realized by any one or a combination of all kinds of devices capable of outputting the processing result processed by the control unit 11. The output unit is not limited and includes, as an example, a touch panel, a touch display, a speaker (audio output), a lens (not limited, as an example, 3D (three dimensions) output or hologram output), a printer, and the like.

As a primary embodiment, the input / output unit 12 includes a display unit 13.

The display unit 13 is typically realized by a monitor (not limited, but as an example, a liquid crystal display or an OELD (organic electroluminescence display)). The display unit 13 may or may not be a head-mounted display (HDM) or the like. It should be noted that these display units 13 may or may not be able to display display data in 3D. In the present disclosure, the display unit 13 is not limited to these.

The clock unit 19 is a built-in clock of the server 10 and outputs time information (timekeeping information). The clock unit 19 is configured to include an RTC (Real Time Clock) as a hardware clock, a system clock, or the like as an example, not limited to the clock unit 19. The clock unit 19 is not limited, but may be expressed as a time measuring unit or a time information detecting unit as an example.

(3) Others The server 10 stores the program P in the storage unit 15, and by executing the program P, the control unit 11 executes the processing as each unit included in the control unit 11. That is, the program P stored in the storage unit 15 causes the server 10 to realize each function executed by the control unit 11. This program P may or may not be expressed as a program module.
The same applies to other devices.

In each embodiment of the present disclosure, it will be described as realized by executing the program P by the CPU of the terminal 20 and / or the server 10.
The same applies to other devices.

The control unit 21 of the terminal 20 and / or the control unit 11 of the server 10 is formed not only in a CPU having a control circuit but also in an integrated circuit (IC (Integrated Circuit) chip, LSI (Large Scale Integration)) and the like. Each process may or may not be realized by a logic circuit (hardware) or a dedicated circuit. Further, these circuits may be realized by one or a plurality of integrated circuits, and the plurality of processes shown in each embodiment may or may not be realized by one integrated circuit. Further, the LSI may be referred to as a VLSI, a super LSI, an ultra LSI, or the like depending on the degree of integration. Therefore, the control unit 21 may or may not be expressed as a control circuit.
The same applies to other devices.

Further, the program P (for example, a software program, a computer program, or a program module) of each embodiment of the present disclosure may be provided in a state of being stored in a computer-readable storage medium. It does not have to be done. The storage medium can store the program P in a "non-temporary tangible medium". Further, the program P may or may not be for realizing a part of the functions of each embodiment of the present disclosure. Further, it may or may not be a so-called difference file (difference program) that can realize the functions of each embodiment of the present disclosure in combination with the program P already recorded on the storage medium.

The storage medium is one or more semiconductor-based or other integrated circuits (ICs) (such as, but not limited to, field programmable gate arrays (FPGAs) or application-specific ICs (ASICs)), hard disks. Disk drive (HDD), hybrid hard drive (HHD), optical disk, optical disk drive (ODD), optical magnetic disk, optical magnetic drive, floppy diskette, floppy disk drive (FDD), magnetic tape, solid It can include a drive (SSD), a RAM drive, a secure digital card, or drive, any other suitable storage medium, or any suitable combination of two or more of these. The storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile, where appropriate. The storage medium is not limited to these examples, and any device or medium may be used as long as the program P can be stored. Further, the storage medium may or may not be expressed as a memory.

The server 10 and / or the terminal 20 can read the program P stored in the storage medium and execute the read program P to realize the functions of the plurality of functional units shown in each embodiment.
The same applies to other devices.

Further, the program P of the present disclosure may or may not be provided to the server 10 and / or the terminal 20 via an arbitrary transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. .. The server 10 and / or the terminal 20 realizes the functions of the plurality of functional units shown in each embodiment by executing the program P downloaded via the Internet or the like, as an example without limitation.
The same applies to other devices.

In addition, each embodiment of the present disclosure can also be realized in the form of a data signal in which the program P is embodied by electronic transmission.
At least part of the processing in the server 10 and / or the terminal 20 may or may not be realized by cloud computing composed of one or more computers.
At least a part of the processing in the terminal 20 may or may not be performed by the server 10. In this case, at least a part of the processing of each functional unit of the control unit 21 of the terminal 20 may or may not be performed by the server 10.
At least a part of the processing in the server 10 may or may not be performed by the terminal 20. In this case, at least a part of the processing of each functional unit of the control unit 11 of the server 10 may or may not be performed by the terminal 20.
Unless explicitly stated, the configuration of the determination in the embodiment of the present disclosure is not essential, and a predetermined process is operated when the determination condition is satisfied, or a predetermined process is performed when the determination condition is not satisfied. It may or may not be.

The program disclosed in this disclosure is not limited to using script languages such as ActionScript and JavaScript (registered trademark), compiler languages such as Objective-C and Java (registered trademark), and markup languages such as HTML5. Will be implemented.

<Example>
As an example, not limited, multiple users of a group, such as peers belonging to the same department, gather in the real world (offline) to hold a meeting or meeting, and what is discussed at that meeting or meeting. You may want to share online.
In this case, as an example, not limited, voice recognition is performed based on voice data at a conference or meeting in the real world, and after converting the text into data that can be easily viewed and searched, other users and their texts are used. A system for sharing such data is desired.
Hereinafter, examples for realizing this will be described.

In this specification, using a messaging application for realizing a messaging service (MS) including an instant messaging service (IMS), data in which voice data of a meeting or a meeting is documented as an example, not limited to the present. (Hereinafter referred to as "minutes data"), an example of sharing is shown. In the following, a service for exchanging minutes data via a messaging service will be referred to as a "minutes service".

Further, in the following, it will be described that the server 10 is operated and managed by the messaging service provider. Further, in the following, the name of the messaging application will be illustrated and described as appropriate as "Messageing App".

In the present specification, the "content" is a message including simple texts, pictograms, etc. transmitted and received between a plurality of terminals 20 in a messaging service, and is not limited to, but as an example, image information (still image, moving image, etc.). ), Operation information (including buttons, icons, etc.), communication information / link information (including URI, URL, etc.), and various other information that can be transmitted and received between the terminals 20.

The "chat room" is a virtual room for one or more users of the terminal 20 to communicate using a data communication line on a computer network.
As an example, but not a limitation, a chat room can be said to be a "talk room" when content is exchanged using a messaging service in a chat room.

The talk room includes, as an example, not limited to, a talk room for users to talk one-on-one, and a group talk room for talking with a group including a plurality of users formed in a messaging service. ..

Further, the messaging service may or may not include an instant messaging service (IMS: Instant Messaging Service) that enables transmission and reception of contents such as simple messages between terminals 20.

The messaging service: MS (including IMS) can also be considered as one form (one form) of the social networking service: SNS.
Therefore, MS and SNS may or may not be distinguished.

Further, in the following, the user of the terminal 20A will be referred to as "user A.A", the user of the terminal 20B will be referred to as "user BB", the user of the terminal 20C will be referred to as "user CC", and so on.

Further, the messaging application may be provided by the server 10 as a single application having no so-called schedule management service function, or may be provided by the server 10 as a complex application having a schedule management service function. You may do so.

Further, in the examples described below, character information (character data) included in the minutes will be illustrated as character information in the present disclosure.
However, the character information in the present disclosure is not limited to the character information, and other than this, the information such as symbols, contents, and images may be generally displayed in the chat room without limitation. Possible information may also be included in the textual information in this disclosure.

The following two methods can be broadly considered as methods for creating and sharing minutes data in order to realize the minutes service. In the following examples, each method will be described.
(1) Method of executing voice recognition on the server 10 and creating minutes data (2) Method of executing voice recognition on the terminal 20 and creating minutes data

In addition, the timing of creating and sharing the minutes data can be roughly divided into the following three methods.
(A) Create and share minutes data after the meeting / meeting (B) Create / share the minutes data before the meeting / meeting starts, and set the minutes data in real time during the meeting / meeting. Create / share (C) Create / share minutes data before the meeting / meeting starts, and create / share the minutes data after the meeting / meeting ends.

<First Example>
The first embodiment is an example according to the method (2) above.
In the first embodiment, in the minutes service, as an example, not limited, sound data (voice data as an example, not limited; the same shall apply hereinafter) is acquired in the terminal 20 on which the messaging application operates, and is based on the sound data. And create minutes data including character data. Then, the created minutes data is displayed in the chat room (not limited, but as an example, the talk room) of the messaging application.
The contents described in the first embodiment can be applied to any of the other embodiments and the other modifications.

In this embodiment, the chat room is not limited to the user A. of the terminal 20A as an example. A and the user of the terminal 20 other than the terminal 20A, not limited to, as an example, the user B. of the terminal 20B. It is assumed that B is at least participating.

<Processing>
FIG. 1-2 is a flowchart showing an example of the flow of processing executed by the control unit 21 of the terminal 20 in this embodiment.
This flowchart is an example of the processing executed by the control unit 21 of the terminal 20A.

The processes described below are merely examples of processes for realizing the method of the present disclosure, and are not limited to these processes.
Another step may be added to the process described below, or some steps may be omitted (deleted).
This is the same for each flowchart (process) described below.

First, the control unit 21 of the terminal 20A accepts the input to the sound input unit 25, and acquires voice data including the dialogue content of the user as an example, not limited to (A100).

In addition to the voice that is the content of the user's dialogue, the voice data is not limited to data such as the sound of a bell making a "ringing" sound and the sound of a dog making a "one-one" sound. May or may not be included. That is, sound data other than human voice may or may not be included in the voice data.
Further, the human voice data and the other sound data may or may not be distinguished.

Next, the control unit 21 of the terminal 20A executes the voice recognition process as an example, not limited to, on the acquired voice data. Then, the control unit 21 of the terminal 20A converts the voice data into the minutes data which is the character (text) data based on the result of the voice recognition process (A120). It can be said that this process is a process of generating minutes data which is character information from sound information.

The voice recognition process is not limited to converting the utterance content into a character string. As an example, not a limitation, the voice recognition process may include an onomatopoeia conversion process that converts the ringing sound of a bell into the character "Rinrin" and the sound of a dog ringing into the character "Wanwan". It does not have to be.

In addition, in the minutes data conversion process, speaker recognition processing is performed on the voice data, and based on the speaker recognition result, the minutes data in which the character data and the speaker data are linked is generated. It may or may not be.

Then, the control unit 21 of the terminal 20A uses the converted (generated) minutes data as an example of the messaging application, not limited to the user A. A and user B. Input in addition to the chat room (talk room) including B (A160). That is, the character data included in the minutes data is shared among a plurality of users by the chat room of the messaging application.

Finally, the control unit 21 of the terminal 20A causes the display unit 24 to display the minutes data input to the chat room (A180). Then, the control unit 21 of the terminal 20A ends the process.

<Effect of the first embodiment>
In the first embodiment, the user A. A (not limited, but an example of a terminal user) and user B. Terminal 20A (an example of a terminal, not a limitation) that displays a chat room including B (an example of a user of a first terminal different from the user of the terminal, not a limitation) has voice data and information related to voice (an example of a terminal, not a limitation). (An example of sound information) is acquired by the sound input unit 25 (not limited, but an example of a terminal acquisition unit). Then, the minutes data (not limited, but an example of character information) based on the acquired voice data is used by the selected user A. A and user B. The chat room that is input to the chat room including B (an example of the selected chat room, not limited) and contains the minutes data is displayed on the display unit 24 of the terminal 20A (an example of the display unit of the terminal, not limited). Shows the configuration to be displayed in.
As an example of the effect of the embodiment obtained by such a configuration, the user of the terminal and the user of the first terminal can share character information based on the sound information displayed in the chat room. As a result, a plurality of users can refer to the character information based on the sound information, and the convenience of the user can be improved.
Further, the user of the terminal and the user of the first terminal can browse the character information which is more excellent in the list than the sound information. As a result, the user of the terminal can accurately obtain the necessary information, and the convenience of the user can be improved.

<First modification (1)>
In the first embodiment, the method (2) above is applied, and the minutes data conversion process is executed on the terminal 20 based on the voice data to create the minutes data, but the present invention is not limited to this. ..
As an example, not a limitation, the method (1) above may be applied.

In this case, as an example, not limited to, when the control unit 21 of the terminal 20 executes the step A100 of FIG. 1-2, the acquired voice data is transmitted to the server 10 by the communication I / F 22.
Then, the control unit 11 of the server 10 executes the minutes data conversion process based on the received voice data. Then, the control unit 11 of the server 10 transmits the generated minutes data to the terminal 20 by the communication I / F14.

As an example of the effect of the modified example obtained by such a configuration, the minutes data conversion process including the voice recognition process, which has a high calculation cost, is controlled by the server, which has less restrictions such as thermal design power than the control unit of the terminal. It can be executed in the department. As a result, the processing speed can be increased. In addition, it is possible to prevent the battery in the terminal from being consumed.

<First modification (2)>
In the first embodiment, adding (inputting) the minutes data to the chat room is executed by the terminal 20, but the present invention is not limited to this. As an example, but not a limitation, the server 10 may or may not be added to the chat room.

In this case, as an example, not limited to, when the control unit 21 of the terminal 20 executes the step A120 of FIG. 1-2, the minutes data is transmitted to the server 10 by the communication I / F22. Further, the control unit 21 of the terminal 20 acquires the minutes posting destination selection information for designating the chat room to which the minutes data is added based on the input to the input / output unit 23 of the user, as an example, not limited to. .. Then, the control unit 21 of the terminal 20 transmits the minutes posting destination selection information to the server 10 by the communication I / F 22.

Then, the control unit 11 of the server 10 adds (inputs) the minutes data to the chat room based on the received minutes data and the minutes posting destination selection information. Then, the control unit 11 of the server 10 transmits the contents of the chat room in which the minutes data is input to the terminal 20 by the communication I / F14. After that, the terminal 20 executes the step A180 of FIG. 1-2.

When the minutes data is generated by the server 10 as described in the first modification (1), the terminal 20 need only transmit the minutes posting destination selection information to the server 10.

In this modification, the chat room is selected based on the input (minutes posting destination selection information) by the user of the terminal.
As an example of the effect of the modified example obtained by such a configuration, the user can specify an arbitrary chat room as an input destination of character information. Thereby, the convenience of the user can be improved.

<Second Example>
The second embodiment is an example according to the method (1) relating to the timing of (A) above.
In the second embodiment, in the minutes service, the control unit 21 of the terminal 20 transmits the voice data stored in the storage unit 28 of the terminal 20 to the server 10. When the control unit 11 of the server 10 creates the minutes data based on the received voice data, the control unit 11 transmits the minutes data to the terminal 20.
Hereinafter, the method according to the second embodiment will be referred to as a "batch addition mode".
The contents described in the second embodiment can be applied to any of the other embodiments and the other modifications.

<Data structure>
(1) Server FIG. 2-1 is a diagram showing an example of information stored in the storage unit 15 of the server 10 in this embodiment.
The storage unit 15 is not limited, but as an example, the MS management processing program 151 which is read by the control unit 11 and executed as the messaging application management process, and is read by the control unit 11 and executed as the voice data recognition process. The voice recognition program 153, the MS user registration data 155, the group management database 157, and the calendar information management database 159 are stored.

The MS user registration data 155 is registration data relating to the terminal 20 that uses the messaging application or the user of the terminal 20, and an example of the data structure is shown in FIG. 2-2.
In the MS user registration data 155, as an example, the user name, the MS application ID, the terminal telephone number, and other registration information are stored in association with each other.

The user name is the name of the user of the terminal 20 who uses the messaging application, and the name registered when the user of the terminal 20 uses the messaging application is stored as an example without limitation.

The MS application ID is the information used to identify the account of the messaging application, or the account itself. This MS application ID is preferably a unique value for each account, and is not limited, but as an example, a unique value (unique value) for each account is set and stored by the server 10. The MS application ID is information associated with the terminal 20 or the user of the terminal 20, and is an example of information about the terminal or information about the user of the terminal.

The terminal telephone number is the telephone number of the terminal 20 of the user with this user name, and the telephone number of the terminal 20 registered when the user of the terminal 20 uses the messaging application is stored as an example without limitation.

Other registration information includes, as an example, not a limitation, the ID of the terminal 20: the terminal ID (IMEI (International Mobile Equipment Identity), as an example, not the limitation), and the email address (terminal email address) of the terminal 20 of the user with this user name. ), Information such as authentication information such as passwords (login password, authentication password) used for various authentications in messaging applications can be included.

The identification information for identifying the terminal 20 is not limited, but may be a terminal ID as an example.
Further, the identification information for identifying the user of the terminal 20 is not limited, but may be an MS application ID as an example. It should be noted that this may or may not be a "user ID".

Further, in the case of an application in which only one account can be registered for one terminal 20, the application is not limited, and as an example, "identification information for identifying the terminal 20 = identification information for identifying the user of the terminal 20 =" It can be "MS application ID".

As an example, not limited to this, it may or may not be possible to assign a plurality of terminal IDs to one user ID.

The group management database 157 is a database that stores data for managing information about a group created by a user in a messaging service, and an example of the data structure is shown in FIG. 2-3.

Group management data is stored in the group management database 157 as management data for each group.
The group ID, the group name, and the group member data are stored in each group management data as an example without limitation.

The group ID is an ID that functions as identification information for identifying the group. This group ID is not limited, but as an example, a unique ID is set and stored for each group by the server 10.

The group name is the name of the group identified by the group ID.

The group member data is data related to a member (user) belonging to the group, and is not limited but is stored as an example in which the MS application ID, which is the information of the member belonging to the group, and the user name are associated with each other.

The calendar information management database 159 is a database for managing information about a schedule management application that functions in cooperation with the messaging application. In the present embodiment, the schedule management application service will be described as being provided on the same server as the server 10 of the messaging application as an example without limitation.

The schedule management application service may or may not be provided by a server different from the server 10.
Further, it may have a schedule management function as one of the functions in the messaging application.
The calendar information management database 159 may be excluded from the configuration of this embodiment.

Figure 2-4 shows an example of the data configuration of the calendar information management database 159A, which is an example of the calendar information management database 159 in this embodiment.
The calendar information management database 159A stores calendar information management data as management data for each event registered in the calendar of the schedule management application.

Each calendar information management data is not limited, but as an example, event ID, MS application ID, event name, event start date and time, event end date and time, event venue, related URI, and invitee data. Is remembered.

The event ID is an ID that functions as identification information for identifying an event. This event ID is not limited, but as an example, a unique ID is set and stored for each event by the server 10.

The MS application ID is an ID that functions as identification information for identifying the registrant of the event. The MS application ID is not limited to, but as an example, the identification information of the messaging application is used.

The event name is the name of the event identified by the event ID.

The event start date and time is the date and time when the event identified by the event ID is scheduled to start.

The event end date and time is the date and time when the event identified by the event ID is scheduled to end.

The event venue is the name of the venue where the event identified by the event ID will be held.
The event venue may or may not be stored by adding location information that can be identified by latitude and longitude as an example, not limited to the name of the venue.

A related URI (Uniform Resource Identifier) is information for identifying a resource (file) accessible via the Internet, as an example, not limited, in which information such as materials related to the event identified by the event ID is stored. Is.

In the invitee data, as information of the user invited to the event identified by the event ID, the MS application ID and the user name are stored in association with each other as an example, not limited to the information. As an example, not limited to, user information (MS application ID / user name) invited to the event is added and stored in the invitee data based on the operation input to the schedule management application of the event registrant. Then, this calendar information management data is shared with the users registered in the invitee data.

When all the members belonging to the group in the event are invited in a batch, the group ID and the group name may or may not be stored in the invitee data in association with each other. ..

(2) Terminal FIG. 2-5 is a diagram showing an example of information stored in the storage unit 28 of the terminal 20 in this embodiment.
The storage unit 28 stores, as an example, not limited to, an MS processing program 281 read by the control unit 21 and executed as a messaging application process, an MS application ID 283, and a voice database 285.

The MS application ID 283 is an MS application ID associated with the user of the own terminal 20 or the own terminal 20.

The voice database 285 is, but not limited to, a database that stores voice data acquired by the sound input unit 25 of the terminal 20 as an example.

The data stored in the audio database 285 may or may not include the video data captured by the imaging unit 27 of the terminal 20 as an example without limitation.

Further, the data stored in the voice database 285 is not limited to the data acquired by the terminal 20. As an example, not limited, audio (video) data acquired by a terminal or an electronic device (not limited, as an example, a video conference device) different from the terminal 20 may be data received by the terminal 20. You don't have to.

<Display screen>
In the following, the case where the terminal 20 is a smartphone including the display unit 24 of a vertically long display will be illustrated as an example, not a limitation.

As an example, not limited to a smartphone, a touch panel that functions as an input unit is arranged to face the display, thereby forming a touch screen. When an element such as an icon, button, item, or input area is displayed on the display, when the user operates a part of the touch panel area facing the area where the element is displayed, the area is the same. The program associated with the element or the subroutine of that program is executed.

In the following, the operation by the user will be described as a tap (tap operation) as an example without limitation.
Tap (tap operation) is not limited to, but as an example, the user touches the display unit 24 (touch screen), which is integrally composed of a touch panel, with a finger or a pen tip, and then releases the touch panel. It is an operation.

The transition of the display screen described below is only an example of the transition of the display screen for realizing the method of the present disclosure. Regarding the transition of the display screens illustrated below, the display of some display screens may be omitted, or another display screen may be added.

FIG. 2-6 is a diagram showing an example of a home screen of a messaging application displayed on the display unit 24 of the terminal 20 in this embodiment. This display screen is displayed by the user A. This is an example of a screen displayed on the display unit 24 of the terminal 20A, which is the terminal 20 of the A.
In the center of the top of the screen, the characters "Messageing App" are displayed as the name of the messaging application. Further, on the upper right side of the screen, an icon image and a user name (user AA in this example) of the user of the terminal 20 in the messaging application are displayed.

Below that, a current position display area indicating the current position in the messaging application is configured. In this example, the character "home" indicating that the current position is the home of the messaging application displays the current position. It is displayed in the area.

Below the current position display area, a messaging application function search area, which is an input area for searching for any function in the messaging application, is configured. In this example, a "search" prompting the user to enter a search keyword is configured. Is displayed in the messaging application function search area.

Below the messaging application function search area, icon images and user names of the user of the terminal 20 in the messaging application are displayed, and below that, a plurality of function icons corresponding to various functions of the messaging application are displayed. .. Function icons include, but are not limited to, icons corresponding to each of a plurality of functions, such as "friend", "business operator", and "service".

"Friend" is one of the functions of the messaging application, and a user of one terminal 20 (one account) adds a user of another terminal 20 (another account) as a friend from his / her own terminal 20. By executing the process for adding / registering friends (friend addition process, friend registration process) by the server 10, messages are sent and received between the terminals registered as friends via the server 10. Etc. can be performed. An account of a messaging application owned by an individual user is called a "general account". The history of messages sent and received between the terminals 20 of the user who owns the general account is displayed as a talk room or the like on the display unit 24 of the terminal 20.

The "business operator" is not a limitation but an example, which is a kind of the above-mentioned friend's function, and is not a general account but an account of a messaging application owned by a user of the business operator (hereinafter referred to as "official account"). By registering as a friend, it is possible to receive and display a message sent from a business operator having an official account on the terminal 20 via the server 10.

It should be noted that the devices and terminals of the business operator who owns the official account can also send and receive messages to and from other devices via the server 10 in the same manner as the terminals 20 of the individual users who own the general account. It can be possible. As an example, not a limitation, a message is sent from the terminal 20 of a user who owns a general account to the device or terminal of a business operator who owns an official account, as in the case where a message is sent from one terminal 20 to another terminal 20. The message is also delivered to the device or terminal of the business operator via the server 10. Then, the history of the message can be displayed as a talk room or the like on the display unit (not shown) of the device or terminal of the business operator.

FIG. 2-6 shows a state in which the function icon of "service" is tapped among the plurality of function icons. Then, an icon indicating a recommended service is displayed below the display area of the function icon, and the icon for using the minutes service is included in this icon. When the icon for using this minutes service is tapped, the audio used to create the minutes data shown in FIG. 2-7 is displayed on the displayed home screen as an example, not as a limitation. The audio data list information display area WR1 for displaying the list of data stored in the database 285 is displayed. The data displayed in the voice data list information display area WR1 is the sound input unit as an example, not a limitation, according to a meeting or a meeting before the chat room (a talk room as an example, not a limitation) is selected. It is the data acquired (recorded) in advance by 25.

In this example, in the voice data list information display area WR1, a file name for identifying the voice data (not limited but as an example "voice2020214") and a date and time when the voice data was created (not limited but as an example "2020"). .02.14 10:54 ") and the time indicating the length of the voice data (" 1:07:35 "as an example, not a limitation) are displayed together for each line. In addition, at the right end of this line, there is a voice data confirmation function icon represented by a speaker mark for outputting the voice data recorded by being tapped by the user by the sound output unit 26 and checking the recorded contents. It is displayed.

The data stored in the voice database 285 is not limited to voice data. As an example, not limited to, video data including audio data may or may not be stored.
In this case, as an example, not limited, the audio data confirmation function icon is a video data confirmation function icon represented by a right triangle mark or the like, as an example, not limited, for outputting and previewing the video to the display unit 24. can do.

When the user taps the line indicating the audio data (in this example, the second line "voice20200228") that creates the minutes data in the minutes service, it is not limited, but as an example, as shown in FIG. 2-8. , The minutes title input area WR2 is displayed, which prompts the input of a name for identifying the minutes data generated from this audio data.

As an example, not a limitation, in FIG. 2-7, a line indicating a plurality of voice data in the voice data list information display area WR1 is selected by the user (tapped as an example, not a limitation), followed by a voice (not shown). One minute data may or may not be created from a plurality of voice data by tapping the data selection decision button or the like.

In the example of FIG. 2-8, the characters of the minutes title "project X February meeting" are input to the minutes title input area WR2 as an example, not limited to, based on the user operation.
The user taps the function button of the character "save", which is a function button for starting the creation of the minutes data regarding the input contents of the minutes title input area WR2. Then, the control unit 21 of the terminal 20 transmits the voice data selected in FIG. 2-7 (data of the file name “voice20200228” in this example) to the server 10 by the communication I / F 22.

When the voice data is received from the terminal 20 by the communication I / F 14, the control unit 11 of the server 10 executes the minutes data conversion process of converting the received voice data into the minutes data. Then, based on the processing result of the minutes data conversion process, the minutes submission candidate selection process described later is executed.

FIG. 2-9 is an example of a screen for selecting the posting destination (sharing target) of the minutes data displayed on the display unit 24 of the terminal 20 after the control unit 11 of the server 10 executes the minutes posting candidate selection process. It is a figure which shows. In FIG. 2-9, the posting destination talk room selection area WR3 is displayed as an example, not the limitation, on the displayed home screen as an example, not the limitation.

In the posting destination talk room selection area WR3, the characters "Please select the talking room to post" are displayed as an example, not as a limitation, in order to prompt the user to select the posting destination of the minutes data.

Below that, the posting of the minutes data selected by the minutes posting candidate selection process among the group to which the user of the terminal 20A (in this case, the user AA) belongs as a member, or the friends of the user of the terminal 20A. Displaying destination candidates The posting destination candidate display area is displayed. In this example, the number of posting destination candidates is "2" in the posting destination candidate display area, and below that, the group with the group name "Second Development Office" and the group name "Project X" are listed as posting destination candidates. It is displayed that the group of "" has been selected. In addition, the number of members belonging to the group is displayed in parentheses on the right side of each group name. In this example, as an example, not limited, the number of members constituting the group "Second Development Office" is "8", and the number of members constituting the group "Project X" is "5".

In the posting destination talk room selection area WR3, a posting destination group display area for displaying the group to which the user of the terminal 20A belongs as a member is displayed below the posting destination candidate display area. Then, below the posting destination group display area, a posting destination friend display area for displaying the friends of the user of the terminal 20A is displayed.
The display contents of the posting destination group display area and the posting destination friend display area are not limited, but are the same as the posting destination candidate display area, and thus the description thereof will be omitted.

It should be noted that the posting destination talk room selection area WR3 may or may not display only the posting destination candidate display area.
Further, the posting destination group display area and the posting destination friend display area may be displayed in the posting destination talk room selection area WR3, and the posting destination candidate display area may not be displayed. Further, in this case, it is not necessary to execute the minutes submission candidate selection process.

When a group or friend is tapped by the user in the posting destination talk room selection area WR3, the group or friend is selected as the posting destination (sharing target) of the minutes data. In this example, the state in which the group "Project X" is selected by the user is shown.

The most plausible posting candidate (first minutes posting candidate) obtained by the minutes posting candidate selection process is automatically selected as the posting destination of the minutes data, not the user operation for the posting destination talk room selection area WR3. It may or may not be. In this case, the posting destination talk room selection area WR3 does not have to be displayed.

After that, the minutes content confirmation display area WR4 as shown in FIG. 2-10 for confirming the contents of the minutes data converted (generated) by the minutes data conversion process is displayed on the display unit 24.

In the minutes content confirmation display area WR4, as an example, not limited to, content (minutes data) obtained by converting the conversation content in the voice data into characters is displayed in chronological order based on the voice data.

In the minutes content confirmation display area WR4, as an example, not limited to the user A. of the terminal 20A which is the own terminal, according to the content display format in the talk room of the messaging application. The remarks of A and the remarks of other users are displayed as contents in balloons on the left side.
It should be noted that the identification of the user who has spoken in the voice data is not limited, but can be executed in the minutes data conversion process as an example.

Specifically, in FIG. 2-10, in the upper part of the minutes content confirmation display area WR4, the characters "Project X February meeting", which is the minutes title input to the minutes title input area WR2, are displayed on the minutes. It is displayed in the title display content TTL1. Since the minutes title display content TTL1 is not the utterance content in the voice data, it is not limited but is displayed in the minutes content confirmation display area WR4 in a centered manner as an example.

Under the minutes title display content TTL1, at the beginning of the audio data, the user A. The text content CT1 based on A's utterance "Then we will start the meeting in February 2020" is displayed. On the left side of the text content CT1, the voice of this content content is displayed by the user A. The characters "00:00:00" indicating the time from the beginning of the voice data spoken by A are displayed.

In this specification, the description will be referred to as "text content", but this may be referred to as "message (message information)" in the messaging application.

Under the text content CT2, the user A. The text content CT2 based on A saying "Is it noisy outside, can you hear me?" Is displayed.
As an example, not a limitation, a "?" Symbol indicating that the spoken sentence has been identified as an interrogative form is added to the end of the sentence of the text content CT2.

Under the text content CT3, the user B. The text content CT3 based on B's utterance "When was the product released?" Is displayed. The "?" Symbol at the end of the text content CT3 sentence indicates that this sentence is questionable, as in the text content CT2.
Similarly, the text content is displayed based on the utterance content in the voice data.

The text content CT4 is the user B. This is text content based on B saying, "I'll check with the content planning office just in case." In the text content CT4, as an example, not limited, the mention function in the messaging application automatically works for the group "content planning room" to which the user of the terminal 20A belongs as a member based on the utterance "content planning department". The underlined notation "@Content Planning Department" is used to indicate.

Here, the mention function is a function of a messaging application for sending a message (content) to a specified user.
Note that the terminal of the user who receives the message by the mention function may or may not display a notification emphasizing that the message has been received.

As an example, not a limitation, when the mention function in the text content CT4 is activated, the user B. A message is sent with B as the sender, saying "I will check with the Content Planning Department just in case." That is, the members of the content planning department are user B. It can be easily determined that an important message has been received from B.

The text content CT5 is the user E.I. This is text content based on E's utterance, "What do you think of Mr. DD?" In the text content CT5, as an example, not limited to, the user D. who is a member belonging to the group for posting the minutes data. For D, the underlined notation "@ DD" indicates that the mention function automatically operates based on the utterance "DD".

In this way, when the user name or group name is spoken in the voice data, the mention function can be automatically activated according to the spoken name in the minutes data conversion process.

However, the mention function is not essential, and the mention function may not work.
Further, as an example, not limited to, the mention function may or may not be activated only for users or groups who have not attended the conference or meeting for which voice data has been acquired, based on the user operation on the terminal 20. May be good. Alternatively, the mention function may or may not be activated only for the users or groups who attended the conference or the meeting from which the voice data was acquired. Alternatively, the mention function may or may not work only for specific users or groups set by the user.

In addition, the target for which the mention function works is not limited to the user name and the group name. As an example, not a limitation, when a predetermined keyword is uttered, the mention function may or may not work for the keyword.

At the bottom of the minutes content confirmation display area WR4, the minutes data is confirmed with the contents displayed in the minutes content confirmation display area WR4, and "Post" for posting to the talk room selected as the posting destination. The minutes posting button SBT1 including the characters "" is displayed.

FIG. 2-11 is an example of a screen for correcting the minutes data displayed in the minutes content confirmation display area WR4 shown in FIG. 2-10.
In FIG. 2-11, as an example, not limited to, when the text content CT2 is tapped by the user, the text content CT2 is highlighted, and the content edit for correcting the content related to the text content CT2 on the text content CT2. The selection menu MBT1 is displayed.

In this example, the content edit selection menu MBT1 is configured to enable three types of editing operations, "delete", "add line spacing", and "comment", as an example, not limited.

The “deletion” is an editing operation for deleting the selected text content CT2 as an example, not a limitation.
The "line spacing addition" is not a limitation, but is, as an example, an editing operation of inserting and displaying a text content or the like under the selected text content CT2.
The "comment" is not a limitation but an editing operation for adding a comment to the selected text content CT2 as an example.

The editing operations that can be selected in the content editing selection menu MBT1 are not limited to the above three types. As an example, not limited to, it may be possible to select an editing operation for deleting a comment, an editing operation for adding a URI (Uniform Resource Identifier) that can refer to the content according to the content of the text content, and so on. It does not have to be.

In FIG. 2-11, a state in which the "Delete" button, which is an editing operation for deleting the text content CT2, is tapped based on a user operation is shown as an example, not a limitation.

FIG. 2-12 is an example of a screen showing the result of performing the editing operations of “delete” and “add line spacing” on the content edit selection menu MBT1 in FIG. 2-11.
In FIG. 2-12, instead of the text content CT2, the edited text content MCT1 indicating that the content of the text content CT2 has been deleted is displayed by a strikethrough. Further, under the edited text content MCT1, the text "Talking about the scheduled release date", which is the text inserted by the "add line spacing" operation, is displayed in the edited text content MCT2.
Since the edited text content MCT2 is not the utterance content in the voice data, it is displayed in the minutes content confirmation display area WR4 in the center as an example, not as a limitation.

In the "add line spacing" operation, the edited text content may or may not be displayed in the same manner as the user's utterance content by adding user information and inputting the text. May be good.

In FIG. 2-12, as an example, not limited to, when the minutes posting button SBT1 is tapped by the user, the minutes data displayed in the minutes content confirmation display area WR4 at the time of tapping is used as the posting destination. It will be posted to the selected talk room (in this example, the talk room of the group "Project X").

FIG. 2-13 shows User A. After A taps the minutes posting button SBT1 and posts the minutes data to the group "Project X", the user B. It is a figure which shows an example of the screen of the talk room of the messaging application displayed on the display part 24 of the terminal 20B which is the terminal 20 of B.

In the current position display area indicating the current position in the messaging application, characters indicating that the current position is the talk room of the group "Project X" are displayed.
Below the current position display area, the content display area TR1 for displaying the content (message) transmitted (posted) to the talk room is displayed.
Further, below the content display area TR1, a content input area for inputting / selecting the content to be transmitted to the talk room is displayed.

In FIG. 2-13, the content display area TR1 is not limited to the user A. Under the text content of the characters "The minutes of the previous meeting" sent by A, the minutes data summary content PRC1 is displayed based on the minutes data posted by the operation of Fig. 2-12. .. That is, each operation shown in FIGS. 2-6 to 2-12 is performed by the user A. It shows that A was executed after sending the text content of the words "The minutes of the previous meeting" to the talk room of the group "Project X".

Minutes data summary content PRC1 is not limited, but as an example, the characters of the minutes title "Project X February meeting" entered in the minutes title input area WR2 and the date and time when the minutes data was last modified. The characters "Final correction 2020/02/28/16: 30" are displayed. In addition, the time when the minutes data was transmitted (posted) to this talk room is displayed at the lower right of the minutes data summary content PRC1.

Minutes Data Summary Under Content PRC1, as an example, not a limitation, User E.I. The text content of the characters "There was a correction, so I'll fix it" sent by E is displayed. Below that, user B. The text content of the characters "Thank you for the minutes. I will confirm it." Sent by B is displayed.
In the content display area TR1, the time when the message is transmitted to this talk room is displayed on the side of each text content.

In the content display area TR1, the minutes data summary content PRC1 is tapped by the user as an example, not limited to. Then, the minutes data summary content PRC1 is not limited, but is extended downward as an example, and the contents of the minutes data are displayed according to the display mode of the minutes content confirmation display area WR4. This configuration is to prevent the talk room display from being filled with minutes data and reducing the readability of the talk room.

FIG. 2-14 shows User B. This is an example of a display screen after the minutes data summary content PRC1 is tapped by B.
In FIG. 2-14, in the content display area TR1, the minutes data display area PRW1 extending downward from the position of the minutes data summary content PRC1 is displayed between the content display area TR1 and the content input area as an example. There is.

Unlike this, when the minutes data is transmitted to the talk room, the minutes data display area PRW1 may be displayed from the beginning instead of the minutes data summary content PRC1 or not. You may.

At the top of the minutes data display area PRW1, the minutes title similar to the minutes data summary content PRC1 and the date and time when the minutes data was last modified are displayed. On the upper right side of the minutes data display area PRW1, an editing function button BT1 for editing the minutes data is displayed.

As an example, not a limitation, when the editing function button BT1 is tapped by the user, the editing operation as shown in FIG. 2-11 becomes possible. When the edit function button BT1 is not tapped, the content edit selection menu is not displayed even if the text content is tapped, not as a limitation. This prevents the user from accidentally editing the minutes data while browsing.

At the lower right of the minutes data display area PRW1, the minutes data display area PRW1 is contracted (the display is folded), and the minutes data summary content PRC1 is displayed on the fold function button BT2 and the minutes data display area PRW1. The filter function button BT3 for narrowing down the displayed contents by the speaker or the keyword is displayed as an example, not limited.
The folding function button BT2 and the filter function button BT3 are not limited but are displayed in a floating manner as an example, and the display position is constant regardless of the scroll operation with respect to the minutes data display area PRW1.

In the minutes data display area PRW1 of FIG. 2-14, the contents confirmed / corrected in the minutes content confirmation display area WR4 are displayed according to the display terminal 20 or the user of the display terminal 20.
As an example, not a limitation, the text content CT1 may be described by User A.M. Since the content is based on the utterance content of A, the user B. In the terminal 20B of B, it is displayed by a balloon from the left side. The text content CT3 is the user B. Since the content is based on the utterance content of B, it is displayed in a balloon from the right side on the terminal 20B.

In the minutes data display area PRW1 of FIG. 2-14, as an example, not limited to, the deleted / edited operation is performed on the edited text content MCT2 in which the deleted / edited operation is performed before posting, although the original content cannot be confirmed. The text "The statement has been deleted" is displayed to indicate that.

Further, in the minutes data display area PRW1 of FIG. 2-14, the edited text content MCT3 indicating that the editing operation was performed after posting to the talk room for the text content CT3 is displayed. In the edited text content MCT3, as an example, not limited, the correction (editing) from the character "Kyoto" to the character "Kobe" is displayed as the correction history. Further, the edit history content MRF1 which is an editor's comment on the edited text content MCT3 is inserted and displayed under the edited text content MCT3.

The edit history content MRF1 is displayed in association with the icon of the user who performed the editing (in this case, the user EE) as an example, not limited to the editing history content MRF1. In addition, the edit history content MRF1 includes the user E.I. The characters "wrong location" entered by E and user E.I. The URI about the Kobe workshop entered by E is displayed. Under the URI, thumbnails related to the link destination of the URI are automatically generated and displayed as an example, not a limitation. As an example, but not a limitation, when the thumbnail is tapped by the user, the terminal 20 launches a web browser to display this URI information.

It should be noted that the editing operation performed before posting may or may not be displayed by leaving a history so that the editor and the edited content can be understood, as an example, not limited. In addition, it is not necessary to display the editor or its comment for the editing operation performed after posting. Editing operations may or may not be displayed in the form of additions or annotations.

As an example, not limited to the minutes data display area PRW1, when the user drags up and down, the contents in the minutes data display area PRW1 are scrolled up and down and displayed.
Here, the drag operation (drag) is not limited, but as an example, the user touches the display unit 24 (touch screen) in which the touch panel is integrally configured with a finger or a pen tip, and the finger or the pen is used as it is. It is an action to move the tip.

On the right side of the minutes data display area PRW1, a navigation bar indicating the position area of the content displayed in the minutes data display area PRW1 among the minutes data is displayed. As an example, not a limitation, when the minutes data display area PRW1 is scrolled down, the navigation bar also moves downward.

FIG. 2-15 is an example of a display screen when the filter function button BT3 is tapped by the user in FIG. 2-14.
In FIG. 2-15, at the lower part of the minutes data display area PRW1, the filter function button BT3 is not limited but as an example, and the filter function button BT3 is on the left side as an example when the user taps the filter function button BT3. The filter selection area FLW1 is displayed.

In the filter selection area FLW1, as an example, not limited to, user icons for narrowing down the speakers (users) to be displayed in the minutes data display area PRW1 are displayed side by side. When these user icons are tapped, the check mark on the upper right of the icon is highlighted, and it is selected whether or not to display the remarks (posted contents) for each user in the minutes data display area PRW1.

As an example, but not a limitation, in FIG. 2-15, user C.I. C and user E. The check mark of the icon with E is highlighted, indicating that the display target has been narrowed down.
At this time, the minutes data display area PRW1 is not limited to the user C.I. C and user E. Only the content related to the statement with E is displayed. As an example, not a limitation, the user B. The text content "Workshop is difficult because it is three months later" and the text content CT4, which expresses B's remark, are not displayed in the minutes data display area PRW1 in FIG. 2-15, and the text content CT5 closes the gap. Is displayed.

Titles and comments without a speaker may or may not be displayed regardless of the filter content.
Further, the filter target in the filter selection area is not limited to the speaker. As an example, not a limitation, words that frequently appear in the minutes data may or may not be automatically extracted, or text content containing a keyword specified by the user may or may not be extracted as a filter target. ..
Further, as a filter target, a specific section (paragraph) in the minutes data associated with the tag may or may not be extracted as an example rather than a limitation.

In FIG. 2-15, the user C.I. Below the text content CT6 based on C's utterance "Please see page 5 of the agenda", the link destination confirmation area LK1 is displayed as an example, not a limitation. The link destination confirmation area LK1 is not limited but is based on the content of the utterance "Agenda" in the text content CT6, and is not limited but based on the calendar information management database 159A. The agenda data stored in the related URI of the attached calendar information management data is read out. Then, a link destination (URI) corresponding to page 5 of the agenda data is automatically generated and displayed based on the utterance content of "page 5". As an example, but not a limitation, when this link is tapped by the user, the terminal 20 launches a web browser to display this URI information.

The agenda data may or may not be data manually or voice-input by the user.

In the edit history content MRF1, the URI is input by the user and the thumbnail is displayed, but in the link destination confirmation area LK1, the URI is automatically generated based on the utterance content.

<Processing>
FIG. 2-16 is a flowchart showing an example of the flow of processing executed by each device in this embodiment.
From left to right, examples of processing executed by the control unit 21 of the terminal 20A and processing executed by the control unit 11 of the server 10 are shown.

First, the control unit 21 of the terminal 20A uses the voice data used to create the minutes data from the voice data stored in the voice database 285 as an example, not limited, based on the user operation. (A110).

Then, the control unit 21 of the terminal 20A transmits the selected voice data to the server 10 by the communication I / F 22 (A130).

When the control unit 11 of the server 10 receives the voice data by the communication I / F 14, the control unit 11 executes the minutes data conversion process of converting the voice data into the minutes data which is the character (text) data (S120). The minutes data conversion process can be realized by the same process as the step A120 in FIG. 1-2.

In the minutes data conversion process, in addition to the voice recognition process and the speaker recognition process, a content addition process for adding content according to the utterance content converted into characters may or may not be executed. You may.

The content addition process is not limited, but as an example, a mention function addition process for uttering a name or keyword, an image addition process for adding an image related to the utterance content, and a predetermined word (as an example, not a limitation, "Agenda"". ) Can be included, such as a URI addition process for adding a URI (not a limitation, for example, a related URI of the calendar information management database 159A) related to the predetermined word.

Then, the control unit 11 of the server 10 executes the minutes posting candidate selection process for selecting the posting destination candidate of the minutes based on the generated minutes data (S130).

In the minutes posting candidate selection process, the control unit 11 of the server 10 identifies, as an example, not a limitation, a group of utterance users who are speaking in the minutes data based on the speaker recognition processing result. Then, among the group management data stored in the group management database 157, a group having a high correlation between the utterance user group and the member registered in the group member data is selected as a candidate for posting minutes, as an example, not limited. .. Alternatively, if there are two speaking users, the talk room including that user is selected as a candidate for posting minutes.
The minutes submission candidate may be one at the top, or a plurality of candidates may be selected from the top.

Further, the method of selecting the minutes submission candidate is not limited to using the utterance user group obtained from the voice data. As an example, not limited to, the user of the terminal 20 existing in the reception range of the beacon signal transmitted from the electronic device such as the beacon device (proximity beacon device) installed at the venue of the conference (meeting) may be the utterance user group. And you don't have to.

Further, in this case, as an example, not limited to the case where the beacon signal is transmitted from the terminal 20B and the terminal 20A exists within the reception range of the beacon signal, the user B. B may or may not be one user in the speaking user group. That is, as an example, not limited to each other, the users of the terminals 20 who mutually transmit the beacon signals and exist in the reception range of the beacon signals may or may not be the uttering user group. More specifically, in the position calculation information detection unit 29B of the terminal 20A, a UWB positioning unit may be used to grasp a group of terminals existing nearby, and a user of the terminal group may be a speaking user group.

In addition to this, as a method for identifying the utterance user group, as an example, the imaging unit 27 of the terminal 20A acquires a face image including each user, and recognizes the user group reflected in the face image as a person. It may or may not be specified by doing so.

Further, as an example, not limited to, the date and time when the meeting (meeting) was held is acquired from the time stamp of the voice data, and based on the calendar information management database 159A, the user A. Of the calendar information of A, the speaking user group may or may not be specified from the invitee data of the calendar information management data that matches the date and time when the meeting was held.

The utterance user group may or may not be estimated by combining the above-mentioned methods for specifying the utterance user group.

After that, the control unit 11 of the server 10 transmits the obtained minutes posting candidate to the terminal 20A by the communication I / F14.

It should be noted that the minutes submission candidate selection process may be executed by the terminal 20A instead of the server 10.

When the control unit 21 of the terminal 20A receives the minutes posting candidate from the server 10 by the communication I / F 22, the minutes data is added (input) from the minutes posting candidate based on the selection by the user operation as an example, not limited. ) Select the minutes posting destination, which is the chat room for. Then, the control unit 21 of the terminal 20A transmits the minutes posting destination to the server 10 by the communication I / F 22 (A150).

Chat rooms other than the minutes posting candidates may or may not be selected as the minutes posting destination.
If there is only one candidate for posting minutes, this step may or may not be omitted by automatically setting the candidate for posting minutes as the destination for posting minutes.

The control unit 11 of the server 10 receives the minutes posting destination by the communication I / F14. Then, the control unit 11 of the server 10 transmits the minutes data generated in the step of S120 to the terminal 20A by the communication I / F14 as the minutes draft data indicating the draft of the minutes (S140).

When the control unit 21 of the terminal 20A receives the minutes draft data from the server 10 by the communication I / F 22, the control unit 21 displays the minutes draft data on the display unit 24. Then, the control unit 21 of the terminal 20A accepts an operation of modifying the minutes draft data based on the operation input to the input / output unit 23 of the user (A170).

The process of modifying the draft minutes data can include, as an example, not limited to, editing operations such as "delete", "add line spacing", and "comment" for the content edit selection menu described above.

Then, the control unit 21 of the terminal 20A transmits the minutes draft data correction information, which is the content of the editing operation, to the server 10 by the communication I / F 22.

When the control unit 11 of the server 10 receives the minutes draft data correction information by the communication I / F 14, the control unit 11 executes the minutes draft data correction process for updating the minutes data based on the minutes draft data correction information (S150). ..

Then, the control unit 11 of the server 10 adds (inputs) the modified minutes data to the chat room designated as the minutes posting destination (S160).

After that, the control unit 11 of the server 10 transmits the minutes data added (input) to the chat room to the terminal 20 of the user belonging to the chat room including the terminal 20A by the communication I / F 14 (S170). Then, the control unit 11 of the server 10 ends the process.

When the minutes data is received by the communication I / F 22, the control unit 21 of the terminal 20A causes the display unit 24 to display the minutes data (A180). Then, the control unit 21 of the terminal 20A ends the process.

If the minutes data is added to the chat room and then corrected, the minutes data is regarded as the minutes draft data on the terminal 20 to which the minutes data is transmitted, and the minutes data are regarded as the minutes draft data, A170 to A180 and S150 to S170. Just follow the steps with.

Further, instead of executing the step S160 on the server 10, the control unit 21 of the terminal 20 may add (input) the minutes data to the chat room after receiving the minutes data, or do so. It does not have to be.

<Effect of the second embodiment>
In the second embodiment, the talk room (not limited, but an example of a chat room) is the user A. of the terminal 20A. The configuration selected from the minutes submission candidates based on the input by A (not limited, but an example of the terminal user) is shown.
As an example of the effect of the embodiment obtained by such a configuration, the chat room can be selected from the minutes submission candidates, so that the user can select the chat room more appropriately and easily.

In addition, the second embodiment shows a configuration in which the talk room is selected based on the information acquired by the terminal 20A (not limited, but an example of the first information).
As an example of the effect of the embodiment obtained by such a configuration, a candidate chat room can be selected based on the first information. Therefore, the selection of the chat room can be performed more appropriately.

In this case, the first information may include voice data (an example of sound information, not a limitation) acquired by the sound input unit 25 (not a limitation, an example of the first acquisition unit) of the terminal 20. can.
As an example of the effect of the embodiment obtained by such a configuration, the sound information which is the source of the character information can be used as a chat room selection criterion, and a more appropriate chat room can be selected.

Further, in this case, the first information may include information transmitted from the electronic device.
As an example of the effect of the embodiment obtained by such a configuration, even when it is difficult to identify the speaking user from the sound information, it is possible to appropriately select the input destination chat room.

Further, in this case, the electronic device can be a beacon device that transmits a beacon signal.
As an example of the effect of the embodiment obtained by such a configuration, a chat room can be easily and appropriately selected based on the information transmitted from the electronic device installed in advance.

Further, in this case, the electronic device may be a terminal 20 other than the terminal 20A, and not a limitation but a terminal 20B (not a limitation but an example of the first terminal).
As an example of the effect of the embodiment obtained by such a configuration, it is possible to acquire the first information without installing an electronic device in advance. As a result, user convenience is improved.

Further, in this case, the first information may include image information (not limited, but as an example, a face image) of the captured image captured by the imaging unit 27 of the terminal 20.
As an example of the effect of the embodiment obtained by such a configuration, even when it is difficult to identify the uttering user from the sound information, the chat room of the input destination can be selected by using the image information.

In addition, the first information is the user A. The calendar information of A (not limited, but an example of the schedule information of the user of the terminal) can be included.
As an example of the effect of the embodiment obtained by such a configuration, it is possible to select an appropriate chat room as a posting destination by using the schedule information, and even when it is difficult to identify the speaking user from the sound information, it is more accurate. You can select the chat room to enter in.

Further, in the second embodiment, the first character information (limited) based on the first sound information acquired by the sound input unit 25 (not limited, but as an example, the utterance "Then, the meeting in February 2020 will be started"). Instead, as an example, the text content CT1) is placed in the selected chat room (not limited, but as an example, the talk room of the group "Project X"). Second character information (as an example, not as a limitation) based on the second sound information (not limited, but as an example, the utterance "When was the product released?") That was input in association with A and acquired by the sound input unit 25. , Text content CT3) is in the chat room. A configuration is shown in which a chat room, which is input in association with B and includes first character information and second character information, is displayed on a display unit (not limited to the display unit 24 as an example).
As an example of the effect of the embodiment obtained by such a configuration, the first character information based on the first sound information and the second character information based on the second sound information are associated with different users. Can be easily identified by the user. As a result, user convenience is improved.

Further, in the second embodiment, the voice data (sound information) is acquired by the sound input unit 25 before the chat room is selected, and the minutes data (text information) is that the chat room is selected. Based on this, the configuration to be entered in the chat room is shown.
As an example of the effect of the embodiment obtained by such a configuration, character information based on the sound information before selecting the chat room can be input to the chat room later. That is, since it is not necessary to select a chat room when acquiring sound information, it is possible to quickly acquire sound information without complicated operations. Therefore, the user can easily determine to acquire the sound information without being aware of the existence of the chat room, and the convenience of the user can be improved.

Further, in the second embodiment, the minutes data (character information) is obtained by the user A. A (not limited, but an example of a terminal user) and user B. B (not limited, but an example of a user of the first terminal) and user C.I. It is input to the first chat room including C (an example of a user of the second terminal, not the limitation) (as an example, not the limitation, the group "Talk Room of Project X"), and at least the terminal 20C (not the limitation, the first, not the limitation). The configuration to be transmitted to (an example of two terminals) is shown.
As an example of the effect obtained by such a configuration, character information is input to a chat room including a terminal user, a first terminal user, and a second terminal user, and then at least in the second terminal. By being transmitted, it can be made available to at least the user of the second terminal.

<Second modification (1)>
In the second embodiment, the minutes data conversion process is executed on the server 10 based on the voice data to create the minutes data, but the present invention is not limited to this. As an example, not limited to the above, the process may or may not be realized by the method (2) above.

In this case, as an example, not limited to, the control unit 21 of the terminal 20 executes the minutes data conversion process when the step A110 of FIG. 2-16 is executed. Since each process can be executed in the same manner as the step of S120 of FIG. 2-16, the description thereof will be omitted in detail. Then, the control unit 21 of the terminal 20 transmits the minutes data generated by the minutes data conversion process to the server 10 by the communication I / F 22.
When the control unit 11 of the server 10 receives the minutes data by the communication I / F 14, the control unit 11 of the server 10 executes the steps of S130 and the following in FIG. 2-16.

As an example of the effect of the modified example obtained by such a configuration, when the data capacity of the sound information is large, the terminal side converts the sound information into character information whose amount of information can be more easily compressed and then transmits the sound information to the server. This makes it possible to reduce the amount of communication.

The step S130 in FIG. 2-16 may or may not be performed at the terminal 20. In this case, it can be said that the first information is generated by the terminal.

Further, the minutes data conversion process may be executed by the terminal 20, and the minutes data may or may not be added to the chat room by the server 10.

<Third Example>
The third embodiment is an example according to the method (1) regarding the timing of (B) above.
In the third embodiment, in the minutes service, when the control unit 21 of the terminal 20 acquires a part of the voice data in the meeting or the meeting by the sound input unit 25, it transmits it to the server 10. When the control unit 11 of the server 10 creates partial minutes data based on the received voice data, the control unit 11 transmits the partial minutes data to the terminal 20. The above process is repeated until the end of the meeting (meeting).
Hereinafter, the method according to the third embodiment will be referred to as "real-time mode".
The contents described in the third embodiment can be applied to any of the other embodiments and the other modifications.

<Display screen>
In the following, as a chat room for adding (inputting) minutes data, an example of a display screen when a talk room of the group "Project X" is selected based on user input is shown as an example, not limited.

In FIG. 3-1 in FIG. 2-6, "friend" is tapped (selected) by the user, and then the group "project X" is tapped (selected) from the user account or group registered as "friend". This is an example of the display screen of the talk room of the group "Project X" in the case where.

In FIG. 3-1 in the current position display area, characters indicating that the current position is the talk room of the group “project X” are displayed. Further, in the content display area TR1, the user A. The text content of the characters "It's time to start the meeting" entered (sent) by A and the user D. The text content of the characters "I will be a little late due to the train delay" entered (transmitted) by D is displayed.

In the content input area, as an example, not limited to, when the user taps the talk room function call icon indicated by the character "+", the talk room function selection area WR5 that calls various functions of the messaging application related to this talk room. Is displayed.
In the talk room function selection area WR5, a plurality of function icons corresponding to various functions of the messaging application related to the talk room are displayed. Function icons include, but are not limited to, icons corresponding to each of a plurality of functions, such as "file", "contact", and "location information".

In this example, among the plurality of function icons, the state in which the function icon of "minutes" is tapped is shown. Then, the minutes data is added to the talk room in real time based on the voice data acquired by the sound input unit 25. That is, the timing at which the function icon of "minutes" is tapped is before the start of the meeting or meeting.

When the "minutes" function icon is tapped, only a specific user among the members belonging to this talk room will make a statement based on the user operation on the user selection screen (not shown), and thereafter, the minutes data. It may or may not be recorded as.

FIG. 3-2 is an example of a talk room screen when the function icon of “minutes” is tapped in FIG. 3-1.
In FIG. 3-2, the voice data acquisition status confirmation indicating that the voice data is being acquired by the sound input unit 25 as an example, not limited, based on the minutes function of the messaging application on the upper part of the content display area TR1. The area STW1 is displayed.

In the voice data acquisition status confirmation area STW1, as an example, the characters "minutes are being recorded" indicating that the terminal 20A is acquiring voice data for generating the minutes data are displayed. Has been done. On the right side, the elapsed time (“00:03” in this example) since the start of acquiring the voice data is shown.

On the right side of the elapsed time, the recording end button BT4 for stopping the acquisition of the audio data and ending the addition of the minutes data to the talk room and the image pickup unit 27 are used to acquire the image, and this talk room is used. The image pickup button BT5 for adding to the above and the above-mentioned editing function button BT1 are displayed side by side.

Further, in the lower right portion of the content display area TR1, the filter function button BT3, the content emphasis function button BT6 indicated by the balloon icon, and the tag function button BT7 indicated by the sharp character icon are displayed vertically side by side. There is.
The filter function button BT3, the content emphasis function button BT6, and the tag function button BT7 are not limited but are displayed in a floating manner as an example, and the display position is constant regardless of the scroll operation with respect to the content display area TR1.

As an example, not a limitation, when the tag function button BT7 is tapped by the user, the tag function button BT7 is extended to the left to reflect the tag input area and the input tag as shown in FIG. 3-2. A tag reflection function button indicated by the characters "setting" is displayed. As an example, not a limitation, in the example of FIG. 3-2, the characters "Agenda 1" are input in the tag input area.

When the tag reflection function button is tapped by the user, the tag of "Agenda 1" is added to the minutes data converted from the audio data after that, as an example, not limited.

Based on this tag, as an example, not a limitation, it is possible to quickly extract and display a part of the minutes data to which an arbitrary tag is added by the filter function button BT3.

In this example, the tag may or may not be automatically added according to the content of the utterance such as "From now on Agenda 1" by the user, as an example rather than a limitation.

Further, in the content display area TR1, the text content CT1 is generated from the partial voice data separated by the utterance delimiter as an example, not the limitation, and is input to the talk room in real time and displayed. .. In this embodiment, the time when the content is generated and added to the talk room is displayed next to the content (message), not as a relative time from the start of acquiring the audio data, but as an example rather than as a limitation. Has been done.
As an example, not a limitation, the text content CT1 indicates that it was added at the time of "13:42".

It should be noted that the relative time from the start of acquiring the voice data may or may not be displayed instead of the time.

FIG. 3-3 is an example of a display screen in the case where the acquisition of voice data continues after the tag reflection function button is tapped in FIG. 3-2.

In FIG. 3-3, as the tag is input, the tag content TG1 in which the characters “About Agenda 1” indicating the content of the input tag is displayed under the text content CT1 is displayed.
Since the content of the tag content TG1 is not the utterance content in the voice data, it is not limited but is displayed in the content display area TR1 centered as an example.

The color of the band painted on both ends of the tag content TG1 is not limited, but as an example, corresponds to the color of the band displayed in association with the left end of the content display area TR1 for the content added later. ing.

As an example, not a limitation, a band associated with the tag "Agenda 1" is displayed on the left side of the text content CT3.

In addition, user E. Under the text content based on E's utterance "Can't you make it in time for the workshop in Kyoto?" The image content PC1 for display is automatically added / attached (input). In FIG. 3-3, an image of a temple is added and displayed on the image content PC 1 as an example of the result of an image search based on the place name "Kyoto" of the utterance content, not as a limitation.

The images to be added / attached are not limited to images based on place names.
In addition to this, as an example, not limited to, the user A.A. An image such as an employee ID image of A may or may not be added. In addition, any image may or may not be added as long as it is related to the content of the conversation.

FIG. 3-4 is an example of a display screen in the case where the acquisition of voice data is continued following FIG. 3-3.

In FIG. 3-4, as an example, not a limitation, when the text content CT5 is added to the talk room, the user D.C. D sends a message MSG1 to the talk room with the words "I'm moving, so I'll check the contents later." Then, as an example, not limited to, a message is sent to the display unit 24 of the terminal 20A by the user D. The message incoming notification area NTW1 indicating that the message has been received from D is displayed.

Since the message MSG1 is not the minutes data based on the voice data, the band associated with the tag is not displayed on the left side of the message MSG1 in the content display area TR1 as an example without limitation.

In addition, even in the message transmitted (input) by the user, the tag may be linked and the band may be displayed on the left side thereof, or it may not be so.

Further, in the content display area TR1, the link destination confirmation area LK2 is displayed under the text content CT6 as an example, not a limitation. Since the content displayed at the bottom of the link destination confirmation area LK2 is the same as that of the link destination confirmation area LK1, the description thereof will be omitted. At the upper part of the link destination confirmation area LK2, as an example, not limited to, thumbnails related to the link destination of the URI are automatically generated and displayed.
This makes it easy to refer to what you think you will need during the meeting without having to manually follow the link.

FIG. 3-5 is an example of a display screen when the acquisition of audio data continues after a lapse of time from FIG. 3-4.

As an example, but not limited to, when the image pickup button BT5 is tapped by the user, the control unit 21 of the terminal 20A executes the photographing function of the messaging application, and the image pickup unit 27 acquires an image. The captured image is embedded in the minutes data at the time of imaging and displayed in the content display area TR1.

In FIG. 3-5, as an example, not limited to, the image of the whiteboard acquired by tapping the image pickup button BT5 is embedded in the minutes data and displayed on the image content PC2.

In this case, as an example, the image content PC 2 may or may not be displayed by trimming according to the size of the image content PC 2 after imaging.

Further, when the image pickup button BT5 is tapped, the image pickup unit 27 may or may not acquire a moving image. In this case, the moving image is embedded in the image content PC2, and the moving image is displayed in time series while the image content PC2 is displayed in the content display area TR1 as an example, not limited to the image content PC2. Then, as an example, not limited to, when the image content PC2 is no longer displayed by scrolling and is displayed in the content display area TR1 by scrolling again, the moving image is reproduced from the head on the image content PC2.

Further, the timing at which the image content PC2 is displayed does not depend on the tapping of the image pickup button BT5. In addition to this, when the moving image is continuously acquired by the imaging unit 27 and the difference between frames is large to some extent (when the difference between frames becomes equal to or greater than the threshold value or exceeds the threshold value), the frame is not limited. The image may or may not be automatically displayed as the image content PC2.

FIG. 3-6 is an example of a display screen in the case where the acquisition of voice data is continued following FIG. 3-5.

As an example, not a limitation, User E.I. The text content CT7 based on E's utterance "Because time is pushing, let's move on to the next agenda" is displayed under the image content PC2.

At this time, the user E.I. When the tag function button BT7 is tapped by the user of the terminal 20A according to the utterance content of E, the tag function button BT7 is extended to the left, and the tag input area and the tag reflection function button are displayed. As an example, not a limitation, in FIG. 3-6, the characters "Agenda 2" are input in the tag input area.

When the tag reflection function button is tapped by the user, the tag of "Agenda 2" is added to the minutes data converted from the subsequent audio data as an example, not a limitation.

FIG. 3-7 is an example of a display screen in the case where the acquisition of voice data continues after the tag reflection function button is tapped in FIG. 3-6.

In FIG. 3-7, as the tag is input, the tag content TG2 in which the characters “About Agenda 2” indicating the content of the input tag is displayed under the text content CT7 is displayed. The bands painted on both ends of the tag content TG2 are the same as those of the tag content TG1, and thus the description thereof will be omitted.

As an example, not a limitation, User A. On the left side of the text content CT8 based on A's utterance, "Then, about the contents of Agenda 2," a band associated with the tag "Agenda 2" is displayed.
That is, by paying attention to the color of the band, the user can easily determine which part should be referred to when looking back at the minutes data later.

FIG. 3-8 is an example of a display screen when the acquisition of audio data is continued after a further time is taken from FIG. 3-7.

In FIG. 3-8, the content emphasis selection area MBT2 is displayed as the content emphasis function button BT6 is tapped by the user, not as a limitation but as an example.

In the content emphasis selection area MBT2, cooperative function selection buttons for changing the display mode of the text content are arranged based on the user operation. By tapping the text content after tapping these buttons, the display mode of the text content can be meaningfully changed.

As an example, not a limitation, the edited text content MCT4 is applied to the text content by tapping the selection button indicating the "point" of the minutes. In the edited text content MCT4, the shape of the balloon changes to a quadrangle with a reduced diagonal, and a finger mark is further added.
As a result, the user can easily recognize that this content is treated as a point of the minutes.

Similarly, as an example, but not a limitation, the content related to "approval" of the minutes will be the hexagonal balloon shown in the edited text contents MCT5 to MCT7.

In the content highlighting selection area MBT2, the selection button indicating "return" may be tapped, and the highlighted content may be tapped continuously to restore the original balloon-type text content. It does not have to be.
As a result, even if the content is mistakenly emphasized as an example rather than a limitation, it is possible to return to the original display mode.

In FIG. 3-8, in the content emphasis selection area MBT2, the user taps the selection button indicating the “conclusion” of the minutes. Then, user A. Continue tapping the text content CT9 based on A's utterance "The release will be postponed for about a year."

FIG. 3-9 is an example of the screen after the text content CT9 is tapped in FIG. 3-8.

In FIG. 3-9, the edited text content MCT8 emphasized by the balloon of the double line meaning “conclusion” is displayed in the content display area TR1 with respect to the text content CT9.

It should be noted that the emphasis function for changing the display mode of the text content may be automatically applied from the sentence analysis (semantic analysis) result of the minutes data as an example, and is not limited to that. It does not have to be.

In addition to "points" and "conclusions", as content to be collaborated, "homework" or the like, which means a task for each user, may or may not be provided as an example rather than a limitation. .. As an example, rather than a limitation, if the content is emphasized as "homework", the content may or may not be automatically added to the user's TODO list.

In FIG. 3-9, under the edited text content MCT8, the user A. The text content CT10 based on A saying "This is the end of the meeting" is displayed.

As an example, not a limitation, the user of the terminal 20A taps the recording end button BT4 for stopping the acquisition of the voice data and the addition of the minutes data based on the utterance of the text content CT10. Then, the minutes end display message EXM1 with the characters "minutes completed" is added to the talk room. This message informs the members of this talk room that the addition of minutes data has been completed.

When the acquisition of audio data and the addition of minutes data are stopped, the function buttons related to editing and displaying the minutes data, such as the edit function button BT1 and the filter function button BT3, are not limited to the content display area TR1. Will not be displayed in.

As an example, not limited to this, when the user scrolls up on the content display area TR1 and the contents of the minutes data are displayed, the edit function button BT1 and the filter function button BT3 are re-executed in the content display area TR1. By displaying the minutes data, the editing operation may or may not be executed at any time after the minutes data is added.

Further, when the content of the minutes data is displayed, the folding function button BT2 may be displayed in the content display area TR1 so that the minutes data portion can be folded, or it is not necessary to do so.

<Processing>
FIG. 3-10 is a flowchart showing an example of the flow of processing executed by each device in this embodiment.
From left to right, examples of processing executed by the control unit 21 of the terminal 20A and processing executed by the control unit 11 of the server 10 are shown.

This flowchart is a flowchart in which the flowchart of FIG. 2-16 is rewritten into the processing for the real-time mode in this embodiment.

First, the control unit 21 of the terminal 20A selects the minutes posting destination and transmits the minutes to the server 10 by the communication I / F 22 (A150).

In addition, before the step of A150, as an example, not limited to, the minutes posting candidate selection process may be performed on the server 10, and the terminal 20A may or may not receive the minutes posting candidate. .. Alternatively, the minutes submission candidate selection process may or may not be executed on the terminal 20A. In these cases, a method other than the method based on audio data is used in the minutes submission candidate selection process.

Next, the control unit 21 of the terminal 20A determines whether or not to end the acquisition of voice data based on the user input to the input / output unit 23 (A190).

When the user chooses to end the acquisition of the audio data (A190: YES), the control unit 21 of the terminal 20A provides the minutes termination information for terminating the acquisition of the audio data and the addition of the minutes data. , Transmit to the server 10 by communication I / F22. Then, the control unit 21 of the terminal 20A ends the process.

When the user does not choose to end the acquisition of voice data (A190: NO), the control unit 21 of the terminal 20A accepts the input to the sound input unit 25, and the user's dialogue content is used as an example, not a limitation. Acquire the audio data including. Then, as an example, the control unit 21 of the terminal 20A stores the voice data as partial voice data in the storage unit 28 when a silence state exceeding a predetermined time occurs in the voice data (as a delimiter). A210). In this case, the partial audio data is a section of audio data separated by a silent section.

After that, the control unit 21 of the terminal 20A transmits the acquired partial voice data to the server 10 by the communication I / F 22 (A220).

The control unit 11 of the server 10 receives the minutes posting destination from the terminal 20A by the communication I / F14. After that, the control unit 11 of the server 10 determines whether or not the minutes termination information has been received from the terminal 20A by the communication I / F 14 (S180).

When the minutes termination information is received from the terminal 20A (S180: YES), the control unit 11 of the server 10 ends the process.

When the minutes termination information is not received from the terminal 20A (S180: NO), the control unit 11 of the server 10 receives the partial voice data from the terminal 20A by the communication I / F14.
Then, the control unit 11 of the server 10 executes the minutes data conversion process on the received partial voice data, and creates (converts) the partial minutes data (S190).

Since the minutes data conversion process can be executed in the same manner as the step of S120 of FIG. 2-16 as an example without limitation, detailed description thereof will be omitted.

When the partial minutes data is created, the control unit 11 of the server 10 adds (inputs) the partial minutes data to the chat room designated as the minutes posting destination (S200).

After that, the control unit 11 of the server 10 transmits the partial minutes data added (input) to the chat room to the terminals of the users belonging to the chat room including the terminal 20A by the communication I / F 14 (S210). Then, the control unit 11 of the server 10 returns the process to the step of S180.

When the partial minutes data is received by the communication I / F 22, the control unit 21 of the terminal 20A adds the partial minutes data to the display unit 24 and displays it (A230). Then, the control unit 21 of the terminal 20A returns the process to the step of A190.

The control unit 21 of the terminal 20 may or may not add the partial minutes data to the chat room after receiving the partial minutes data without executing the step of S200.

<Effect of the third embodiment>
In the third embodiment, the terminal 20 uses the sound input unit 25 (an example of an acquisition unit, not a limitation) to provide the first sound information (as an example, not a limitation) of the user A.A. Based on the acquisition of the partial voice data including the utterance "I will start"), the first character information (not limited, but as an example, "Let's start the meeting in February 2020" linked to user AA. Partial minutes data including the characters "masu" is input to the chat room, and the chat room containing the first character information (text content CT1 as an example, not limited) is displayed on the display unit 24. Then, the terminal 20 has acquired the second sound information (not limited, but as an example, partial voice data including the utterance "When was the product released?" By the user BB) by the sound input unit 25. Based on this, the second character information (partial minutes data including the characters "When was the product released?" Associated with user BB as an example, not limited) is input to the chat room, and the second character information is input. A configuration is shown in which a chat room containing two-character information (not limited, but as an example, text content CT3) is displayed on the display unit 24.
As an example of the effect of the embodiment obtained by such a configuration, when the first sound information is acquired by the acquisition unit, the first character information is obtained, and when the second sound information is acquired, the second character information is obtained from the terminal. Since the characters are sequentially displayed on the display unit, the time from the acquisition of the sound information by the acquisition unit to the display of the character information based on the sound information can be shortened. As a result, the user can refer to the necessary information in real time, and the convenience of the user can be improved.

Further, in the third embodiment, the terminal 20 includes image information (not limited, but not limited) including at least a part of the captured image (for example, an image acquired by tapping the image capturing button BT5). As an example, the image content PC2) is input to the chat room, and the chat room containing the image information (content display area TR1 as an example, not limited) is displayed on the display unit 24.
As an example of the effect of the embodiment obtained by such a configuration, even if it is difficult to express with text information, by using image information together, information can be shared more accurately in a chat room.

Further, in the third embodiment, the character information (text content CT3 as an example, not limited) is displayed in the first display area (content display area TR1 as an example, not limited) of the chat room, and the image information is displayed. The configuration shown is displayed in the second display area of the chat room (not limited, but as an example, below the text content of the content display area TR1).
As an example of the effect of the embodiment obtained by such a configuration, the image information and the character information can be displayed in the chat room so as not to interfere with each other.

<Third modification (1)>
In the third embodiment, the minutes data conversion process is executed on the server 10 based on the voice data to create the minutes data, but the present invention is not limited to this. As an example, not limited to the above, the process may or may not be realized by the method (2) above.

In this case, as an example but not a limitation, the control unit 21 of the terminal 20 executes the partial minutes data conversion process when the step A210 of FIG. 3-10 is executed. Since each process can be executed in the same manner as the step of S190 in FIG. 3-10, the details will be omitted. Then, the control unit 21 of the terminal 20 transmits the partial minutes data generated by the partial minutes data conversion process to the server 10 by the communication I / F 22.
When the control unit 11 of the server 10 receives the partial minutes data by the communication I / F 14, the control unit 11 of the server 10 executes the steps of S200 and the following in FIG. 3-10.

As an example of the effect of the modified example obtained by such a configuration, when the data capacity of the partial sound information is large, the terminal side converts the partial sound information into character information whose amount of information can be compressed more easily, and then transfers the information to the server. It becomes possible to transmit, and the amount of communication can be reduced.
Further, when determining the delimiter of the partial voice data, it is possible to determine by detecting the delimiter of the phrase from the character information as an example, not the limitation, and the partial voice data can be acquired more appropriately. As a result, the accuracy of the partial minutes data conversion process is improved.

The step S200 in FIG. 3-10 may or may not be performed at the terminal 20.

<Fourth Example>
The fourth embodiment is an example according to the method (1) relating to the timing of (C) above.
In the fourth embodiment, in the minutes service, when the chat room to which the minutes data is added is selected, the control unit 21 of the terminal 20 sequentially adds the minutes data (real-time input) to the chat room or voices. Display a screen for selecting whether to add data all at once after the meeting.
The contents described in the fourth embodiment can be applied to any of the other embodiments and the other modifications.

<Display screen>
FIG. 4-1 is an example of a talk room screen when the function icon of “minutes” is tapped by the user in FIG. 3-1.

In FIG. 4-1 the minutes mode selection area WR6 is superimposed and displayed on the content display area TR1.
In the minutes mode selection area WR6, as an example, not limited to, a real-time mode selection button identified by the characters "real-time addition" and a batch addition mode selection button identified by the characters "add later" are displayed. Has been done.

When the real-time mode selection button is tapped by the user, the format of adding minutes data sequentially during the meeting is selected according to the format of the real-time mode as an example, not limited.
Further, when the batch addition mode selection button is tapped by the user, a format for batch addition of minutes data after the end of the meeting is selected according to the format of the batch addition mode, not as a limitation but as an example.

FIG. 4-2 is an example of the talk room screen when the batch addition mode selection button is tapped in FIG. 4-1.

In FIG. 4-2, the audio data acquisition status confirmation area STW1 is displayed in the content display area TR1 as an example, not limited to the time, in accordance with the time when the batch addition mode selection button is tapped.
That is, the message MSG1 indicates that the message is input (transmitted) to the talk room after the start of voice data acquisition.

FIG. 4-3 is an example of a talk room screen when the recording end button BT4 is tapped by the user after the meeting is completed, and then the minutes data summary content PRC1 added to the talk room is tapped.

In FIG. 4-3, the minutes data display area PRW1 is displayed as the minutes data summary content PRC1 is tapped.
In the minutes data display area PRW1, as an example, the message MSG1 received during the acquisition of voice data is inserted and displayed immediately below the text content CT5. The display time of the message MSG1 is automatically replaced with the display of "00:03:30" indicating the relative time from the beginning of the voice data from the time shown in FIG. 4-2.

As shown in FIG. 4-4, the message MSG1 received during the acquisition of the voice data may or may not be displayed without being inserted into the minutes data display area PRW1.
As an example, not a limitation, in the display example of FIG. 4-4, the text content based on the acquisition result of the voice data is displayed immediately below the text content CT5 of the minutes data display area PRW1, and the message MSG1 displays the minutes data. It is displayed in the content display area TR1 outside the area PRW1 while maintaining the display time.

Further, when the real-time mode selection button is tapped in FIG. 4-1 as an example, the display screen is the same as that in FIGS. 3-2 and later.

<Processing>
FIG. 4-5 is a flowchart showing an example of the flow of processing executed by each device in this embodiment when the batch addition mode selection button is tapped.
From left to right, examples of processing executed by the control unit 21 of the terminal 20A and processing executed by the control unit 11 of the server 10 are shown.

In this flowchart, in the flowchart of FIG. 2-16, the step of A150 is moved forward as the process of the terminal 20A, the step of A110 is replaced with the step of A150 and A100, and the step of S130 is deleted as the process of the server 10. It is a flowchart of the above.

First, the control unit 21 of the terminal 20A selects the minutes posting destination and transmits the minutes to the server 10 (A150).

In addition, before the step of A150, as an example, not limited to, the minutes posting candidate selection process may be performed on the server 10, and the terminal 20A may or may not receive the minutes posting candidate. .. Alternatively, the minutes submission candidate selection process may or may not be executed on the terminal 20A. In these cases, a method other than the method based on audio data is used in the minutes submission candidate selection process.

Next, the control unit 21 of the terminal 20A acquires voice data by the sound input unit 25 as an example, not limited to (A100).
Then, the control unit 21 of the terminal 20A finishes the process after executing each step of A130 to A180.

The control unit 11 of the server 10 ends the process after executing each step of S120 to S170.

When the real-time mode selection button is tapped, the process can be performed in the same manner as in FIG. 3-10 as an example without limitation, and thus detailed description thereof will be omitted here.

<Effect of Fourth Example>
In the fourth embodiment, the talk room (an example of a chat room, not limited) is not limited, but as an example, a tap operation on the real-time mode selection button or the batch addition mode selection button (not limited, but input by the terminal user). The configuration selected based on (one example) is shown.
As an example of the effect of the embodiment obtained by such a configuration, character information can be input in a chat room designated by the user by a method intended by the user. Thereby, the convenience of the user can be improved.

<Fourth modification (1)>
In the fourth embodiment, the server 10 executes the minutes data (partial minutes data) conversion process based on the voice data to create the minutes data (partial minutes data), but the present invention is not limited to this. As an example, not limited to the above, the process may or may not be realized by the method (2) above.

In this case, when the real-time mode selection button is tapped, the third modification example (1) is performed, and when the batch addition mode selection button is tapped, the second modification example (1) is processed in the same manner. You just have to do.

<Fourth modification (2)>
In the fourth embodiment, the real-time mode and the batch addition mode are collectively applied to all user utterances, but are not limited thereto.

As an example, not a limitation, the real-time mode is applied to the utterances of a specific user (user CC as an example, not the limitation), and the batch addition mode is applied to the utterances of other users. It may or may not be.

In this case, during a meeting or meeting, User C.I. The content of C's utterance is converted into minutes data and shared in the talk room. Then, after the meeting or the meeting is over, the utterance contents of other users are converted into minutes data and added to the talk room. Therefore, as an example, not a limitation, the user C.I. When C is the key person of the meeting, the content of the key person's remarks is shared in the talk room during the meeting, so that the meeting can be facilitated. After the meeting is over, other users' remarks are also added to the talk room, making it easier to look back at the flow of the entire meeting.

The minutes data in the real-time mode and the minutes data in the batch addition mode may be merged into either one after the meeting is completed, or may be stored independently.

<Fourth modification (3)>
In the fourth embodiment, the real-time mode and the batch addition mode are collectively applied to all the times during the meeting, but are not limited thereto.

As an example, not limited, minutes data is created / added in real-time mode only during the time when important matters that are the points of the meeting are discussed, and minutes data is created / added in batch addition mode at other times. You may or may not do so.

In this case, since the content of the discussion about the important matter is shared in the talk room in real time, the user can look back on the spot and prevent misunderstanding about the important matter.

<Fifth Example>
In the above embodiment, the minutes data was created based on the voice data stored in the voice database 285. On the other hand, in this embodiment, the minutes data is created based on the audio data (audio track) included in the video data by using the video (moving image) data.
The contents described in the fifth embodiment can be applied to any of the other embodiments and the other modifications.

<Display screen example>
FIG. 5-1 is a screen view showing an example of the display mode of the minutes data display area PRW1 when the minutes data is created using the video data.
In FIG. 5-1 as an example, not a limitation, the user A. The display screen in the terminal 20A of A is shown.

FIG. 5-1 is a screen corresponding to the display screen of FIG. 2-14 as an example, not a limitation, and the user A. This is an example of a screen after the minutes data summary content PRC1 displayed on the display unit 24 of the terminal 20A by A is tapped.

In FIG. 5-1, a video data display area PP1 for displaying the video data used for creating the minutes data in a picture-in-picture format is displayed in the lower left portion of the minutes data display area PRW1. That is, the display position of the video data display area PP1 is fixed even if the minutes data display area PRW1 is scrolled.

The video data display area PP1 is not limited to, but as an example, an image of the whiteboard of the meeting at the time of utterance of the text content CT5 displayed at the lowermost end of the minutes data display area PRW1 of the video data is captured. It is displayed.

At the lower part of the video data display area PP1, as an example, not limited to, the elapsed time from the beginning of the video data in the video being reproduced in the video data display area PP1 is displayed. That is, in FIG. 5-1 it is shown that the video corresponding to the utterance time (time determined by the elapsed time from the start of the conference) of the text content CT5 is displayed in the video data display area PP1.

As an example, not limited to, when the user performs an up-scrolling operation on the minutes data display area PRW1, the content (text content or the like) displayed in the minutes data display area PRW1 returns to the start time direction of the meeting. At this time, the video displayed in the video data display area PP1 is also rewound and displayed according to the content displayed in the minutes data display area PRW1.
On the contrary, when the user performs a downward scroll operation on the minutes data display area PRW1, the content displayed in the minutes data display area PRW1 advances in the direction of the end time of the meeting. In this case, the video displayed in the video data display area PP1 is also fast-forwarded and displayed according to the content displayed in the minutes data display area PRW1.

It should be noted that the video may or may not be fast-forwarded or rewound based on the user's utterances such as "fast-forward" and "rewind" instead of the scrolling operation by the user. Further, when the tap operation is performed on the minutes data display area PRW1, the video may or may not be paused and displayed.

That is, the video displayed in the video data display area PP1 is a video synchronized on the time axis with the content displayed in the minutes data display area PRW1.

When the minutes data is created using the video data, it is not essential that the video data display area PP1 is displayed in the minutes data display area PRW1. As an example, not the limitation, the video data display area PP1 may or may not be overlaid on the minutes data display area PRW1 based on the user's selection operation.

Further, as an example, not limited to, in FIG. 2-7, a line indicating a plurality of audio data and video data in the audio data list information display area WR1 is selected (tapped) by the user, followed by audio data (not shown). By tapping the selection decision button, one minute data may or may not be created from a plurality of audio data and video data.

<Processing>
The processing in this embodiment is not limited, but as an example, it can be realized in the same manner as in FIG. 2-16 by regarding the audio track accompanying the video data as audio data. Therefore, detailed description thereof will be omitted here.

In this embodiment, the video (moving image) displayed in the video data display area PP1 is synchronized with the content displayed in the minutes data display area PRW1 on the time axis to control playback. That is, the reproduction position of the moving image is changed according to the position of the content displayed in the minutes data display area PRW1.

The method for achieving this is not limited, but as an example, in the minutes data conversion process, a chapter mark (scene) is set at the utterance start point of the audio track used to generate each content for the video data. Insert a pointer (pointer indicating each delimiter) in association with the content. Then, among the contents displayed in the minutes data display area PRW1, if the video data is played back using the chapter mark associated with the reference content of the video displayed in the video data display area PP1 as the playback start point. good.

In addition to the above, as an example, not limited to, the utterance start time of the audio track used to generate the first (first) content displayed in the minutes data display area PRW1 in the minutes data conversion process. , It can also be realized by setting the reproduction reference point (reproduction start time) of the image displayed in the image data display area PP1. Once the playback reference point (playback start time) is determined, the video (moving image) displayed in the video data display area PP1 is recorded in the minutes by playing back the video from the time (timing) corresponding to the speech time of the text content. It is possible to synchronize with the content displayed in the data display area PRW1 on the time axis.

In addition, as an example, not limited, the information of the time when the minutes creation is started (minutes creation start time) and the information of the time when the recording of the moving image is started (recording start time) are saved in the storage unit 28. I will do it. Then, the above-mentioned reproduction reference point may or may not be determined based on the difference between the minutes creation start time and the recording start time.

<Effect of the fifth embodiment>
In the fifth embodiment, the moving image (not limited, but an example of image information) of the whiteboard including at least a part of the video data (an example of a captured image, not limited) is a talk room (not limited, but an example of image information). , An example of a chat room), and the terminal 20 shows a configuration in which the content display area TR1 including the moving image is displayed on the display unit 24.
As an example of the effect of the embodiment obtained by such a configuration, even if it is difficult to express with text information, by using image information together, information can be shared more accurately in a chat room.

Further, in the fifth embodiment, the content such as text content (not limited, but an example of character information) is displayed in the minutes data display area PRW1 (not limited, but an example of the first display area) of the talk room. The image information such as a moving image shows a configuration displayed in the video data display area PP1 (not limited, but an example of the second display area) of the talk room.
As an example of the effect of the embodiment obtained by such a configuration, the image information and the character information can be displayed in the chat room so as not to interfere with each other.

Further, in the fifth embodiment, the terminal 20 controls the scroll control of the content in the talk room (not limited, but an example of control for changing the position of the first display area) based on the input by the user of the terminal 20. 21 is performed, and the position of the video data display area PP1 in the talk room (not limited, but an example of the position of the second display area in the chat room) shows a configuration that does not change based on the input by the user of the terminal 20.
As an example of the effect of the embodiment obtained by such a configuration, the position of the second display area is fixed regardless of the control of changing the position of the first display area, so that the viewability of the second display area is improved. Can be made to.

Further, in the fifth embodiment, the image information is moving image information, and the terminal 20 is based on operations such as up scrolling and down scrolling by the user of the terminal 20 (not limited, but an example of input by the user of the terminal). The control unit 21 is used to control the rewinding and fast-forwarding of the moving image (not limited to the control of reproducing the moving image).
As an example of the effect of the embodiment obtained by such a configuration, the control for reproducing the moving image can be executed based on the input of the user, so that the convenience of the user can be improved.

Further, in the fifth embodiment, the terminal 20 uses the minutes data of a part of the contents including the text contents (not limited, but an example of the third character information) based on the input by the user of the terminal 20. The control unit 21 controls the display in the display area PRW1 (not limited, but an example of the first display area). Then, the terminal 20 shows a configuration in which the control unit 21 controls to reproduce the moving image based on the content displayed in the minutes data display area PRW1.
As an example of the effect of the embodiment obtained by such a configuration, a control for displaying the third character information, which is a part of the character information, in the first display area based on the input by the user of the terminal is performed. Will be done. Then, since the control for reproducing the moving image can be executed based on the third character information displayed in the first display area, the convenience of the user can be improved.

Further, in this case, as an example, the reproduction position of the moving image can be changed based on a part of the contents (third character information) displayed in the minutes data display area PRW1.
As an example of the effect of the embodiment obtained by such a configuration, the moving image displayed in the second display area can be synchronized with the content displayed in the first display area on the time axis, so that the user can use it. Intuitive and easy-to-understand video reproduction can be realized.

Further, in the fifth embodiment, the terminal 20 controls the scroll control of the content in the talk room (not limited, but an example of control for changing the position of the first display area) based on the input by the user of the terminal 20. 21 shows a configuration in which the control unit 21 controls to reproduce a moving image corresponding to the content displayed on the display unit 24 based on the scroll control of the content.
As an example of the effect of the embodiment obtained by such a configuration, the control of reproducing the moving image corresponding to the character information displayed on the display unit of the terminal is executed based on the control of changing the position of the first display area. Since it is possible, the convenience of the user can be improved.

Further, the fifth embodiment shows a configuration in which the input by the user of the terminal 20 includes the input by the scroll operation.
As an example of the effect of the embodiment obtained by such a configuration, the user of the terminal can reproduce the moving image by a simple and intuitive operation called a scroll operation, which improves the convenience of the user. be able to.

<Fifth modification (1)>
In the fifth embodiment, an example in the batch addition mode has been described, but the present invention is not limited to this. As an example, not limited to, the minutes data may or may not be created based on the video data even in the real-time mode.

In this case, the image data may be acquired by using the imaging unit 27 and partial audio data may be created from the audio track of the video data as an example, not limited to this.

Further, as a display screen, the video data display area PP1 is displayed on the lower left side of the content display area TR1, and the display position of the video data display area PP1 is fixed even if the content display area TR1 is scrolled. It should be done.

At this time, when the user performs an up-scrolling operation on the content display area TR1, the video displayed in the video data display area PP1 is rewound and displayed according to the content displayed in the content display area TR1. ..
On the contrary, when the user performs a downward scroll operation on the content display area TR1, the video displayed in the video data display area PP1 is displayed in fast forward according to the content displayed in the content display area TR1.

<Fifth variant (2)>
In the fifth embodiment, when the minutes data is created using the video (moving image) data, the video is displayed in the video data display area PP1, but the present invention is not limited to this. As an example, not a limitation, when the edit function button BT1 is tapped and the insertion operation of a video (moving image) is selected, the video may be displayed in the video data display area PP1 or not. You may.

Even in this case, the video displayed in the video data display area PP1 is not limited, and as an example, a scroll operation for the content display area TR1 is performed, and a display operation such as fast forward or rewind is performed as an example, not limited. It may or may not be.

<Sixth Example>
The sixth embodiment is an embodiment relating to a display mode of the content (message) displayed in the chat room (talk room).
In the sixth embodiment, the mode in which the same content is displayed in the chat room differs depending on the type of messenger (messaging service, messaging application).
The contents described in the sixth embodiment can be applied to any of the other embodiments and the other modifications.

FIG. 6-1 is a diagram showing an example of a display format table TBL1 in which a display mode is defined for each type of messenger.
The display format table TBL1 is stored in the storage unit 15 of the server 10 as an example without limitation.

In the display format table TBL1, for each type of messenger, the line feed position, the balloon frequency, and the pictogram display are defined in association with each other as an example, not limited.

The line feed position is the number of characters that specify the length of one line in the content, and if it exceeds the number of characters specified by the line feed position, it is not limited but is automatically displayed as a line feed as an example.

The callout frequency defines the number of lines that can be included in a single text content, and if the number of lines specified by the callout frequency is exceeded, the text content is automatically divided as an example, not a limitation. , Displayed as multiple contents.

Pictogram notation is a regulation related to the display of pictograms, and is not limited, but as an example, the pictograms included in the content are displayed as pictograms (defined as "pictograms"), or converted to text that matches the content of the pictograms and displayed. Take the notation of either letting (specified as "text conversion") or deleting and displaying pictograms (specified as "delete").

Here, the pictogram is not a limitation but, as an example, refers to a thing or a thing defined as Emoji in the Unite 6 format or later, which symbolically suggests a thing or a thing by using a picture like a character.

The display format table TBL1 is not limited, but as an example, in "Messenger A", a line break is performed every 28 characters of the text in the content, and if it exceeds 10 lines, it is divided into another content and the pictogram is displayed as a pictogram. It stipulates that the aspect of causing is applied.

In addition, as an example, not limited, in "Messenger B", a line break is applied to every 16 characters of the text in the content, and if it exceeds 4 lines, it is divided into another content and the pictogram is displayed as a pictogram. It stipulates that.

When displaying sentences composed of character strings of the same length, the number of characters contained in a single content is smaller in "Messenger B" than in "Messenger A", and the number of characters per line is also smaller.
That is, it is presumed that the display area of "Messenger B" is narrower than the display area of "Messenger A".

As an example, not a limitation, in "Messenger C", a line break is performed every 30 characters in the text in the content, and if it exceeds 12 lines, it is divided into another content, and the pictogram is converted into text and displayed. It stipulates that it applies.

In this case, "messenger C" is presumed to be a messenger specialized in displaying text because the number of characters contained in a single content is large and the information in the content is replaced with characters.

As an example, not a limitation, in "Messenger D", a line break is applied to every 140 characters of a sentence in the content, one content contains only one sentence, and pictograms are deleted (omitted) and displayed. It stipulates.

From this, it is inferred that "Messenger D" is a messenger specialized in exchanging short sentences.

Based on the messenger used, the control unit 11 of the server 10 forms the minutes data in the format specified in the display format table TBL1 and adds (inputs) it to the chat room.

After receiving the minutes data from the server 10, the terminal 20A may or may not have different display modes according to the display format table TBL1.

<Effect of the sixth embodiment>
In the sixth embodiment, the text content of the minutes data (an example of text information, not a limitation) is input to the chat room in a manner based on the type of messenger A (an example of a chat service, not a limitation). Is shown.
As an example of the effect of the embodiment obtained by such a configuration, the display mode of the character information can be changed based on the characteristics of the chat service. This makes it possible to improve the readability of the user.

Further, the sixth embodiment shows a configuration in which the character information is broken at a line feed position based on the type of the chat service and is input to the chat room.
As an example of the effect of the embodiment obtained by such a configuration, the number of characters in one line can be changed according to the characteristics of the chat service, and the convenience of the user can be improved.

<Sixth variant (1)>
In the sixth embodiment, the control unit 11 of the server 10 forms the minutes data according to the display format table TBL1, but the present invention is not limited to this.
As an example, not a limitation, the shaping of the minutes data according to the type of the messenger may or may not be performed on the terminal 20.

In this case, the display format table TBL1 is stored in the storage unit 28 of the terminal 20 as an example, not a limitation. Then, when the control unit 21 of the terminal 20 receives the minutes data from the server 10, each content is molded according to the display format table TBL1 according to the type of the messenger, and is displayed on the display unit 24.

<Sixth variant (2)>
In the sixth embodiment, the display format table TBL1 defines the display mode for each type of messenger, but the display mode is not limited to this.

As an example, not a limitation, the display mode may or may not be defined according to the size of the display area of the display unit 24 of the terminal 20.

In this case, as an example, not limited, the display mode of the messenger A is used when the display unit 24 is larger than the 1334 × 750 pig cell, and the display mode of the messenger B is used when the display screen is 1334 × 750 pig cells or less. , Each content may be molded.

<7th Example>
The seventh embodiment is an embodiment in which viewing restrictions are set for the minutes data displayed in the chat room (talk room) according to the attendance status of the meeting or the meeting.
The seventh embodiment is an embodiment relating to the timing of (A) above.
The contents described in the seventh embodiment can be applied to any of the other embodiments and the other modifications.

<Data structure>
FIG. 7-1 shows an example of data configuration of the calendar information management database 159B, which is an example of the calendar information management database 159 in this embodiment.
The calendar information management database 159B stores calendar information management data as management data for each event registered in the calendar of the schedule management application.

Each calendar information management data is not limited, but as an example, event ID, MS application ID, event name, event start date and time, event end date and time, event venue, related URI, and invitee data. And attendance management data are stored.

The event ID, MS application ID, event name, event start date and time, event end date and time, event venue, related URI, and invitee data are not limited, but as an example, the calendar information management database 159A. Since it is the same as the above, the description thereof will be omitted again.

In the attendance management data, as information of the user invited to the event identified by the event ID, the MS application ID, the attendance time, and the leaving time are stored in association with each other as an example, not limited to the information.

The MS application ID is not limited, but is, for example, the user's MS application ID stored in the invitee data.

The attendance time is not limited, but the user's terminal 20 identified by the MS application ID reaches the reception range of the beacon signal transmitted from the beacon device installed at the event venue identified by this event ID. It is the time when

The leaving time is not limited to the user's terminal 20 identified by the MS application ID, but as an example, the user's terminal 20 leaves the reception range of the beacon signal transmitted from the beacon device installed at the event venue identified by this event ID. It is the time when

That is, the attendance time and the leaving time are stored in the attendance management data after the event start date and time identified by this event ID.

If a terminal repeatedly approaches and leaves the beacon device after the event start date and time, a plurality of pairs of attendance time and attendance time may be stored in the attendance management data, and so on. It does not have to be. Further, when the terminal does not approach the beacon device, the value of "Null" may be entered in the attendance time and the leaving time as an example, not a limitation, indicating that the user is absent. , You don't have to do that.

Further, the method of acquiring the attendance time and the leaving time is not limited to the method using the beacon device. As an example, not a limitation, the position information is acquired by the position calculation information detection unit 29B of each terminal 20 based on the position information that can be identified by the latitude and longitude of the event venue. When the terminal 20 moves to a position that matches the position information of the event venue within a predetermined range, the time may or may not be the attendance time of the user of the terminal 20. When the terminal 20 leaves a position that matches the position information of the event venue within a predetermined range, that time may or may not be the time when the user of the terminal 20 leaves the desk.

Further, as an example, not limited to, the attendance of the user is performed by acquiring an image during the event using the camera installed at the event venue or the imaging unit 27 of the terminal 20 and performing face recognition on the image. You may or may not get the status.

Alternatively, as an example, not limited to, a sound (including voice) is acquired during the event using a microphone installed at the event venue or the sound input unit 25 of the terminal 20, and sound recognition (including voice) is performed for the sound. By performing (including voice recognition), the attendance status of the user may or may not be acquired.

Alternatively, as an example, not a limitation, the attendance status of the user may or may not be acquired based on the user input to the terminal 20 that stamps the attendance time and the leaving time.

When performing the minutes data conversion process, the mention function may be activated only for the users or groups who have not attended the meeting or the meeting for which the voice data has been acquired, based on the attendance time and the leaving time. You don't have to.

<Display screen>
FIG. 7-2 is an example, not limited to, an example of the display screen of the talk room of the messaging application when the minutes posting button SBT1 is tapped by the user in FIG. 2-12.

In FIG. 7-2, the content display area TR1 is not limited to the user A. The minutes data summary content PRC2 is displayed under the text content of the characters "The minutes of the previous meeting" sent by A.

The minutes data summary content PRC2 is not limited, but as an example, the minutes title and the date and time when the minutes data was last modified are displayed as in the minutes data summary content PRC1. Further, in the minutes data summary content PRC2, an attendance confirmation function button BT8 for confirming the attendance status of each user at the time of acquiring the audio data used for creating the minutes data is displayed.

FIG. 7-3 is an example of a display screen when the attendance confirmation function button BT8 is tapped by the user.
As an example, not a limitation, when the attendance confirmation function button BT8 is tapped by the user, the attendance confirmation status display area WR7 is displayed in the content display area TR1.

At the left end of the attendance confirmation status display area WR7, icons indicating the user of the MS application ID stored in the attendance management data are displayed side by side as an example, not limited to the attendance confirmation status display area WR7. To the right of each user's icon, as an example, not a limitation, a graph showing whether or not the user was present during the time from the start of acquisition to the end of acquisition of the audio data that is the source of this minutes data is lined up. It is displayed for each.
These graphs are drawn based on the attendance time and the attendance time of each user stored in the attendance management data as an example, not a limitation.

In the attendance confirmation status display area WR7, as an example, not limited to the user A. A and user B. B and user C. C indicates that he was present from the start to the end of the meeting (event). In addition, user D. It is said that D has been present since 20 minutes from the start. It is displayed that E has left the seat one hour after the start.

FIG. 7-4 shows the user D. This is an example of a talk room display screen when the minutes data is displayed on the terminal 20D of D.

In FIG. 7-4, the minutes data display area PRW2 is displayed in the content display area TR1 based on the fact that the minutes data summary content PRC2 is tapped by the user. At the upper part of the minutes data display area PRW2, an attendance confirmation function button BT8 is displayed in addition to the display elements of the minutes data display area PRW1.

User D. Based on the fact that D was absent from the start of voice data acquisition until 20 minutes, the minutes data display area PRW2 indicates that viewing restrictions are applied to the content part before 20 minutes. The text "Browsing is locked" and the lock status icon IC1 of the locked key mark are displayed. That is, in the terminal 20D, the user D. The contents of the minutes data of the time zone when D was absent cannot be displayed.

As an example, not a limitation, when the lock status icon IC1 is tapped by the user, the user A. A screen for selecting whether or not to release the viewing restriction is displayed on the display unit 24 of the terminal 20A of the terminal A.
When it is selected that the viewing restriction is released based on the user operation, the content portion 20 minutes or earlier of the minutes data display area PRW2 can also be viewed on the terminal 20D, and those contents are displayed. ..

When the lock status icon IC1 is tapped by the user, a screen for selecting whether or not to release the viewing restriction is displayed for all users in the talk room to which the minutes data is posted. It may or may not be.

Similar to the above display screen example, the user E.I. On the display screen of the talk room when displaying the minutes data on the terminal 20D of E, the user E.I. Based on the fact that E has been absent after 60 minutes from the start of voice data acquisition, viewing restrictions are applied to the content portion of the minutes data after 60 minutes.

<Processing>
FIG. 7-5 is a flowchart showing an example of the flow of processing executed by each device in this embodiment.
From left to right, examples of processing executed by the control unit 21 of the terminal 20D, processing executed by the control unit 21 of the terminal 20A, and processing executed by the control unit 11 of the server 10 are shown.

First, the control unit 21 of the terminal 20A and the control unit 11 of the server 10 execute the minutes posting process. The minutes posting process is not limited, but can be realized by the same process as in FIG. 2-16 as an example, and therefore detailed description thereof will be omitted.

Next, the control unit 11 of the server 10 targets the minutes restriction data for invisible the contents of the time zone in which each user was absent (there is a time zone in which the user was absent) for each user. (S300).

Then, the control unit 11 of the server 10 transmits the minutes restriction data to the target terminal 20 by the communication I / F 14 (S310).

When the control unit 21 of the terminal 20D receives the minutes restriction data from the server 10 by the communication I / F 22, the control unit 21 imposes a viewing restriction on the minutes data and displays it on the display unit 24 (D100).

When it is selected to release the viewing restriction of the minutes data based on the user operation on the input / output unit 23, the control unit 21 of the terminal 20D transmits the viewing restriction release request information to the server 10 by the communication I / F 22. (D110). If it is selected not to release the restriction on viewing the minutes data, the control unit 21 of the terminal 20D ends the process.

When the viewing restriction release request information is received from the terminal 20D by the communication I / F 14, the control unit 11 of the server 10 transmits the viewing restriction release confirmation information to the terminal 20A by the communication I / F 14 (S320). When the viewing restriction release request information is not received, the control unit 11 of the server 10 ends the process.

The control unit 11 of the server 10 transmits the viewing restriction release confirmation information to all other user terminals (other than receiving the viewing restriction release request information) belonging to the talk room to which the minutes data is posted. It may or may not be.

When the control unit 21 of the terminal 20A receives the viewing restriction release confirmation information from the server 10 by the communication I / F 22, the control unit 21 causes the display unit 24 to display a screen for selecting whether or not to release the viewing restriction (A250). If the viewing restriction release confirmation information is not received, the control unit 21 of the terminal 20A ends the process.

When it is selected to release the viewing restriction based on the user operation on the input / output unit 23, the control unit 21 of the terminal 20A transmits the viewing permission information to the server 10 by the communication I / F 22 (A260) and processes it. To finish. If it is selected not to release the viewing restriction, the control unit 21 of the terminal 20A ends the process.

When the viewing restriction release confirmation information is transmitted to a terminal other than the terminal 20A, the same process as the steps of A250 and A260 may be executed.

When the viewing permission information is received from the terminal 20A by the communication I / F 14, the control unit 11 of the server 10 executes the viewing restriction canceling process so that the minutes data can be confirmed on the terminal that has sent the viewing restriction release request information. (S330). If the viewing permission information is not received, the control unit 11 of the server 10 ends the process.

Then, when the control unit 11 of the server 10 transmits the minutes data to the terminal 20D by the communication I / F 14 (S340), the process ends.

When the viewing restriction release confirmation information is transmitted to a plurality of terminals, the conditions for executing the viewing restriction release process are not limited, but the following cases can be considered as an example.
1) When the viewing permission information is received from all the terminals that sent the viewing restriction release confirmation information 2) When the viewing permission information is received from the majority of the terminals that sent the viewing restriction release confirmation information 3) The minutes data was posted When receiving browsing permission information from the terminal 4) When receiving browsing permission information from the terminal of the reader specified in the talk room

When the control unit 21 of the terminal 20D receives the minutes data from the server 10 by the communication I / F 22, the control unit 21 updates and displays the minutes data on the display unit 24 (D120). Then, the control unit 21 of the terminal 20D ends the process.

<Effect of the 7th Example>
In the seventh embodiment, the minutes data (not limited, but an example of character information) is obtained by the user A. A (not limited, but an example of a terminal user) and user B. B (not limited, but an example of a user of the first terminal) and user D. It is input to the talk room (an example of the first chat room, not the limitation) of the group "X" including D (an example of the user of the second terminal, not the limitation), and at least the terminal 20D (the second, not the limitation). The configuration to be transmitted to (an example of a terminal) is shown.
As an example of the effect of the embodiment obtained by such a configuration, the terminal user, the user of the first terminal, and the user of the second terminal can share information via the first chat room. Therefore, the convenience of the user can be improved.

Further, in the seventh embodiment, a fourth content (not limited, but a part of the character information, which is a part of the character information) of the minutes data transmitted to the terminal 20D (second terminal). (Example of character information) shows a configuration that is not displayed on the display unit 24 of the terminal 20D.
As an example of the effect of the embodiment obtained by such a configuration, since a part of the character information is not displayed on the display unit of the second terminal, it is possible to limit the information to be shared with the user of the second terminal. ..

Further, in the seventh embodiment, a part of the contents of the minutes data (not limited, but an example of the fourth character information) is obtained by the user A. A (not limited, but an example of a terminal user) and user B. The configuration to be displayed on the display unit of the second terminal is shown by permission based on B (not limited, but an example of the user of the first terminal).
As an example of the effect of the embodiment obtained by such a configuration, the user of the second terminal can browse the restricted information if there is permission based on the user of the terminal and the user of the first terminal. Therefore, the information sharing range can be flexibly set.

Further, in the seventh embodiment, the content (not limited, but an example of the fourth character information) of the minutes data up to the first predetermined time (not limited, but 20 minutes after the start) after the start is a terminal. Before the information (not limited, as an example of the beacon device) regarding the terminal 20D (not limited, but an example of the second terminal) is acquired by the server 10 communicating with the terminal 20 or the terminal 20, the sound input unit 25 Shows the configuration of the acquired audio data that is content (an example of fifth character information, not limitation) based on audio data (an example of third sound information, not limitation) up to the first predetermined time after the start. ing.
As an example of the effect of the embodiment obtained by such a configuration, it is possible to determine the fourth character information in conjunction with the information regarding the second terminal. Further, the fifth character information is character information until the information about the second terminal is acquired. As a result, the character information before the information about the second terminal is acquired can be prevented from being displayed on the second terminal.

Further, in the seventh embodiment, the content (not limited, but an example of the fourth character information) from the start second predetermined time (not limited, but one hour after the start) of the minutes data is a terminal. 20 or the server 10 that communicates with the terminal 20 acquires the information (not limited, but an example of the second terminal) about the terminal 20E (not limited, but as an example, the detachment information of the beacon device) by the sound input unit 25. Shows the configuration of the content (an example of the sixth character information, not the limitation) based on the audio data (an example of the fourth sound information, not the limitation) after the second predetermined time after the start of the voice data. There is.
As an example of the effect of the embodiment obtained by such a configuration, the sixth character information becomes the character information after the information regarding the second terminal is acquired. As a result, it is possible to prevent the character information after the information about the second terminal is acquired from being displayed on the second terminal.

Further, in the seventh embodiment, the above-mentioned information regarding the second terminal shows a configuration including information regarding the position of the second terminal.
As an example of the effect of the embodiment obtained by such a configuration, the character information before the information on the second terminal is acquired is not displayed on the second terminal by being based on the information on the position of the second terminal. In addition, it is possible to easily and appropriately prevent the character information after the information about the second terminal is acquired from being displayed on the second terminal.

<7th modification (1)>
In the seventh embodiment, the case where the viewing restriction is set for the minutes data displayed in the chat room according to the attendance status of the user has been described, but the present invention is not limited to this. As an example, not limited to, multiple chat rooms (talk rooms) may be prepared, and minutes data may be divided and added (input) to each chat room according to the attendance status of the user. It does not have to be.

FIG. 7-6 is an example of the display screen of the talk room in this modified example.
In FIG. 7-6, in the current position display area, "(4/5)" indicating that the talk room is composed of 4 users out of 5 members of the group "Project X" is displayed. Has been added. Below that is User A., a user who belongs to this talk room. A and user B. B and user C.I. C and user E. The icon with E is displayed.

Below that, the content display area TR2 for displaying the content (message) transmitted (posted) to the talk room is displayed.

In the content display area TR2, the user D. Based on the fact that D was absent from the start of the meeting to the time of 20 minutes, the minutes data is input according to the display mode of Example 3, not as a limitation but as an example. That is, the content added (input) to the content display area TR2 is the minutes data from the start of acquisition of the audio data to 20 minutes later, and among the members of the group "Project X", the user D. Content other than D can be viewed.

At the bottom of the content display area TR2, a talk room transition function button IC2 in which the icons of all the members of the group "Project X" are displayed is displayed. As an example, not a limitation, when the talk room transition function button IC2 is tapped by the user, a talk room screen that can be viewed by all members of the group "Project X" is displayed.

FIG. 7-7 is an example of a talk room display screen when the talk room transition function button IC2 is tapped by the user in FIG. 7-6.

In FIG. 7-7, in the current position display area, "(5/5)" indicating that the talk room is composed of 5 users out of 5 members of the group "Project X" is displayed. Has been added. Below that, the icons of all the users who belong to this talk room are displayed.

Below that, the content display area TR1 for displaying the content (message) transmitted (posted) to this talk room (that is, the talk room of the group "project X") is displayed.

In the content display area TR1, the user D. D attended from 20 minutes after the start of the meeting, and user E. Based on the fact that E was absent one hour after the start of the meeting, the minutes data is input according to the display mode of the third embodiment, not as a limitation but as an example.

The content displayed in the content display area TR1 is content that can be viewed by all members of the group "project X". Then, among the minutes data, the minutes data from 20 minutes to 1 hour after the start of acquisition of the audio data is added and displayed in the content display area TR1.

At the top of the content display area TR1, among the members of the group "Project X", the user D. The talk room transition function button IC3 on which an icon other than D is displayed is displayed. Further, at the bottom of the content display area TR1, among the members of the group "Project X", the user E.I. The talk room transition function button IC4 on which an icon other than E is displayed is displayed.

As an example, not a limitation, when the talk room transition function button IC3 is tapped by the user, among the members of the group "Project X", the user D. A talk room (that is, the content display area TR2) that can be viewed by anyone other than D is displayed.
As an example, not a limitation, User D. D can browse the content of the content display area TR1, but the talk room does not transition even if the talk room transition function button IC3 is tapped, and the content input to the content display area TR2 cannot be browsed. .. That is, the user D. D cannot view the minutes data from the start to 20 minutes when he is not attending the meeting.

Further, when the talk room transition function button IC4 is tapped by the user, among the members of the group "Project X", the user E.I. A talk room that can be viewed by anyone other than E is displayed.
At this time, the user E.I. Even if E taps the talk room transition function button IC4, the talk room does not transition and the content input to the transition destination talk room cannot be viewed. That is, the user E.I. E cannot view the minutes data that has passed one hour since the start of not attending the meeting.

It is not essential to display the talk room transition function button in the content display area TR1. When the talk room transition function button is not displayed, the transition between talk rooms may or may not be executed based on the user operation as an example, not limited to the above.

Also, as an example, not limited, there is a talk room whose members are users (in this case, users AA, user BB, and user CC) who are present from the start to the end of the conference (event). The entire minutes data, which is automatically generated, may or may not be added (input) to this talk room.

In this modified example, the content (not limited, but an example of the fourth character information) up to the third predetermined time after the start (not limited, but 20 minutes after the start) of the minutes data is described by the user A. A (not limited, but an example of a terminal user) and user B. Included as B (not limited, but an example of a user of the first terminal), user D. Characters that are input to the second talk room (content display area TR2 as an example, not limited) that does not include D (an example of a user of the second terminal, not limited) and do not include the fourth character information of the character information. The information shows a configuration to be input to the first talk room (content display area TR1 as an example, not limited).
As an example of the effect of the modification obtained by such a configuration, the fourth character information is in the second talk room that does not include the user of the second terminal, and the character information that does not include the fourth character information is in the first talk room. Since it is input, the character information to be input differs for each talk room. Therefore, the viewing restriction of the character information can be performed more naturally without making the user of the second terminal aware of it.

Further, in this modification, the transition from the second talk room (second chat room) to the first talk room (first chat room) and the transition from the first talk room to the second talk room are It is executed by the terminal 20A (not limited, but an example of a terminal) and the terminal 20B (not limited, an example of a first terminal). On the other hand, the transition from the first talk room to the second talk room is not limited by the terminal 20 other than these terminals 20, but by the terminal 20D or the terminal 20E (not limited, but an example of the second terminal). Indicates a configuration that will not be executed.
As an example of the effect obtained by such a configuration, it is possible to appropriately realize the restriction of viewing the character information to the user of the second terminal.

<7th variant (2)>
In the seventh embodiment, the minutes data received by the terminal 20 is restricted from being viewed, but the present invention is not limited to this. As an example, not a limitation, the server 10 may restrict the viewing of the minutes data, and the control unit 11 of the server 10 may or may not transmit a part of the minutes data to each terminal 20. You may.

FIG. 7-8 is a flowchart showing an example of the flow of processing executed by each device in this embodiment.
From left to right, examples of processing executed by the control unit 21 of the terminal 20D, processing executed by the control unit 21 of the terminal 20A, and processing executed by the control unit 11 of the server 10 are shown.

First, the control unit 21 of the terminal 20A and the control unit 11 of the server 10 execute the minutes posting process. The minutes posting process is not limited, but can be realized by the same process as in FIG. 2-16 as an example, and therefore detailed description thereof will be omitted. However, in the step of S170 of FIG. 2-16, the minutes data is not transmitted to the terminal of the user who has not participated in the event (meeting) in which the audio data has been acquired.

Next, the control unit 11 of the server 10 targets the partially restricted minutes data, which is the minutes data that does not include the contents of the time zone in which each user was absent (there is a time zone in which the user was absent). Generated for each user (S350).

Then, the control unit 11 of the server 10 transmits the partially restricted minutes data to the target terminal 20 by the communication I / F 14 (S360).

When the control unit 21 of the terminal 20D receives the partial restriction minutes data from the server 10 by the communication I / F 22, the control unit 21 displays it on the display unit 24 (D150).

When the control unit 11 of the server 10 executes the viewing restriction release process (S330), the minutes data is transmitted to the terminal 20D by the communication I / F 14 (S370), and the process ends.

When the control unit 21 of the terminal 20D receives the minutes data from the server 10 by the communication I / F 22, the control unit 21 updates and displays the minutes data including the full text on the display unit 24 (D120).

As an example of the effect of the modified example obtained by such a configuration, it is not essential to transmit the entire minutes data to the terminal 20 whose viewing is restricted, so that the amount of communication can be reduced.

1 communication system 10 servers 20 terminals

Claims (29)

A program executed by a terminal of the user who displays a chat room including the user and a user of a first terminal different from the user.
Acquiring sound information by the acquisition unit of the terminal,
Character information based on the sound information acquired by the acquisition unit is input to the selected chat room, and the chat room containing the character information is displayed on the display unit of the terminal. Will be done. The program according to claim 1.
The chat room is selected based on input by the user of the terminal. The program according to claim 1.
The chat room is selected based on the first information acquired by the first acquisition unit of the terminal. The program according to claim 3.
The first information includes sound information acquired by the acquisition unit. The program according to claim 3.
The first information includes information transmitted from an electronic device. The program according to claim 5.
The electronic device includes the first terminal. The program according to claim 3.
The first information includes image information of an captured image captured by an imaging unit of the terminal. The program according to claim 3.
The first information includes schedule information of the user of the terminal. The program according to any one of claims 1 to 8.
The first character information based on the first sound information acquired by the acquisition unit is input to the selected chat room in association with the user of the terminal, and the second sound information based on the second sound information acquired by the acquisition unit is input. The two-character information is input to the chat room in association with the user of the first terminal, and the terminal can display the chat room including the first character information and the second character information on the display unit. Will be executed. The program according to any one of claims 1 to 9.
The sound information is acquired by the acquisition unit before the chat room is selected.
The character information is input to the chat room based on the selection of the chat room. The program according to claim 9.
Based on the acquisition of the first sound information by the acquisition unit, the first character information is input to the chat room, and the chat room including the first character information is displayed on the display unit. ,
Based on the acquisition of the second sound information by the acquisition unit, the second character information is input to the chat room, and the chat room including the second character information is displayed on the display unit. Is executed by the terminal. The program according to any one of claims 1 to 11.
The character information is input to the chat room in a manner based on the type of chat service. The program according to claim 12.
The character information is broken at a line feed position based on the type of the chat service and is input to the chat room. The program according to any one of claims 1 to 13.
Image information including at least a part of the captured image is input to the chat room, and the terminal executes the display of the chat room including the image information on the display unit. The program according to claim 14.
The character information is displayed in the first display area of the chat room, and is displayed.
The image information is displayed in the second display area of the chat room. The program according to claim 15.
The terminal executes control to change the position of the first display area in the chat room based on the input by the user of the terminal by the control unit of the terminal.
The position of the second display area in the chat room does not change based on input by the user of the terminal. The program according to claim 15 or 16.
The image information is moving image information, and is
Based on the input by the user of the terminal, the control unit of the terminal controls the reproduction of the moving image by the terminal. The program according to claim 17.
Based on the input by the user of the terminal, the control unit controls to display the third character information, which is a part of the character information, in the first display area.
The terminal executes control for reproducing the moving image based on the third character information displayed in the first display area. The program according to any one of claims 16 to 18.
The input by the user of the terminal includes the input by the scroll operation. The program according to any one of claims 1 to 19.
The character information is input to a first chat room including a user of the terminal, a user of the first terminal, and a user of the second terminal, and is transmitted to at least the second terminal. The program according to claim 20.
The fourth character information, which is a part of the character information transmitted to the second terminal, is not displayed on the display unit of the second terminal. The program according to claim 21.
The fourth character information is displayed on the display unit of the second terminal with permission based on the user of the terminal and the user of the first terminal. 21 or the program according to claim 22.
The fourth character information is fifth character information based on the third sound information acquired by the acquisition unit before the information about the second terminal is acquired by the terminal or the server communicating with the terminal. The program according to any one of claims 21 to 23.
The fourth character information is sixth character information based on the fourth sound information acquired by the acquisition unit after the information about the second terminal is acquired by the terminal or a server communicating with the terminal. The program according to claim 23 or 24.
The information regarding the second terminal includes information regarding the position of the second terminal. The program according to claim 21.
The fourth character information is input to a second chat room that includes the user of the terminal and the user of the first terminal and does not include the user of the second terminal.
Character information that does not include the fourth character information among the character information is input to the first chat room. The program according to claim 26.
The transition from the second chat room to the first chat room and the transition from the first chat room to the second chat room are executed by the terminal and the first terminal.
The transition from the first chat room to the second chat room is not executed by the second terminal. A method of displaying a terminal of the user, which displays a chat room including a user and a user of a first terminal different from the user.
Acquiring sound information by the acquisition unit of the terminal,
Character information based on the sound information acquired by the acquisition unit is input to the selected chat room, and the chat room including the character information is displayed on the display unit of the terminal. A terminal of the user that displays a chat room that includes the user and a user of a first terminal different from the user.
The acquisition unit that acquires sound information,
Character information based on the sound information acquired by the acquisition unit is input to the selected chat room, and includes a display unit that displays the chat room including the character information.

Images