JP2023147490A - Processing device, processing method and processing program - Google Patents

Abstract

To divide text information obtained by voice recognition into one or a plurality of pieces of element information and to generate output information by associating attribute information with the element information.SOLUTION: At least one processor included in a processing device is configured to perform processing for receiving input of voice information including contents of utterances of one or a plurality of speakers based on input operation by a user, generating text information indicating the contents of the utterances of the one or the plurality of speakers based on the received voice information, dividing the generated text information into one or a plurality of pieces of element information, associating at least one attribute information with each divided element information, and generating output information in which at least a part of each element information is associated with any item in one or a plurality of items in form information having the one or the plurality of items based on the attribute information associated with each element information.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to a processing device, a processing method, and a processing program that can convert and edit audio information into text information.

Speech recognition, which converts voice information into text information, is commonly used. Additionally, devices are commonly used to identify the speaker who uttered the voice. For example, Patent Document 1 discloses a system that generates minutes from audio information of a meeting or the like. Further, Cited Document 2 discloses a device for identifying a speaker. However, Patent Document 1 divides text information obtained by speech recognition, associates element information obtained by the division with attribute information indicating meaning, speaker, etc., and associates it with items included in an output form. do not disclose.

JP2022-028776A Japanese Patent Application Publication No. 10-313357

This embodiment was developed based on the above-mentioned background, and involves dividing text information obtained through speech recognition into one or more element information, and generating output information by associating attribute information with each element information. The task is to Furthermore, the present embodiment aims to generate output information by further associating element information with speaker information.

A processing device according to the present disclosure is a processing device including at least one processor, and the at least one processor processes audio information including the content of utterances of one or more speakers based on an input operation by a user. Receive the input, generate text information indicating the content of the one or more speakers' utterances based on the received audio information, and divide the generated text information into one or more element information. At least one attribute information is associated with each element information, and based on the attribute information associated with each element information, at least a part of each element information in form information having one or more items is The device is configured to execute processing for generating output information associated with any one of the one or more items.

Further, the processing method according to the present disclosure is a method performed in a computer including at least one processor, in which the at least one processor executes a predetermined instruction command, and the processing method is performed by executing a predetermined instruction command based on an input operation by a user. or a step of receiving an input of voice information including contents of utterances of a plurality of speakers; and a step of generating text information indicating contents of utterances of the one or more speakers based on the received voice information; dividing the text information into one or more element information; associating at least one attribute information with each divided element information; Alternatively, in form information having a plurality of items, at least a part of each element information is associated with any one of the one or more items.

A processing program according to the present disclosure causes a computer including at least one processor to accept input of voice information including utterance content of one or more speakers based on an input operation by a user, and based on the received voice information. generating text information indicating the content of the one or more speakers' statements, dividing the generated text information into one or more element information, and assigning at least one attribute information to each of the divided element information. and, based on the attribute information associated with each element information, in form information having one or more items, at least a part of each element information corresponds to one of the one or more items. Act as a processor configured to perform processing that generates output information associated with the item.

According to the present disclosure, it is possible to divide text information obtained through speech recognition into one or more pieces of element information, associate attribute information with each piece of element information, and generate output information. Further, according to the present disclosure, output information can be generated by further associating element information with speaker information.

Note that the effects described above are merely illustrative for convenience of explanation, and are not limiting. In addition to or in place of the effects described above, any effects described in this disclosure or effects obvious to those skilled in the art may be achieved.

FIG. 1 illustrates a form 10 showing a display format in output information 14 of element information 12, attribute information 16, and speaker information 18 generated by editing text information obtained by voice recognition of voice information. This is a diagram. FIG. 2 shows a process of processing audio information from a meeting, etc. to identify text information indicating the contents of the remarks made in the meeting, as well as speakers who spoke in the meeting, and generating minutes of the meeting according to the form 10 shown in FIG. 1. FIG. 2 is a diagram illustrating the configuration of a terminal device 100 that executes. FIG. 3A is a diagram showing a form information table. FIG. 3B is a diagram showing the audio information table. FIG. 3C is a diagram showing an element information table. FIG. 4A is a diagram illustrating text information H included in the audio information table shown in FIG. 3B. FIG. 4B is a diagram illustrating conversion information included in the audio information table (FIG. 3B) generated from the text information shown in FIG. 4A. FIG. 4C is a diagram illustrating element information 12 (element information J) included in the audio information table (FIG. 3B) generated from the conversion information shown in FIG. 4B. FIG. 4D is a diagram illustrating a UI image in which check boxes, element information 12 shown in FIG. 4C, flags F1 to F4 indicating attributes, and speaker information 18 indicating speakers A to C are associated with each other. . FIG. 5A is a flowchart showing a process in which the user registers information indicating form information C of the output information 14 of the form information included in the form information table (FIG. 3A) in the terminal device 100. FIG. 5B is a diagram illustrating the form 10 indicated by the form information C of the output information 14 indicated by the form information. FIG. 6A is a flowchart showing audio information processing by the terminal device 100. FIG. 6B is a diagram illustrating a list of element information 12 associated with flags indicating attributes. FIG. 6C is a diagram illustrating a UI image used to move the element information 12 and the like to a box included in the form 10 of the output information 14 and associated with the attribute information 16.

Hereinafter, as an embodiment of the present disclosure, a process of processing text information indicating the contents of a meeting uttered obtained by voice identification processing etc. on voice information and generating minutes according to a predetermined form will be described with reference to the drawings. This will be explained in detail with reference to the following. Note that substantially the same components, processes, and information in the drawings are given the same reference numerals and names. Furthermore, "information" and "data" are not strictly distinguished.

Further, in the drawings, the numbers and types of components and data are shown by way of example, and may be increased, decreased, or changed as appropriate. Furthermore, in the drawings, the order of communication between devices is shown as an example, and may be changed as appropriate. Furthermore, in the drawings, components that are not related to the essential description of the invention may be omitted as appropriate. Furthermore, for convenience of illustration, some names of components and information such as "information" and "module" may be omitted as appropriate in the drawings. Furthermore, when it is not necessary to distinguish between multiple terms such as "element information 12a to 12d", the subscript of the code may be omitted and the term may be written as "element information 12" or the like.

1. Outline of Processing by Terminal Device 100 First, with reference to FIGS. 1 and 2, an overview of the process of generating minutes from the audio of a meeting according to the present embodiment will be described. FIG. 1 illustrates a form 10 showing a display format in output information 14 of element information 12, attribute information 16, and speaker information 18 generated by editing text information obtained by voice recognition of voice information. This is a diagram. FIG. 2 shows a process of processing audio information from a meeting, etc. to identify text information indicating the contents of the remarks made in the meeting, as well as speakers who spoke in the meeting, and generating minutes of the meeting according to the form 10 shown in FIG. 1. FIG. 2 is a diagram illustrating the configuration of a terminal device 100 that executes.

Note that the form 10 is defined in order to output one or more items such as element information 12, attribute information 16, speaker information 18, etc. in the output information 14 in a format desired by the user. A user can define an arbitrary form 10 and cause the terminal device 100 to display element information 12 and the like in output information 14 according to the defined form 10.

The terminal device 100 shown in FIG. 2 processes various audio information, such as audio information from an online conference, and audio information collected via an offline microphone 119, and displays the information in accordance with the form 10 shown in FIG. (Illustrated) etc. However, for clarity and specificity of explanation, a case will be described below in which the terminal device 100 receives input of audio information of a conference held offline. Further, a case will be described in which the terminal device 100 generates text information indicating the content of remarks made in a conference by voice identification, and divides the generated text information to generate element information 12. Also, a case will be described in which the terminal device 100 identifies a speaker who has uttered conversion information including the element information 12. Further, a case will be described in which the terminal device 100 associates the element information 12 with attribute information 16 indicating the attribute of the element information 12 and speaker information 18 indicating the speaker corresponding to the element information 12. Note that speaker recognition is to identify a speaker who has made a statement that includes the element information 12. Furthermore, a specific example is a case where the terminal device 100 associates the element information 12, the attribute information 16, and the speaker information 18 and displays them according to the form 10.

In a conference or the like, audio information indicating the voices of one or more speakers such as conference attendees is recorded by the terminal device 100 via the microphone 119 or the like. By performing voice recognition processing on the recorded voice information, the voice information is converted into one or more pieces of text information. The text information obtained from the conference audio may include multiple sentences. Therefore, text information can be divided into multiple sentences. On the other hand, this text information includes, for example, a character string such as "Well, Mr. A, please generate seminar materials by month x date." Therefore, the sentences obtained by dividing the text information may also include portions that do not need to be included in the minutes, such as honorific expressions such as "please." In this way, parts that do not need to be included in the minutes may be deleted.

Note that the text information obtained by division does not include delimiter information such as punctuation marks "," and ".", commas and periods "," and ".", and spaces, which indicate delimiters between pieces of text information. On the other hand, in order to delimit a plurality of pieces of text information, it is necessary to insert delimiter information between the pieces of text information, and it is also necessary to post the delimiter information after the first and last text information. Note that in Japanese text information, the delimiter information that delimits text sentences may be punctuation mark information that indicates the delimitation of sentences, such as punctuation marks "," and ".". In this way, by removing unnecessary honorific words at the end of sentences from text information and inserting punctuation information between pieces of text information, conversion information can be obtained that has been converted so that the boundaries between pieces of text information are clear. .

The conversion information is composed of words and word groups, and may include one or more element information 12 having some meaning. For example, the conversion information ``Well, first, Mr. A will generate seminar materials by month x date.'' is converted into multiple pieces of element information 12 each having a specific meaning through morphological analysis processing, functional analysis processing, etc. Can be divided. For example, this conversion information includes element information 12a to 12d each consisting of words and word groups such as "Eh~", "First of all", "Mr. A by month x date", and "Generate seminar materials". can be divided into

The element information 12 is included in the conversion information, and one piece of conversion information can be presumed to have been uttered by the same speaker. Therefore, the speaker of the element information 12 can change the speech information corresponding to the conversion information including the element information 12. It can be identified by performing speaker identification processing on the speaker. In other words, speaker identification processing can be performed on the text information via the conversion information, and through this processing, the speaker of the element information 12 is specified, and the speaker information indicating this speaker is added to the element information 12. 18 can be associated.

"E~" (not shown in FIG. 1) of the element information 12a included in this conversion information is an interjection and does not need to be included in the minutes. "First" in the element information 12b means the former (order) of the action order or deadline of "generating seminar materials", and therefore needs to be included in the minutes. Element information 12c, “Mr. A, by month x date” means the latter (deadline) of the action order or deadline of “generate seminar materials” in element information 12d, and the subject. must be included in the record. "Generate seminar materials" in the element information 12d means an action, and therefore needs to be included in the minutes.

As explained above, the element information 12 includes, for example, "Good morning" (not shown) as a greeting before a speaker makes a statement in a meeting, and Japanese words such as "E~" in the element information 12a that is prefixed to a statement. It can include words called interjections in grammatical terms. Furthermore, the element information 12 may include a group of words such as "Good morning", which is an interjection "Good morning" with the honorific word "Mazaimasu" (not shown) added thereto. Words included in these interjections and word groups corresponding to interjections with honorifics added to them generally do not need to be included in the minutes of the meeting. Therefore, there is no need to associate attribute information 16 with words included in interjections and word groups corresponding to interjections with honorifics added to them, so they are treated as having no attributes when generating minutes. Ru. Note that hereinafter, "words included in interjections and word groups corresponding to interjections in which honorifics are added to interjections", etc. will be referred to as "words and word groups corresponding to interjections, etc.".

The element information 12b to 12d that is not attributeless and should be included in the minutes has some meaning, such as the subject of the action, the deadline, the action, and the conclusion. In other words, "first" in the element information 12b indicates the order of the action ("generate seminar materials" in the element information 12d), and "Mr. A by month x date" in the element information 12c indicates the subject of the action. and the deadline. Therefore, these element information 12 need to be included in the minutes of the meeting. Therefore, as shown above, element information 12 indicating the order or deadline of an action is associated with attribute information 16 called "deadline", for example.

The element information 12d "generate seminar materials" indicates an action that has been decided to be performed at the meeting, and therefore needs to be included in the minutes of the meeting. Therefore, for an action that has been decided to be performed, the attribute information "To-Do" is included as part of the "To-Do List" which means "list of things to do". 16 are associated. Note that an action that has been decided to be performed is generally called a "task" or the like. Further, attribute information 16 called "conclusion" is associated with element information 12d indicating the conclusion of the meeting.

Furthermore, as described above, whether each element information 12 was included in the speech of one of the speakers attending the conference is determined by the speaker identification based on the audio information corresponding to the conversion information including the element information 12. It can be identified by performing the following processing. For example, if the number of attendees at a meeting is m (m≧2), for example, by setting the number m of attendees at the meeting in advance in the voice recognition device, text information including each element information 12 can be spoken. This facilitates speaker recognition processing for recognizing speakers. As shown in FIG. 1, speaker recognition identifies each element information 12 and the speaker who uttered the sentence including the element information 12 (speakers A to C; m=3 in FIG. 1). can be mapped.

In this embodiment, text information is generated from audio information, and element information 12 is further generated from the text information via conversion information. Each of the generated element information 12 is associated with one or more attribute information 16 of "no attribute", "deadline", "To-Do", and "conclusion". As described above, it can be specified which utterance of a speaker such as a conference attendee includes the generated element information 12. Speaker information 18 indicating the identified speaker is associated with element information 12.

The user can modify and edit the element information 12 itself as appropriate. Furthermore, the user can modify the attribute information 16 and speaker information 18 that are erroneously associated with the element information 12, or edit the attribute information 16 and speaker information 18 themselves. Through these corrections and edits, the element information 12, attribute information 16, and speaker information 18 are correctly associated. Further, as indicated by dotted arrows, the element information 12 can be manually sorted by the user based on the attribute information 16, or the element information 12 can be automatically sorted based on the attribute information 16. Thereby, it becomes easy to generate the minutes of a meeting by the user's work, or the minutes can be automatically generated. The information indicating the minutes of the meeting explained above is made into the output information 14 shown in FIG. 1, and is displayed to the user via the display according to the form 10.

Furthermore, speaker information 18 indicating an attendee (speaker) who made a statement including element information 12 associated with an attribute of To-Do is, for example, presumed to be the person who directed the action indicated by the element information 12. can be included in the output information. Note that the attendee who made this comment may not be included in this comment and be the instructor of the action indicated by the element information 12 associated with the To-Do attribute. In such a case, the person who instructed the action indicated by the element information 12 associated with the attribute of To-Do can be changed to the correct person, for example, by correcting and editing the speaker information 18 by the user. Can be corrected by other speakers. Note that the "user" refers to a person who generates the minutes of a meeting, and may or may not be included in the speakers such as participants of the meeting.

2. Configuration of Terminal Device 100 Hereinafter, with reference to FIG. 2, the configuration of the terminal device 100 that executes the process of generating minutes from the audio of the meeting according to the present embodiment described with reference to FIG. 1 will be described. The terminal device 100 may be a general-purpose information processing device such as a mobile terminal device such as a smartphone, a terminal device such as a tablet computer, a notebook personal computer (PC), a desktop PC, a server device, or a large computer.

Note that the terminal device 100 does not need to include all of the components shown in FIG. 2, and some components of the terminal device 100 may be omitted. Furthermore, other components than those shown in FIG. 2 may be added to the terminal device 100. The terminal device 100 uses the components shown in FIG. 2 to function as a processing device that performs a process of generating minutes from the audio of the meeting according to the present embodiment described with reference to FIG.

However, the process of generating minutes from the audio of the meeting described above does not necessarily need to be executed in the terminal device 100, but rather in a server device connected to the terminal device 100 via a communication network (not shown) such as the Internet. (not shown) may be used as a processing device. In this case, for example, the terminal device 100 transmits voice information to the server device via the communication network, and the server device executes the voice information processing and speaker recognition process described with reference to FIG. and generates output information 14. The server device transmits this output information 14 to the terminal device 100 via the communication network.

As shown in FIG. 2, the terminal device 100 includes an output interface (output IF) 111, a processor 112, a memory 113, a communication interface (communication IF) 114, and an input interface (input IF) 116 that are interconnected via a bus. and a microphone 119.

The memory 113 includes RAM, ROM, nonvolatile memory (NVM), HDD (not shown), SSD (not shown), and the like. Communication interface 114 includes a communication processing circuit 115 and an antenna. Input interface 116 includes a mouse 117 and hard keys 118. These components of the terminal device 100 are electrically connected via a control line (not shown) and a bus, and mutually transmit and receive data and information.

The output interface 111 connects output devices such as speakers and displays (not shown) to the terminal device 100. Note that these output devices may be placed outside the terminal device 100 and connected via the output interface 111, or may be configured integrally with the terminal device 100 and connected to the output interface 111.

A display connected to the output interface 111 functions as a display section that reads out image information stored in the memory 113 and displays various types of information according to instructions from the processor 112. The display displays information for executing text information processing obtained from the audio information according to the embodiment. Note that the display is composed of, for example, a liquid crystal display or an organic EL display. The speaker connected to the output interface 111 functions as an audio output unit that outputs an audio signal obtained from audio data received by the terminal device 100.

The processor 112 includes one or more CPUs (microprocessors) or a combination of one or more CPUs and one or more GPUs specialized for image processing, and peripheral circuits thereof. The processor 112 functions as a control unit that controls other connected components based on various programs stored in the memory 113.

Specifically, the processor 112 reads from the memory 113 and executes an application program including predetermined instruction instructions for executing the process according to the embodiment and a program including predetermined instruction instructions for OS processing. . Further, the OS provides a function for the processor 112 to execute an application program.

In particular, the processor 112 reads from the memory 113 an application program including a predetermined command to accept input of audio information including the content of utterances by one or more speakers based on an input operation by the user and executes the application program. Further, the processor 112 generates text information indicating the contents of one or more speakers' statements based on the received audio information, and divides the generated text information into one or more pieces of element information 12 information. An application program including instructions is read from memory 113 and executed.

Further, the processor 112 associates at least one attribute information 16 with each divided element information 12, and based on the attribute information 16 associated with each element information 12, forms information having one or more items. In this step, an application program including a predetermined instruction command for generating output information 14 in which at least a portion of each element information 12 is associated with one or more items is read out from the memory 113 and executed.

Memory 113 functions as a storage section. Further, a storage medium, a database (not shown), etc., which are removable from the terminal device 100, can be connected to the memory 113. In the memory 113, the ROM stores a program including predetermined instructions for processing the OS and the like.

The RAM is a memory into which data necessary for processing is written and read while the processor 112 is processing application programs and OS programs stored in the ROM. Nonvolatile memory is memory that retains written data without power supply. The processor 112 writes data obtained by executing the program into the nonvolatile memory, and reads written data.

In particular, the memory 113 stores an application program including a predetermined instruction command that accepts input of audio information including the contents of one or more speakers' utterances based on an input operation by a user. The memory 113 also generates text information indicating the contents of one or more speakers' statements based on the received audio information, and divides the generated text information into one or more pieces of element information 12 information. Stores an application program including instructions. The memory 113 also associates at least one attribute information 16 with each divided element information 12, and stores form information having one or more items based on the attribute information 16 associated with each element information 12. At least part of each element information 12 stores an application program including a predetermined instruction command for generating output information 14 associated with one of the one or more items.

The communication interface 114 connects a communication network (not shown) and the terminal device 100 via a communication processing circuit 115 and an antenna, and exchanges information and data with other devices (not shown) connected to the communication network. It functions as a communication unit that sends and receives information. The communication processing circuit 115 performs communication processing for communicating information via an antenna between the communication network and the terminal device 100 using a wideband or narrowband wireless communication method. Note that the wideband wireless communication method is, for example, the LTE method, and the narrowband wireless communication method is, for example, IEEE802.11 and Bluetooth (registered trademark). Further, the communication processing circuit 115 may perform processing for wired communication instead of or in addition to wireless communication.

The input interface 116 is connected to input devices such as a mouse 117 and hard keys 118 through wired or wireless communication, and functions as an input unit that receives user operations and inputs various information. Examples of the input interface 116 include a serial port, parallel port, USB, and the like. Furthermore, in the case where the mouse 117 is connected via wireless communication (for example, Bluetooth (registered trademark)), it is also possible to have a component with a wireless communication function function as the input interface 116 and the communication interface 114. be.

The mouse 117 includes a sensor that detects its movement, a left button, a right button, and the like. The mouse 117 detects a user's operation to move a mouse pointer displayed on the display. Further, the mouse 117 detects a user's click operation using left and right buttons (not shown) on an icon or the like displayed on the display.

For example, the user can use the functions provided by the OS using the mouse 117 to select the element information 12 displayed on the display by operating (clicking) the left button. The user can also move the selected element information 12 on the display screen by moving the mouse 117 while holding down the left button, and move it to a desired position by stopping the left button. Such an operation by the user using the mouse 117 is also generally referred to as drag and drop. The mouse 117 accepts the user's operations as described above and outputs them to the processor 112 via the input interface 116. Note that the mouse 117 may be replaced by a touch pad or the like when the terminal device 100 is a notebook PC or the like.

Hard keys 118 include mechanical switches, accept operations on terminal device 100 by a user, and output them to processor 112 via input interface 116 . Note that the terminal device 100 and the hard key 118 may be configured integrally or separately. When the terminal device 100 and the hard keys 118 are configured separately, the terminal device 100, the mouse 117, and the hard keys 118 are connected by wireless communication or wired communication.

The microphone 119 receives audio such as statements made by one or more speakers at a conference, etc., converts it into an analog audio signal, further converts this audio signal into digital audio information, and outputs it to the processor 112. . However, the terminal device 100 does not need to obtain audio information using the microphone 119. For example, when the terminal device 100 is being used by a user participating in an online meeting, the terminal device 100 can process audio information being transmitted and received with other terminal devices in the online meeting.

3. Information used for processing in the terminal device 100 Information used for processing audio information in the terminal device 100 shown in FIG. 2 will be described below with reference to FIGS. 3A to 3C. FIG. 3A is a diagram showing a form information table. Terminal device 100 uses the form information table shown in FIG. 3A to generate output information 14 according to the output format desired by the user.

The form information table includes form identification information (form ID) A, user identification information (user ID) B, form information C, and number of speakers information D in association with each other. Note that the user identification information B is identification information that uniquely identifies a user who uses the terminal device 100 to generate meeting minutes. Note that, as described above, this user is a person who generates the minutes from the audio information of the conference, and may or may not be a speaker who speaks at the conference.

Form information C is generated by a user and shows a form 10 for output and display in output information 14 of element information 12, attribute information 16, and speaker information 18 as shown in FIG. The number of speakers information D indicates the number of attendees of the meeting for which minutes are to be generated, that is, the number of people who were able to speak including the element information 12, and is set in advance by the user before the start of the meeting. Ru. Form identification information A uniquely identifies associated user identification information B, form information C, and number of speakers information D.

FIG. 3B is a diagram showing the audio information table. The terminal device 100 uses the audio information table to generate output information 14 from audio information. The audio information table corresponds to audio identification information (voice ID) E, user identification information B, audio information G, text information H, conversion information I, element information J (element information 12), and output information K (output information 14). Attach and include. Note that the user identification information B is the same as the user identification information B included in the form information table described with reference to FIG. 3A, and each row of the form information table and each row of the voice information table is the user identification information B. can be correlated by

The audio identification information E uniquely identifies the audio information G, text information H, conversion information I, element information J, and output information K associated with the audio identification information E. The audio information G is audio information that is input from the microphone 119 or the like and is subject to processing by the terminal device 100. The text information H is text information generated by performing voice identification on the associated voice information G. The conversion information I is conversion information generated by converting the associated text information H. The element information J is one or more pieces of element information 12 generated by dividing the associated conversion information I. Output information K is output information 14 generated from corresponding element information J according to form information C corresponding to user identification information B shown in FIG. 3A.

FIG. 3C is a diagram showing an element information table. As shown in FIG. 3C, the element information table includes element information J (element information 12), flag information M indicating attribute information 16, and speaker information N indicating speaker information 18 in association with each other. The element information J is the same as the element information J of the audio information table described with reference to FIG. 3B, and each row of the audio information table and each row of the element information table are correlated by the element information J.

Flag information M is information indicating attribute information 16 of each of one or more pieces of element information J (element information 12) generated from one or more pieces of conversion information included in text information H. As explained with reference to FIG. 1, the flag information M is selected from among "no attribute", "deadline", "To-Do", and "conclusion" of the attribute information 16 of each element information J (element information 12). Indicates one or more of the following. Further, the speaker information N indicates the participants of the conference who have uttered the conversion information including the element information J, as exemplified as speakers A to C in FIG.

4. Information Obtained by Processing Audio Information by Terminal Device 100 Hereinafter, information obtained by processing audio information by terminal device 100 will be described with reference to FIGS. 4A to 4D, using specific examples. FIG. 4A is a diagram illustrating text information H included in the audio information table shown in FIG. 3B.

The terminal device 100 performs voice recognition processing on the voice information G shown in FIG. 3B, and as shown in FIG. Please check by △ month □ day. Then, Mr. A and Mr. B will prepare the materials, so we will finish today.'' Text information H is generated. Note that this text information H is generated by simply performing voice identification processing on the voice information G, so it includes only text and does not include other delimiter information such as punctuation mark information.

FIG. 4B is a diagram illustrating conversion information included in the audio information table (FIG. 3B) generated from the text information shown in FIG. 4A. The terminal device 100 divides text information including a plurality of sentences into sentences, and deletes portions that should not be included in the minutes, such as honorifics added at the end of each of the divided text information. Furthermore, the terminal device 100 generates conversion information shown in FIG. 4B by adding delimiter information such as punctuation mark information to the end of the sentence. The content of the generated conversion information is, for example, "Well, first, Mr. A will generate seminar materials by XX month and date. Mr. B will check it by △ month and □ day. Then, with Mr. A. Mr. B will prepare the materials, so today is over.''

FIG. 4C is a diagram illustrating element information 12 (element information J) included in the audio information table (FIG. 3B) generated from the conversion information shown in FIG. 4B. The terminal device 100 performs a morphological analysis process, a functional element analysis process, a feature word analysis process, and a structuring process on the converted information divided as shown in FIG. 4B, thereby generating the element information 12 shown in FIG. 4C. generate. As described above with reference to FIG. 1, each element information 12 is a word or a group of words to which at least one attribute of "no attribute", "deadline", "To-Do", and "conclusion" is associated. . In this description, a specific example will be taken in which each element information 12 is associated with a flag indicating an attribute.

FIG. 4D is a diagram illustrating a UI image in which check boxes, element information 12 shown in FIG. 4C, flags F1 to F4 indicating attributes, and speaker information 18 indicating speakers A to C are associated with each other. . As shown in FIG. 4D, this UI image includes, from the left side, check boxes, element information 12, flags F1 to F4 (attribute information 16), and speaker information 18 (speakers A to C). Note that the number of speaker information items 18 is not limited to three, A to C, and increases or decreases depending on the number of attendees at the conference. The terminal device 100 associates each of the element information 12 shown in FIG. 4C with one of the flags F1 to F4 indicating an attribute estimated to be most appropriate, and displays it on the display.

In addition, in FIG. 4D, the terminal device 100 determines that "eh" in the element information 12 is a word corresponding to an interjection, etc., and adds a flag F1 indicating no attribute to the element information 12, and the speaker B A case is illustrated in which the speaker information 18 is associated with the speaker information 18 indicating . Further, FIG. 4D illustrates a case where the terminal device 100 associates "first" of the element information 12 with a flag F2 indicating an attribute of a deadline and speaker information 18 indicating speaker B. . In addition, in FIG. 4D, the terminal device 100 adds a flag F2 indicating a deadline attribute to "Mr. A by x month x date" of element information 12, and a flag F2 indicating a deadline attribute and a speaker name indicating speaker B. A case where information 18 is associated is illustrated. Further, in FIG. 4D, the terminal device 100 associates the element information 12 "generate seminar materials" with the flag F3 indicating the To-Do attribute and the speaker information 18 indicating the speaker B. A case is illustrated.

Further, in FIG. 4D, the terminal device 100 corresponds the flag F2 indicating the deadline attribute and the speaker information 18 indicating the speaker A to "Mr. B by △ month □ date" of the element information 12. An example is shown in which it is attached. Further, FIG. 4D illustrates a case where the terminal device 100 associates the flag F3 indicating the attribute of To-Do with "Check" in the element information 12, and the speaker information 18 indicating the speaker A. has been done.

Further, in FIG. 4D, the terminal device 100 determines that "Sorere" in the element information 12 is a word corresponding to an interjection, etc., and sets the flag F1 indicating no attribute and the speaker information 18 indicating the speaker C. An example of a case of association is shown. Further, in FIG. 4D, the terminal device 100 associates the element information 12 "Mr. A and Mr. B create materials" with the flag F4 indicating a conclusion and the speaker information 18 indicating the speaker C. A case is illustrated. Further, in FIG. 4D, the terminal device 100 determines that "That's it for today" is a word or word group corresponding to an interjection, etc., and indicates a flag F1 indicating no attribute and a speaker C. A case is illustrated in which the information is associated with speaker information 18.

As described above, the terminal device 100 includes a check box, element information 12, a flag (attribute information 16) that recommends association with the element information 12, and speaker information in the UI image shown in FIG. 4D. 18 are displayed in association with each other. The user can approve the recommended combination of element information 12, flag (attribute information 16), and speaker information 18 by checking (marking) the leftmost checkbox. Alternatively, the user can edit the element information 12, flags, and speaker information 18 as appropriate by performing editing operations on the UI image shown in FIG. 4D using the mouse 117, hard keys 118, and the like.

In other words, if there is a mistake in the element information 12, the user can correct the mistake by performing an operation on the UI image. Similarly, if there is a mistake in the flag (attribute information 16) associated with the element information 12, the user can modify the flag associated with the element information 12. Similarly, if there is a mistake in the speaker information 18 associated with the element information 12, the user can modify the speaker information 18 associated with the element information 12.

Note that when one piece of element information 12 is divided into two pieces of element information 12, the terminal device 100 replaces the one row that contained this element information 12 in FIG. 4D with the element information 12 after the division. Split into two lines each containing: Further, the terminal device 100 associates a flag and speaker information 18 with each of the two pieces of element information 12 after division, in accordance with the user's input operation. Alternatively, the terminal device 100 automatically associates the flag and the speaker information 18 with each of the two pieces of element information 12 after division.

Alternatively, two adjacent pieces of element information 12 in FIG. 4D may be merged into one piece of element information 12. In this case, the terminal device 100 converts the two rows that included these element information 12 in FIG. 4D into one row that includes one piece of element information 12 after division. Further, the terminal device 100 associates the flag and the speaker information 18 with the merged element information 12 in accordance with the user's input operation. Alternatively, the terminal device 100 automatically associates the flag and the speaker information 18 with the element information 12 after merging.

5. Processing of the terminal device 100 The processing of the terminal device 100 will be described below. First, with reference to FIGS. 5A and 5B, a process in which the user registers form information C indicating the output format of the output information 14 in the terminal device 100 will be described. FIG. 5A is a flowchart showing a process in which the user registers information indicating form information C of the output information 14 of the form information included in the form information table (FIG. 3A) in the terminal device 100.

In S100 shown in FIG. 5A, the processor 112 of the terminal device 100 (FIG. 2) accepts a user's input operation via the input interface 116, and determines whether or not the user identification information of the user has been received. If the processor 112 receives the user identification information (Y), it proceeds to the process of S102, and if it does not receive the user identification information (N), it remains in the process of S100.

In S102, the processor 112 of the terminal device 100 performs processing to authenticate the user who performed the input operation in S100.

In S104, the processor 112 determines whether the user who performed the input operation in S100 has been authenticated through the process in S102. If the user is authenticated (Y), the processor 112 proceeds to the process of S106, and if the user is not authenticated (N), the process ends.

In S106, the processor 112 accepts an input operation for inputting form information C or a selection operation for selecting one of a plurality of form information C by the user in response to the user's input operation via the input interface 116. Processor 112 accepts input form information C or selected form information C via input interface 116 . Note that the output format of the output information 14 may be received from another device connected to the communication network via the communication interface 114.

In S108, the processor 112 of the terminal device 100 updates and registers the form information C included in the form information table shown in FIG. 3A using the form information C of the output information 14 received in the process of S106. Furthermore, processor 112 stores the updated form information table in memory 113. As shown in FIG. 5B, the form information C includes the items and layout of the attribute information 16 included in the output information screen used when displaying the output information 14 shown in FIG. 1 on the display via the output interface 111. Define.

FIG. 5B is a diagram illustrating the form 10 indicated by the form information C of the output information 14 indicated by the form information. As shown in FIG. 5B, the form 10 of the output information 14 includes a portion displaying a conclusion item, a deadline item, and a to-do item. According to the form 10 indicated by the form information C, the element information 12 associated with the flag F4 indicating the attribute of the conclusion is displayed in the upper half of the output information 14 screen. Further, according to this form 10, element information 12 associated with the flag F2 indicating the deadline attribute is displayed on the left side of the lower half.

In addition, according to this form 10, on the right side of the lower half are element information 12 associated with the flag F3 indicating the attribute of To-Do, and speaker information 18 associated with this element information 12 (see FIG. 5B). Speakers A and B) are displayed. Note that, as described above, the speaker indicated by the speaker information 18 associated with the element information 12 is estimated to be the person who directed the element information 12 to which the To-Do attribute is attached. If this estimation is wrong, the user can correct such mistakes using the UI image described above with reference to FIG. 4D.

Next, with reference to FIGS. 6A to 6C and the like, processing from input of user identification information to generation of output information 14 by the terminal device 100 will be described. FIG. 6A is a flowchart showing audio information processing by the terminal device 100. As shown in FIG. 6A, at S120, processor 112 of terminal device 100 (FIG. 2) accepts user identification information from one or more users via input interface 116. The processor 112 updates the form information table and the voice information table (FIGS. 3A and 3B) with the accepted user identification information B, registers the user identification information B, and stores it in the memory 113.

At S122, processor 112 accepts audio information from microphone 119. Alternatively, processor 112 receives audio information from other devices connected to the communication network via communication interface 114. The processor 112 updates the audio information table with this audio information, registers the audio information, and stores it in the memory 113.

In S124, the processor 112 reads the audio information stored in the memory 113, performs audio recognition processing on the read audio information, and generates the text information illustrated in FIG. 4A. Processor 112 updates the audio information table with the generated text information, registers the text information, and stores it in memory 113.

In S126, the processor 112 reads text information from the memory 113 and performs morphological analysis processing on the read text information. Through this morphological analysis process, the processor 112 divides the range of input text information into morphemes and morpheme groups including a plurality of morphemes.

A morpheme is a term in linguistics, and is the smallest unit of meaningful expressive elements, and is a group of phonemes extracted in a language by dividing them to the point where they no longer make sense if broken down any further. Note that in languages such as English where each word is written separately, one word is almost always one morpheme, although there are exceptions. That is, the text information including a plurality of morphemes is very general text information, such as text information generated by a user using a word processor and text information provided by a web server.

Note that a specific example is a case where the terminal device 100 generates minutes from Japanese text information. For example, if the text information includes the character string "text information", the morphemes included in this text information are "text", "information", and "ni". Note that in this embodiment, phonemes that include multiple morphemes, such as "text information" that includes multiple morphemes "text" and "information", and "information ni" that includes multiple morphemes "information" and "ni" A group of morphemes is described as a "morpheme group."

In S128, the processor 112 performs functional element analysis processing for relationship analysis processing on each of the morphemes and morpheme groups obtained through the processing in S126, and specifies the functions of each of the morphemes and morpheme groups.

In S130, the processor 112 uses the feature word dictionary to perform feature word analysis processing for relationship analysis processing on each of the morphemes and morpheme groups obtained through the processing in S126.

In S132, the processor 112 performs relationship analysis processing based on the results of the functional element analysis processing and the characteristic word analysis processing in S128 and S130, and performs relationship analysis processing between morphemes, between morphemes and morpheme groups, and between morphemes. Identify relationships between groups. Note that "between morphemes", "between a morpheme and a morpheme group", and "between morphemes" are collectively referred to as "between a morpheme and a morpheme group."

In S134, the processor 112 generates the conversion information shown in FIG. 4B based on the result of the relationship analysis process. Specifically, the processor 112 generates converted information by deleting honorific words etc. from the end of the converted information and inserting delimiter information such as punctuation mark information between the converted information based on the result of the relationship analysis process. do. The processor 112 updates the audio information table with the generated conversion information, registers the conversion information, and stores it in the memory 113.

In S136, the processor 112 processes the conversion information generated in the process of S134, and divides it into a plurality of element information 12 each having a specific meaning, as shown in FIG. 4C. The processor 112 updates the audio information table with the element information 12 generated by such division, registers the element information 12, and stores it in the memory 113.

In S138, the processor 112 performs speaker identification processing on the audio information corresponding to the conversion information including the element information 12 generated in the process of S136. As a result of this processing, processor 112 identifies the speaker who uttered the transformation information including element information 12. The processor 112 also generates speaker information 18 indicating the identified speaker. Furthermore, the processor 112 updates the speaker information N included in the element information table shown in FIG. 3C using the generated speaker information 18. Further, the processor 112 stores the element information J and flag information M12 corresponding to the speaker information N in the memory 113 in association with each other.

In S140, the processor 112 performs processing based on the results of the relationship analysis processing. Through this process, the processor 112 associates the element information 12 with one or more of the flags F1 to F4 (attribute information 16). Furthermore, the processor 112 further associates the associated element information 12 and one or more of the flags F1 to F4 with the check box and the speaker information 18 generated by the process of S138. The processor 112 displays check boxes associated with the element information 12, element information 12, flags F1 to F4, and speaker information 18 on the UI image shown in FIG. 4D.

By displaying this UI image, the processor 112 recommends the attribute information 16 and speaker information 18 associated with each element information 12 to the user. As described above, the flag F1 indicates no attribute, the flag F2 indicates the deadline attribute, the flag F3 indicates the To-Do attribute, and the flag F4 indicates the conclusion attribute. As described above with reference to FIG. 4D, the user performs editing and modification operations as appropriate on the element information 12, flag, and speaker information 18 associated in the process of S138. The processor 112 accepts editing and modification operations by the user, and reflects them on the contents of the element information 12, attribute information 16, and speaker information 18, and the correspondence of these information. When the user performs an operation on the button labeled "Confirm" in the UI image, editing and modification such as the association of element information 12, attribute information 16, and speaker information 18 is completed. Note that the user may perform an operation to further add information necessary for the minutes, such as information such as the agenda of the meeting and attendees, to the UI image shown in FIG. 4D. Processor 112 accepts such operations via input interface 116 , adds them to output information 14 , and displays them on a display according to form 10 via output interface 111 .

FIG. 6B is a diagram illustrating a list of element information 12 associated with flags indicating attributes. In S142, the processor 112 associates the element information 12 edited and corrected by the processing in S140, one or more of the attribute information 16 excluding non-attribute, and the speaker information 18. Further, the processor 112 displays the information associated in this way in the list format shown in FIG. 6B on the display via the input interface 116.

In S142, the processor 112 generates a list that includes the element information 12, attribute information 16, and speaker information 18 that were edited and modified in the process of S140 in association with each other, as shown in FIG. 6B. Further, processor 112 displays the generated list on a display via output interface 111. The user can perform operations on the displayed list. That is, the user approves the attribute information 16 associated with each element information 12 by the processor 112 by checking the check box 20 using the mouse 117 or the like. Alternatively, the user operates the attribute information 16 using the mouse 117 or the like to change the attribute information 16 to the desired attribute information 16. Alternatively, the user performs an operation on the speaker information 18 to change it to desired speaker information 18.

These operations by the user in the process of S142 include changing or modifying the association between the element information 12, one or more attribute information 16 of deadline, to-do, and conclusion attributes, and speaker information 18. This is an editing operation. The processor 112 accepts an editing operation by the user via the input interface 116, and changes the association between the element information 12 and one or more of the attributes of deadline, to-do, and conclusion in accordance with the editing operation by the user. do. However, since the element information 12, the attribute information 16 and the speaker information 18 associated with the element information 12 have already been edited in the process of S140, the editing operation in S142 can be omitted.

In S144, when the user checks the list shown in FIG. 6B and performs an operation on a button included in the list and marked as OK, the processor 112 accepts this operation via the input interface 116. In response to this operation, the processor 112 determines the association between the element information 12 edited in the process of S140 and the attributes and speakers.

FIG. 6C is a diagram illustrating a UI image used to move the element information 12 and the like to a box included in the form 10 of the output information 14 and associated with the attribute information 16. In S146, the processor 112 of the terminal device 100 displays the attribute information 16 other than the check box 20, the speaker information 18, and the element information 12 as shown in FIG. 6B. Further, processor 112 reads form information C corresponding to user identification information B of the user from the form information table shown in FIG. 3A. Further, the processor 112 displays the output information 14 according to the form 10 indicated by the read form information C. Processor 112 combines information other than the checkboxes shown in FIG. 6B with output information 14. As a result, the processor 112 includes attribute information 16 indicating an attribute other than no attribute, element information 12 associated with the attribute and its speaker information 18, and output information 14, as shown in FIG. 6C. Display UI image.

In S148, the user selects a box that is included in the output information 14 of the UI image shown in FIG. The attribute information 16 is referred to. Furthermore, the user performs an operation on this UI image to move the element information 12 associated with the attribute information 16 into a box in the output information 14 associated with the same attribute information 16. This operation may be an operation of dragging and dropping the element information 12 and the like using the mouse 117.

Note that if the element information 12 is associated with the speaker information 18, the user can select the element information 12 and the speaker information 18 associated with this element information 12 in the output information 14. Perform the operation to move it to the box. Processor 112 of terminal device 100 accepts user operations via input interface 116 . The processor 112 moves and displays the element information 12 or the element information 12 and the speaker information 18 into each box of the output information 14 according to the accepted user's operation.

At S150, FIG. 1 shows that once all the element information 12 has been moved to any of the boxes included in the output information 14, the processor 112 generates the minutes of the meeting. Furthermore, the processor 112 generates information on the minutes of the meeting and displays it on a display via the output interface 111 as the output information 14 shown in FIG. (not shown).

Note that although a case has been described here in which the terminal device 100 performs all of S120 to S150 shown in FIG. 6A, it is not necessary for the terminal device 100 to perform all of these processes. For example, the terminal device 100 may transmit the text information generated in the process of S124 to a server device (not shown), and the server device may execute the processes of S126 to S132. In this case, before the process in S134, the server device receives the results of the processes in S126 to S132 to the terminal device 100, and the terminal device 100 receives these processes and performs the processes in S134 to S150. Become.

In addition, the case where the user manually moves the element information 12 using the mouse 117 into a box included in the form 10 of the output information 14 and attached with each attribute information 16 has been described. On the other hand, by attaching conclusion, deadline, and to-do attribute information 16 to each box included in the output information 14, the processor 112 can automatically perform such movement.

In other words, by comparing the attribute attached to the element information 12 and the attribute information 16 attached to the form 10 of the output information 14, the processor 112 automatically performs such movement. can be done. Specifically, the processor 112 automatically moves the element information 12 to which the conclusion attribute information 16 is attached into the box to which the conclusion attribute information 16 is attached in the form 10 of the output information 14. It will be done. Further, the processor 112 automatically moves the element information 12 to which the deadline attribute information 16 is attached to the box to which the deadline attribute information 16 is attached in the form 10 of the output information 14 . Further, the processor 112 automatically converts the element information 12 to which the To-Do attribute information 16 and speaker information 18 are attached into the form 10 of the output information 14. be moved into the box.

According to the terminal device 100 described above, it is possible to generate element information 12 from audio information such as a conference, and to appropriately associate the generated element information 12 with its attributes. Therefore, appropriate minutes and various records and materials can be generated from the audio information. Further, since the attributes associated with the element information 12 are automatically selected and recommended to the user, the user's effort to generate minutes and the like from audio information is greatly reduced. Furthermore, the terminal device 100 is useful for automatically generating minutes and the like from the audio of offline meetings and online meetings in which many people participate.

6. Modifications, etc.Although a case has been described above with reference to FIG. 6A in which an attribute such as "deadline" is associated with the element information 12 via a flag and edited, the attribute is not necessarily associated with the element information 12 via the flag. It is not necessary to associate the element information 12 with the element information 12. Furthermore, with reference to FIG. 6A, a case has been described in which the element information 12 is distributed to items included in the form 10 of the output information 14 according to the user's operation. However, S144 and S146 shown in FIG. 6A are not essential. That is, at the time when the user establishes the association between the element information 12 and the attributes, the processor 112 may automatically allocate each piece of element information 12 to each item included in the form 10 of the output information 14.

Furthermore, the audio information processing method described above can be applied to the generation of various records and materials in addition to the generation of meeting minutes. In addition, the attribute information 16 associated with each element information 12 may include, for example, "no attribute," "deadline," "To-Do," and "conclusion," depending on the purpose of the audio information processing method. Other attribute information 16 such as "subject" may be included. Alternatively, the attribute information 16 may not include all of "no attribute", "deadline", "To-Do", and "conclusion". Further, the display mode of various information shown in FIG. 1 and the like is an example, and the display mode can be changed as appropriate depending on the user's preference, the purpose of the terminal device 100, and the like.

Further, although the case where four types of attribute information 16 such as non-attribute, conclusion, deadline, and To-Do are associated with the element information 12 has been exemplified above, the types of attribute information 16 are not limited to these four types, and can be changed as appropriate. , may be increased or decreased. Further, the case where the attribute information 16 is associated with all of the element information 12 has been exemplified above. On the other hand, for example, only two types of attribute information 16, non-attribute and non-attribute, are associated with the element information 12, and the user can add any type of attribute to the element information 12 that is associated with the attribute information 16 other than the non-attribute. Information 16 may be associated. Alternatively, the user may arbitrarily associate any type of attribute information 16 such as conclusion, deadline, and to-do with element information 12 associated with attribute information 16 other than non-attribute.

It can be realized not only by the devices explicitly described in the embodiments, but also by software, hardware, or a combination thereof. Specifically, the processes and procedures described in the embodiments are realized by implementing logic corresponding to the processes in a medium such as an integrated circuit, volatile memory, nonvolatile memory, magnetic disk, or optical storage. sell. Furthermore, the processes and procedures described in the embodiments can be implemented as computer programs and can be executed by various computers including terminal devices and server devices.

In embodiments, processes and procedures described as being performed by a single device, software, component, and/or module may be performed by multiple devices, software, components, and/or modules. can be executed by Furthermore, in the embodiments, various types of information described as being stored in a single memory and storage device may be stored in multiple memories included in a single device or multiple memories distributed and arranged in multiple devices. can be distributed and stored. Furthermore, the software and hardware described in the embodiments can be implemented by integrating them into fewer components or by decomposing them into more components.

Although one embodiment has been described above, this embodiment is presented as an example and is not intended to limit the scope of the invention. These novel embodiments may be implemented in various other forms, and various omissions, substitutions, and changes may be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and gist of the embodiments, as well as within the scope of the invention described in the claims and its equivalents.

10 form, 12 element information, 14 output information, 16 attribute information, 18 speaker information, 20 check box, 100 terminal device, 111 output interface, 112 processor, 114 communication interface, 116 input interface, 117 mouse, 118 hard key, 119 Mike

Claims (12)

A processing device comprising at least one processor,
The at least one processor includes:
Accepts input of audio information including the contents of one or more speakers' utterances based on an input operation by a user,
generating text information indicating the content of the utterances of the one or more speakers based on the received voice information;
Divide the generated text information into one or more element information,
Associate at least one attribute information with each divided element information,
Based on the attribute information associated with each element information, in form information having one or more items, at least a part of each element information is associated with any one of the one or more items. generate output information,
configured to perform processing for
Processing equipment. The at least one processor is configured to further associate speaker information indicating one of the one or more speakers who made the utterance corresponding to the text information with the element information. The processing device according to claim 1. 3. The processing device of claim 2, wherein the at least one processor is configured to generate the speaker information based on the audio information and a preset number of speakers. The processing device according to any one of claims 1 to 3, wherein the user is included in the one or more speakers. The at least one processor is configured to perform processing for generating conversion information in which delimiter information indicating a semantic delimiter of the text information is inserted based on the text information. 4. The processing device according to any one of 4. The at least one processor executes processing for dividing the text information into the element information, which is a group of meaningful words or word groups including a plurality of words, based on the generated conversion information. 6. A processing device according to claim 5, configured to. The processing device according to claim 5 or 6, wherein the delimiter information is punctuation mark information indicating a punctuation mark. The processing device according to any one of claims 1 to 7, wherein the element information is editable based on the user's operation. The attribute information is information indicating at least one of a conclusion indicated by the text information, a task for the speaker or another speaker different from the speaker indicated by the text information, and a deadline for the task. The processing device according to any one of items 1 to 8. The processing device according to any one of claims 1 to 9, wherein the association of the attribute information is determined based on the user's operation on the attribute information associated with the element information. In a computer including at least one processor, the method is performed by the at least one processor executing a predetermined instruction command, the method comprising:
receiving input of audio information including utterances from one or more speakers based on an input operation by a user;
generating text information indicating the content of the one or more speakers' utterances based on the received voice information;
dividing the generated text information into one or more element information;
Associate at least one attribute information with each divided element information,
Based on the attribute information associated with each element information, in form information having one or more items, at least a part of each element information is associated with any one of the one or more items. generating output information based on the
processing methods including; a computer including at least one processor;
Based on the input operation by the user, one receives the input of voice information including the content of utterances from a plurality of speakers;
Generating text information indicating the content of the one or more speakers' statements based on the received voice information,
Divide the generated text information into one or more element information,
Associate at least one attribute information with each divided element information,
Based on the attribute information associated with each element information, in form information having one or more items, at least a part of each element information is associated with any one of the one or more items. generate output information,
act as a processor configured to perform processing;
Processing program.

Images