JP5414865B1 - REPRODUCTION DATA GENERATION DEVICE AND REPRODUCTION DATA GENERATION METHOD - Google Patents

Abstract

In a limited time, reproduction data in which voice messages are selected and arranged so as to be easily understood by a listener is generated.
A reproduction data generating apparatus according to an aspect of the present invention includes a weighting unit that sets a weight for a plurality of voice messages according to attribute information thereof, and a reproduction time based on the weight of the voice message. Scheduling means for selecting one or more voice messages from the voice messages so that the total is within the time limit, and determining a playback order of the selected voice messages based on the attribute information; Reproduction data generating means for generating reproduction data in which voice messages are arranged and combined, and transmission means for transmitting the reproduction data or link information to the reproduction data to a user terminal.
[Selection] Figure 1

Description

  The present embodiment relates to a reproduction data generation apparatus and a reproduction data generation method.

  In medical settings where multiple staff members of multiple occupations work in a distributed and asynchronous manner, especially in the field of regional medicine, it is necessary to closely share information on observation results, awareness, and communication items among patients. .

  Staff generally share information using electronic chart system, telephone, fax, etc., but information sharing systems using voice messages are known (hereinafter referred to as voice tweets system). Call).

  In a voice tweet system, when a staff member registers a voice message using a smartphone, the server recognizes the voice and extracts keywords included in the voice. At the same time, the subject patient ID of the message, the staff ID of the speaker, the information such as the time of speech, the location of the speech, the estimation work, etc., the importance and type of the message (hereinafter referred to as tag) are automatically attached to the voice message or the user (ie ) Is attached and distributed to the necessary staff. Such information consisting of a voice message and accompanying information is hereinafter referred to as a voice tweet.

  The staff can grasp the recent situation of the patient by organizing and viewing the received voice tweet or the voice tweet stored in the server in order of time for each patient on a portable terminal or a personal computer. At that time, the voice recognition result and the keyword can be browsed on the screen, but it is more convenient to listen to the voice itself when moving on foot or by car. In addition, the user may hear the voice while viewing the voice recognition result, keywords, and other accompanying information on the screen.

JP 2011-113490 JP JP 2011-61263 A

  However, when listening to a series of accumulated tweets, there is no guarantee that playback will be completed within a limited time such as travel time. Also, until the staff themselves hear the voice tweets, it is not clear what order to play to make it easier to understand.

  As a prior art, a technique is known in which a speech recognition result of a conversation at a call center is analyzed according to a structure based on a predetermined template to create a summary. However, this technique has a problem that the time limit related to reproduction cannot be taken into consideration, and a problem that rearrangement cannot be performed for easy understanding.

  An object of one aspect of the present invention is to generate reproduction data in which a voice message is selected and arranged so as to be easily understood by a listener within a limited time.

  A reproduction data generation apparatus as one aspect of the present invention includes weighting means, scheduling means, reproduction data creation means, and transmission means.

  The weighting means sets a weight according to attribute information of the voice message for a plurality of voice messages.

  The scheduling means selects one or more voice messages from the voice messages based on the weight of the voice messages so that the total playback time is within the time limit, and plays the selected voice message. The order is determined based on the attribute information of the selected voice message.

  The reproduction data creating means generates reproduction data by arranging and combining the voice messages according to the reproduction order.

  The transmission means transmits the reproduction data or link information to the reproduction data to a user terminal.

1 is a block diagram of a tweet playback apparatus as a playback data generation apparatus according to a first embodiment. FIG. It is a figure which shows the creation flow of the reproduction data with respect to the voice tweet reproduction request received from the user terminal. It is a figure which shows an example of the calculation flow of the weight to each tweet. FIG. 10 is a diagram showing an operation flow according to a sixth embodiment. It is a figure for demonstrating the determination of the reproduction | regeneration order using a lattice structure.

  Embodiments of the present invention will be described below with reference to the drawings.

(First embodiment)
FIG. 1 is a block diagram of a tweet playback apparatus as a playback data generation apparatus according to the first embodiment of the present invention.
In the present embodiment, a case will be described in which the reproduction data generation apparatus is applied to a medical site where a plurality of staff members of a plurality of occupations carry out work in a distributed and asynchronous manner, particularly a site such as regional medical care. However, the present embodiment is not limited to a medical site, and can be applied to a wide variety of sites as long as a plurality of staff members perform operations in a distributed and asynchronous manner such as equipment maintenance sites such as elevators.

  The tweet reproduction apparatus 101 is a server that can communicate with a staff terminal (user terminal) 102 via a network 103. The user terminal 101 is a mobile terminal that can be carried by a staff member, and examples thereof include a mobile terminal, a notebook PC, a smartphone, and the like. However, the user terminal 101 is not limited to these forms, and may be a stationary terminal such as a desktop PC.

  The tweet playback device 101 includes a communication device 11, a CPU 12, and a storage device 13. The storage device 13 is an arbitrary device capable of storing or reading data, and is, for example, a memory device, a hard disk device, or a read drive for reading data from a recording medium such as a CD-ROM or a USB memory. The storage device 13 may be a combination of two or more of these. In addition to the communication device 11, CPU 12, and storage device 13, the tweet playback device 101 expands programs executed by the CPU 12, and functions as a temporary work area for the CPU 12, and between components. Although there are buses to be connected, these illustrations are omitted.

  The communication device 11 communicates with the user terminal 102 via the network 103. The network 103 may be a wireless network, a wired network, or a combination of these. The network 103 may include the Internet.

  The user terminal 102 includes a playback request transmission unit that transmits a playback request for a voice tweet to the tweet playback device 101.

  The voice tweet playback request includes, for example, the following items. Some may be empty.

・ User ID
・ Target patient ID
・ Playback time limit

  The communication device 11 includes a reproduction request receiving unit 1 and a reproduction data transmitting unit 7.

  The reproduction request receiving unit 1 receives a reproduction request from the user terminal 102. The reproduction request receiving unit 1 passes the received reproduction request to the reproduction tweet candidate extracting unit 3 of the CPU 12.

  The CPU 12 includes reproduction tweet candidate extraction means 3, weighting means 4, reproduction tweet scheduling means 5, and reproduction data creation means 6. These means are realized by causing the CPU 12 to execute a program. The program is stored in the storage device 13 or the like, read into a nonvolatile memory such as a RAM, and executed by the CPU 12.

  The storage device 13 includes a tweet holding means 2. The tweet holding means 2 is means for holding a voice tweet registered by the user. Specifically, a relational database or an XML database can be used. The tweet holding means 2 may be provided not in the server but in a storage device externally attached to the server, or may be arranged on a network accessible by the server.

  Each voice tweet includes the following accompanying information (attribute information). Some may be empty. For example, when the topic patient ID is empty, the tweet is not a tweet about any patient, but a tweet such as an administrative communication between staff members. The following information is an example, and additional information may be included. The voice data itself may be included instead of the URL of the voice data (voice message).

・ Tweet ID
・ URL of voice data of tweets (or file path of voice data)
・ Speech time of voice data (second or millisecond)
・ Speaker ID
・ Utterance date and time ・ Multiple keywords in utterance
・ Topic patient ID
・ Reply destination tweet ID
・ Number of times (the number of times a registered tweeter viewed the tweet)
-Tag (depending on system settings such as importance or tweet type)
・ Speech location (latitude and longitude, nearest Bluetooth device ID, etc.)
・ Actions when speaking (walking, meeting, working, etc.) and specific work contents

  Based on the reproduction request received from the reproduction request receiving unit 1, the reproduction tweet candidate extracting unit 3 inquires the tweet holding unit 2 about a tweet as a reproduction candidate. Specifically, a query for extracting a tweet as a reproduction candidate is issued, and the issued query is output to the tweet holding means 2. As a result, the reproduction tweet candidate extraction unit 3 acquires a tweet as a reproduction candidate from the tweet holding unit 2.

  The weighting means 4 assigns a weight to the tweet extracted by the reproduction tweet candidate extracting means 3. The weight is a real number or an integer. The weight is determined based on, for example, the accompanying information (attribute information) of the tweet.

  The reproduction tweet scheduling means 5 selects a tweet candidate from among tweet candidates given weights so that the total reproduction time is within the time limit. For example, a set of tweet candidates is selected so that the total reproduction time is within the time limit and the total weight is maximized. Further, the reproduction tweet scheduling means 5 determines the reproduction order of these tweets based on the attribute information of the selected tweets. The reproduction tweet scheduling means 5 arranges the reproduction target tweets according to the determined order.

  The reproduction data creation means 6 creates reproduction data based on the tweet string arranged by the reproduction tweet scheduling means 5. The reproduction data includes voice data (such as wav) and may further include text data (such as when voice is synthesized at a terminal). The voice data is downloaded from a URL included in the tweet as necessary. Text data is created based on attribute information, for example, and is used when speech synthesis or display is performed on a user terminal. The audio data may include the attribute information converted into audio data. The reproduction data may not include the attribute information, or may be included as text so that the user can display or synthesize the attribute information on the terminal during reproduction of tweets.

  The reproduction data transmission means 7 transmits the reproduction data created by the reproduction data creation means 6 or link information (URL, etc.) thereof to the user terminal that sent the reproduction request. When transmitting link information, the tweet playback device stores playback data at a position corresponding to the link information. The storage destination may be in the tweet playback device or a server on the network accessible from the user terminal.

(Example of how to create playback data)
FIG. 2 shows a flow of creating reproduction data in response to a voice tweet reproduction request from the user terminal.

Step S11: Reproduction Request Reception The reproduction request receiving means 1 receives a reproduction request from the user terminal and passes the reproduction request to the reproduction tweet candidate extracting means 3. As described above, the playback request includes the user ID of the user, the target patient ID of the tweet to be played back, and the playback time limit on the user terminal.

  Here, in the reproduction request, the target patient ID may be empty. This is the case when you want to replay a tweet about a general contact between staff, not a tweet about any patient.

  In some cases, the time limit is empty. In this case, the time limit can be calculated on the user terminal or the server by a method described later, and the calculated value can be used.

Step S12: Issuance of replay tweet candidate query The replay tweet candidate extraction means 3 issues a query for extracting a replay candidate target tweet from the tweet holding means 2 based on the replay request, and sends it to the tweet holding means 2 Output the query. The query includes a condition for extracting tweets.

Step S13: Acquisition of replayed tweet candidates The tweet holding means 2 extracts the tweet corresponding to the query from the replayed tweet candidate extracting means 3, and returns the extraction result to the replayed tweet candidate extracting means 3. The extraction result includes an array of the extracted tweet data objects. The structure of the data object is as follows. The voice data itself may be included instead of the voice data URL.

[Member of the tweet data object]
・ Tweet ID
・ URL of the voice data of the tweet (or the file path of the voice data)
・ Voice data utterance time (seconds or milliseconds)
・ Speaker ID
・ Speech date ・ Keywords in the utterance (text, multiple)
・ Topic patient ID
・ Return destination tweet ID
・ Good times ・ Tags (multiple)
・ Business at the time of utterance ・ Speaking place ・ Weight (default value is set at the time of step S13)
・ Reader ID

Step S14: Giving weights to reproduction tweet candidates Weights are calculated and assigned to the extracted reproduction tweet candidates (data objects).

FIG. 3 shows an example of a calculation flow of the weight w _i for each tweet i (1 to N) regarding playback tweet candidates (N).

In FIG. 3, the index k _i of the tweet i is calculated in step S21, the time decreasing coefficient α _i is calculated in the subsequent step S22, and the weight w _i based on these is calculated in the subsequent step S23.

・ K _i = k1 * number of likes + k2 * number of replies + k3 * speaker weight + k4 * Σ tag weight + k5 * Σ keyword distance ・ α _i = 1 / (tt _i ), t: current time, t _i : tweet i Speech time ・ w _i = α _i × k _i

K1 to k5 in the calculation formula of the index k _i are numerical values.

  As described above, the number of likes is the number of times the user who viewed the tweet has evaluated the tweet.

  The number of replies is the number of tweets that have tweet i as a reply destination.

  The speaker weight is a numerical value determined by a tweeter or a job type of the speaker.

  The job title of the speaker can be acquired from the user master of the server.

  The tag weight is a numerical value determined by the type of tag (indicating the type depending on the content of the tweet. Doctor's instruction, nurse's instruction reception, reply, question, etc.). The tag corresponding to the important content has a larger tag weight.

  The keyword distance is a numerical value indicating how close the keyword included in the utterance of the tweet is from the center of the distribution of keywords included in the latest tweet. That is, the weight of tweets related to the latest symptom is increased. The keyword distance can be calculated using a keyword distance function. For example, prepare a vector in which words are arranged in lexicographical order, and if the tweet contains the same word, set the element at the corresponding position to 1 and set it to 0 if it does not contain the same number of elements as the above vector. Generate a word vector. Compares the value at the same position with other tweet word vectors. If the value is the same, the value is 0. If the value is different, the value is 1. This can be used as a keyword distance.

The time-decreasing coefficient α _i is an example, and any other form may be used as long as it is a function that decreases monotonically as time goes back from the present time.

Step S15: Replay Tweet Scheduling Replay tweet scheduling means 5 selects and selects a replay target tweet from the tweet candidates so that the total replay time is within the time limit and the total weight is equal to or greater than the threshold. Align. Details will be described later.

Step S16: Reproduction Data Creation The reproduction data creation means 6 creates reproduction data based on the reproduction target tweets selected and arranged by the reproduction tweet scheduling means 5. Here, the reproduction data is audio data (wav format or the like) obtained by combining the tweet audio data and the synthesized audio of the accompanying information (attribute information) included in the alignment result in the order of reproduction. The accompanying information portion may be text data on the reproduction data, and may be synthesized on the user terminal.

Step S17: Reproduction Data Transmission The reproduction data transmission unit 7 transmits the reproduction data created by the reproduction tweet scheduling unit 5 to the user terminal that has transmitted the reproduction request. As another method, link information to reproduction data may be transmitted to the user terminal. In this case, the user downloads the reproduction data by accessing the link information. Further, the server stores the reproduction data at the link destination. The link destination may be in the server or another device on the network.

(Example of playback tweet scheduling)
An example of selecting and arranging the tweets to be reproduced by the reproduction tweet scheduling means 5 will be described.

  Replay tweet scheduling can be formulated as an optimization problem as follows.

・ Max. Σ (weight of selected tweet)
・ St selected / sorted tweet playback time total <time limit

An example of the algorithm for solving the above optimization problem is shown.

[Replay target tweet selection and alignment algorithm]
(Step 1) The tweet candidates are arranged in the tweet candidate list (L1) in descending order of weight.
(Step 2) The tweet candidates are extracted from the tweet candidate list (L1) in descending order of weight.
(Step 3) Insert the extracted tweet candidates into the replay tweet list (L2). The list is in ascending order of utterance time.
At this time, it is possible to apply a “placement rule” to be described later, insert it at a position different from the actual utterance time, arrange the tweets so that they are easy to understand at the time of reproduction, and collect the same type of tweets.
In addition, it is possible to remove useless voices by applying an “omitted rule” described later.

(Step 4) The total playback time of the playback tweet list (L2) is calculated. If the time limit is exceeded, the tweet inserted above is removed from the playback tweet list (L2).
(Step 5) If the tweet candidate list (L1) is empty, the process ends. Otherwise, the process returns to 2.
As a modification of the above algorithm, the processing may be terminated when the total weight is equal to or greater than a threshold value. In this case, the process can be completed in a short time.

(Example of calculation of playback time in playback target tweet selection / alignment algorithm)
A method of calculating the total playback time of the playback tweet list in Step 4 of the playback target tweet selection / arrangement algorithm will be described.

  Each tweet arranged in the reproduction tweet list is basically reproduced in the order of the synthesized voice of the speaker name (or common name), the synthesized voice of the utterance date and time, and the voice data of the tweet. Here, the utterance date may be only the utterance date. If it is the same week as the current date and time, only the day of the week may be used.

  However, if the tweet is the same speaker, the same name is reproduced repeatedly and time is wasted. In addition, since unnecessary audio is included in the original audio data, unnecessary audio reproduction can be eliminated by applying the following omission rule.

[Omission rule]
If the utterer ID of a tweet in the playlist (L2) is the same as the utterer ID of the tweet immediately before, the synthesized speech of the speaker name (or common name) is removed.
If the date of the utterance date of a tweet in the playlist (L2) is the same as the date of the utterance date of the tweet immediately before, the synthesized voice of the date is removed.
・ Remove voices of words with little information (such as the name of the patient and “um”).

  When replaying a tweet about a patient, the tweet about which patient may be replayed with synthesized speech only at the beginning of the replay. For this reason, when the voice name of each tweet includes a patient name, it may be deleted.

  With existing voice recognition technology, words such as patient names and “um” can be extracted as keywords that do not need to be played, and the start time and end time (relative time from the start time of voice data) of these keywords can be specified. . In the tweet holding means 2, these pieces of information can be held as keyword accompanying information. For this reason, by deleting portions from the start time to the end time of these keywords from the sound data, it is possible to remove a sound portion with a small amount of information.

(Creating a tweet playback sequence that is easy to understand during playback)
Reply tweets about a certain tweet are easier to understand during playback if they are placed immediately after the tweet of the reply destination. Also, tweets having a causal relationship (described later) based on a keyword or a task at the time of utterance can be easily understood at the time of reproduction by arranging the “fruit” tweets immediately after the “cause” tweets.

For this reason, the above-described object can be achieved by applying the following arrangement rule when inserting a tweet candidate into the reproduction tweet list (L2) in Step 3 of [Reproduction Target Tweet Selection / Arrangement Algorithm].
[Placement rule 1]
・ Insertion rules for reply tweets (causal “fruit” tweets)
When a certain tweet i extracted from the tweet candidate list (L1) is a reply to any tweet j in the playlist (L2), the tweet i is inserted immediately after the tweet j.
・ Return destination tweets (causal “cause” tweets) insertion rules
When a certain tweet i taken out from the tweet candidate list (L1) is the reply destination of some tweet j in the playlist (L2), tweet i is immediately before the top of all tweet j Insert into.
By combining the same type of reply tweets ("fruit" tweets) arranged according to the above two rules as follows, playback of tweets with the same content and a small amount of information can be compressed.
“A sequence of synthesized speech of all utterance names (or common names) of consecutive reply tweets of the same type + synthesized speech of reply contents”
For example, if A, B, and C all responded to a tweet with a reply tweet such as “OK” or “OK”, the playback audio obtained from the above summary is “A” and “B” , Mr. C, I understand. "

[Placement rule 2]
As another tweet arrangement rule, the following embodiment is also possible. When a causal relationship between tweets can be extracted using business knowledge such as a reply relationship by a user and the order of medical treatment, a series of tweets forms a lattice structure. At this time, tweets of the same speaker are reproduced based on the time order within a range that maintains the order relation due to the causal relation. If there is an order relationship due to a causal relationship, the speaker is changed and the oldest tweeter is selected. If the speaker changes, tag speech synthesis is performed.

  FIG. 5 shows an example of determining the playback order utilizing the lattice structure. The ellipses in the figure schematically represent tweets. A tweet about Mr. K who is a patient, Doctor A, Nurse B, and Caregiver C is selected as the target for regeneration. The replay order of these tweets is represented by numbers in the ellipse. Basically, the speakers are selected in the order of having the oldest tweets, and the playback order of tweets is determined so that the tweets are played back in order from the oldest to the newest. However, the speaker is appropriately changed within a range that maintains the order relationship due to the causal relationship. In the illustrated example, the tweet 15 of the doctor A has a causal relationship with the tweet 5 of the nurse B and the tweet 12 of the nurse B. Condition 1). Also, the C caregiver's tweet 17 has a causal relationship with the nurse B's tweet 12, and is regenerated after the nurse B's tweet 12 (condition 2). In order to satisfy these conditions 1 and 2, the reproduction order of tweets is determined according to the above principle. In this example, first, since it is me who has the oldest tweet (tweet 1), the playback order is determined so that his tweet is played back in order from the oldest one. Next, since the C caregiver has the oldest tweet (tweet 8), the playback order is determined so that the tweets of the C caregiver are played in order from the oldest one. At this time, the C caregiver's tweet 17 needs to be played after the nurse B's tweet 12, so that the speaker is changed before the tweet 17 is played. Next, the nurse who has the oldest tweet (tweet 11) is the nurse B, so the speaker is changed to the nurse B. The playback order is determined so that the nurse's tweets are played in order from the oldest. Next, because Doctor A's oldest tweet 15 is older than C Caregiver's tweet 17, the speaker is changed to Doctor A, and the playback order is determined so that Doctor A's tweets are played in order from the oldest. The Finally, the reproduction order is determined so that the C caregivers' tweets 17 and 18 are reproduced in this order. As a result, the replay order of tweets is determined in the numerical order shown in the figure.

(Example of application of business flow in tweet arrangement)
A medical work flow represents the cause-and-effect relationship between tweets applied in the above-described tweet arrangement. As an example of the regional medical work flow,
The flow consists of a chain of tasks: observation by visiting nurses → doctor prescription → dispensing and dispensing by pharmacists → confirmation of remaining medicine by visiting nurses. When a tweet is registered for a patient at each medical position during the above tasks, the voice tweet system estimates the task at the time of utterance from the work schedule, position, utterance keyword, etc. of the medical position and accompanies the tweet. An estimation task is automatically attached as information.

  Using this estimation task, a tweet about a certain estimation task can be placed immediately after the task ahead in the flow when replaying tweet candidates are arranged. With the arrangement of the reproduction train, when the user listens to the reproduction voice, it is possible to understand the situation according to the work flow.

(Second example: Retrieval extraction of replay target tweets)
In the extraction of reproduction tweet candidates in step 2 and step 3 in the above-mentioned [reproduction target tweet selection / alignment algorithm], reproduction tweet candidates can be narrowed down and extracted. Specifically, the query can be narrowed down by issuing a query with the following designation as a query for extracting a tweet candidate to the tweet holding means 2. A query combining these specifications may be used. Implementation of these queries is possible with alignment processing for SQL and extracted results.

・ Target patient ID (when included in retweet request)
・ Unread tweets ・ Recent tweets, the total speech time of voice data is less than K times the time limit (K = 2, etc.)
・ The utterance date is within the past month ・ The speaker is a specific job type (the job type of each user is specified on the user master) The speaker is a specific user ・ Other than just reply tweets (for example, just reply tweets) Judgment can be made by including only a predetermined keyword indicating "simple reply")

(Third example: Variation of playback time constraint)
There are the following designation methods for the time limit for reproduction of tweets in the selection and arrangement of tweets to be reproduced as described above.

-Using the estimated value of the required travel time from the user's current location to the next visit destination A mobile terminal such as a smartphone can acquire the latitude and longitude of the user's current location by receiving GPS data. If the latitude / longitude or address of the user's next visit is registered on the server, the travel time to the next visit can be calculated using an existing map information application (such as google maps). Can be estimated. The estimated value itself or a time obtained by adding a constant time to the estimated value can be used as a time limit for replaying tweets. The current position of the user can be transmitted from the portable terminal to the server, and the latitude / longitude or address of the next visit destination can be transmitted from the server to the portable terminal. Therefore, the function and means for calculating the time limit described above can be arranged on either the mobile terminal or the server.

-The user can specify the time limit on the mobile terminal. When the playback request is transmitted, the time limit can be specified on the screen of the mobile terminal.

Use default values (eg 5 minutes)
If neither of the above methods is used, a default time limit can be used.

(4th example: Change of playback speed and playback content of tweets)
In the tweet playback on the user's terminal, it may be possible to select whether to increase the playback speed, such as double speed designation.
・ It is also possible to specify the playback speed according to the type of tweets (instructions, implementation reports, replies, etc.), keywords, speakers, and the occupation of the speaker.
・ Simple reply (“OK” etc.) is double speed
・ Doctor's instructions are always replayed normally

(Fifth example: Check the tweets you noticed during playback)
While listening to a voice tweet reproduced by the above method, if there is a voice tweet that the user does not understand well or a voice tweet of interest, the tweet can be checked.

  For example, the user adds a check by operating the screen of the portable terminal or the button operation of the headset while the voice tweet is being reproduced or immediately after the reproduction. By checking, the ID of the concerned tweet is added to the “List of tweet IDs of interest”.

  When the playback is completed, or the user's screen operation or headset button operation, the tweet voice corresponding to the ID in the tweet ID list is continuously played back. Or you may enable it to skip a sound reproduction of a tweet during reproduction | regeneration.

  As described above, in addition to listening to a tweet that has become a concern after a series of tweets has been continuously played back, it may be possible to re-listen to the tweet that is of interest at the point of interest. For example, the continuous playback of a series of tweets is temporarily stopped by a user's screen operation or headset button operation, and the tweets of interest are re-listened. With this tweet as a starting point, it may be possible to reproduce a tweet having a causal relationship.

(Sixth embodiment: tweet playback method not limited to playback within the time limit)
In the replay tweet scheduling described so far, past retweets or tweets determined to be insignificant may be excluded from the replay data. However, this may not be desirable for the user.

  Therefore, in this embodiment, in such a case, a method according to the flowchart shown in FIG. 4 is executed. 4A is a flow on the server side, and FIG. 4B is a flow on the user terminal side.

  The server receives an unread tweet playback request from the user terminal (S51), issues an extraction query for the unread tweet (S52), acquires the unread tweet (S53), and retrieves the acquired unread tweet. It transmits to the user terminal (S54). When the user terminal receives an unread tweet (S55), it first plays a summary of a series of tweets in a short time (S56), after the user checks the tweet that they are interested in, and the summary is finished playing Then, he hears the voice of the tweet again checked by the user (S57).

  In the playback of step S56, for the unread tweet, the name of the utterer of the tweet, the utterance time, the target patient designation, the keyword included in the tweet, the type of tweet (observation, instruction, implementation report, question, etc.) by synthetic voice, What is necessary is just to reproduce | regenerate in order of utterance time of a tweet.

  Alternatively, unread tweets may be collected for each target patient, and the tweets about each target patient may be reproduced in the order of utterance times. In that case, the synthesized voice of the target patient designation may be reproduced only at the beginning of the reproduction of a series of tweets of the patient.

  As in the previous example, if the utterers and utterance dates of consecutive tweets are the same, only the summary of the first tweet in a series of tweets is played back, and the synthesized voice of the speaker's name and utterance date is played back. do it.

  When the summary of an unread tweet is being played, if there is a tweet summary of interest, the user can select the tweet by operating the screen of the mobile terminal or the button on the headset while the summary is being played. Can be checked. When the playback of the overview is completed, the voice of the tweet corresponding to the ID in the ID list of the tweet of interest is continuously played back automatically or by the user's screen operation or headset button operation (S57). . You can also skip tweet playback during playback. At the time of reproduction, scheduling similar to the above-described embodiment can be performed on the mobile terminal by applying rearrangement and summarization of tweets, omission of voice with a small amount of information, and the like. Alternatively, by sending the ID list of the tweet of interest to the server, performing scheduling similar to the above-described embodiment for the tweet of interest on the server, and creating and transmitting playback data to the mobile terminal Even can achieve the same purpose.

  The flowchart in FIG. 4 is intended for unread tweets, but by using the conditions listed in the second embodiment and combinations of those conditions as the extraction conditions in step S52, other than “unread tweets” are used. The above method can also be applied to tweets.

  In the above embodiment, a voice message is targeted, but the present embodiment can also be applied to video data. In this case, extracting video data as playback candidates, weighting the video data, selecting and aligning the video data so that the playback time is within the time limit, and generating playback video data It becomes. By replacing words such as tweets or voice messages in the description so far with video data, the same can be applied to video data.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Moreover, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined. Further, various modifications as exemplified below are possible.

Claims (13)

Weighting means for setting a weight according to attribute information of the voice message for a plurality of voice messages;
Based on the weight of the voice message, one or more voice messages are selected from the voice messages so that the total playback time is within the time limit, and the playback order of the selected voice messages is selected. Scheduling means for determining based on attribute information of the voice message,
Replay data creating means for creating replay data by aligning and combining the selected voice messages according to the replay order;
Transmitting means for transmitting the reproduction data or link information to the reproduction data to a user terminal;
An acquisition means for acquiring time from a timing device is provided,
The attribute information includes information related to the utterance time,
The reproduction data generation device, wherein the weighting unit sets the weight according to a difference between the time acquired by the acquisition unit and the utterance time.   The greater the difference between the time acquired by the acquisition means and the utterance time, the smaller the weight.
  The reproduction data generation apparatus according to claim 1. Weighting means for setting a weight according to attribute information of the voice message for a plurality of voice messages;
Based on the weight of the voice message, one or more voice messages are selected from the voice messages so that the total playback time is within the time limit, and the playback order of the selected voice messages is selected. Scheduling means for determining based on attribute information of the voice message,
Replay data creating means for creating replay data by aligning and combining the selected voice messages according to the replay order;
Transmission means for transmitting the reproduction data or link information to the reproduction data to a user terminal,
The scheduling means (1) arranges the first voice message after the second voice message when the first voice message is a reply to the second voice message in the selected voice message, or (2 ) In the selected voice message, when the first voice message is acquired by a task behind the second voice message in the task flow, the first voice message is arranged after the second voice message. Generator. The reproduction data generation apparatus according to any one of claims 1 to 3, wherein the scheduling unit selects the voice message so that the sum of the weights is a maximum or a threshold value or more. Receiving means for receiving a reproduction request from the user terminal;
An extraction means for extracting a voice message candidate to be reproduced from a group of voice messages in response to the reproduction request;
The weighting means sets the weight for the voice message extracted by the extracting means,
The reproduction data generation apparatus according to any one of claims 1 to 4 , wherein the scheduling unit selects the one or more voice messages from the voice messages extracted by the extraction unit. The attribute information includes information related to a speaker, message content type information, information about a keyword included in a voice message, information about the number of replies to the voice message, information about user evaluation of the voice message, and a topic Further comprising at least one piece of information,
The reproduction data generation device according to any one of claims 1 to 5, wherein the weighting unit sets the weight by further using the at least one information. The reproduction request includes an extraction condition for a candidate voice message to be reproduced,
The reproduction data generation apparatus according to claim 5 , wherein the extraction unit extracts a voice message candidate to be reproduced based on the extraction condition and the attribute information. The said scheduling means adds the synthetic voice based on the attribute information of the said selected voice message, or the text showing the said synthesized voice to the beginning of each of the said selected voice message. Reproduction data generation device. 9. The synthesized voice or text having a common content between consecutive voice messages is added to the voice message at the head of the continuous voice message only by the scheduling means adding the synthesized voice or text having the common content. Reproduction data generation device. The attribute information includes a keyword representing a word that does not need to be reproduced,
The reproduction data generation apparatus according to any one of claims 1 to 9, wherein the scheduling unit deletes the voice data of the keyword from the selected voice message. The time required to travel to the visited location is calculated based on the current location of the user terminal and the location of the user visited by the user terminal, and the time limit is calculated based on the required travel time. The reproduction data generation device according to any one of 10. A weighting step for setting a weight according to attribute information of the voice message for a plurality of voice messages;
Based on the weight of the voice message, one or more voice messages are selected from the voice messages so that the total playback time is within the time limit, and the playback order of the selected voice messages is selected. A scheduling step for determining based on the attribute information of the voice message,
A reproduction data creation step of generating reproduction data by aligning and combining the selected voice messages according to the reproduction order;
Transmitting the playback data or link information to the playback data to a user terminal;
An acquisition step of acquiring time from a timing device,
The attribute information includes information related to the utterance time,
The reproduction data generation method in which the weighting step sets the weight according to a difference between the time acquired by the acquisition step and the utterance time. A weighting step for setting a weight according to attribute information of the voice message for a plurality of voice messages;
Based on the weight of the voice message, one or more voice messages are selected from the voice messages so that the total playback time is within the time limit, and the playback order of the selected voice messages is selected. A scheduling step for determining based on the attribute information of the voice message,
A reproduction data creation step of generating reproduction data by aligning and combining the selected voice messages according to the reproduction order;
Transmitting the playback data or link information to the playback data to a user terminal, and
In the scheduling step, (1) when the first voice message is a reply to the second voice message in the selected voice message, the first voice message is arranged after the second voice message, or (2 ) In the selected voice message, when the first voice message is acquired by a task behind the second voice message in the task flow, the first voice message is arranged after the second voice message. Generation method.

Images