JP2016201678A - Recognition device and image content presentation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a recognition device and an image content presentation system for displaying an image with information superimposed thereon with suitable timing, without clicking a comment contribution key or depressing an enter key.SOLUTION: A recognition unit 130 of a viewer terminal 100 obtains, on the basis of acoustic signals including at least non-speech sounds collected from a microphone 120, predetermined visual information corresponding to the non-speech sounds and being other than character strings obtained by writing the non-speech sounds as it is, and transmits the visual information to a moving image distribution server via a communication line.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for superimposing and displaying information input by a person viewing a video.

  Non-Patent Document 1 is known as a prior art that displays text information input by a person viewing a video in a superimposed manner. In Non-Patent Document 1, a viewer can post a comment while watching a moving image.

“Watching videos and posting comments”, [online], NIWANGO.INC, [Search February 2, 2015], Internet <URL: http://info.nicovideo.jp/help/player/howto/>

  However, in the conventional technology, it is necessary to enter a comment after clicking the comment post button or pressing the enter key after you want to comment on the video, so it is delayed from the timing when the viewer wants to comment Comments may be displayed. Conversely, if you know the content of the video in advance, you can enter comments in advance and the viewer can try to click the comment post button or press the enter key. , It may be earlier or later than when the viewer wants to comment. For example, when commenting the text information “8” meaning applause in time with the tempo of a music video or live video composition, there are many cases where the discrepancy is greater than when applause is actually performed, or the discrepancy is larger. Prone.

  An object of the present invention is to provide a recognition apparatus and a video content presentation system for displaying information superimposed on video at an appropriate timing without clicking a comment posting button or pressing an enter key.

  In order to solve the above-described problem, according to one aspect of the present invention, the recognition apparatus is adapted to sound other than utterance based on a sound signal including at least collected sound other than utterance and other than utterance. It includes a recognition unit that obtains predetermined visual information that is not a character string that directly represents the sound.

  According to the present invention, there is an effect that information can be displayed superimposed on video at an appropriate timing without clicking a comment posting button or pressing an enter key.

The functional block diagram of the video content presentation system which concerns on 1st embodiment. The figure which shows the example of the processing flow of the video content presentation system which concerns on 1st embodiment. The figure which shows the example of the video content with visual information. The figure which shows the example of a visual information database.

  Hereinafter, embodiments of the present invention will be described. In the drawings used for the following description, components having the same function and steps performing the same process are denoted by the same reference numerals, and redundant description is omitted.

<Video Content Presentation System 1 According to First Embodiment>
FIG. 1 is a functional block diagram of a video content presentation system 1 according to the first embodiment, and FIG. 2 shows a processing flow thereof.

  The video content presentation system 1 includes one or more viewer terminals 91, a viewer terminal 100, and a moving image distribution server 92 that distributes video content to the viewer terminals 91 and 100. The viewer terminals 91 and 100 and the moving image distribution server 92 can communicate with each other via a communication line.

<Viewer terminal 91>
The viewer terminal 91 is operated by a user who views video content (for example, moving images) (for example, a moving image viewer), and has an input unit (keyboard, mouse, touch panel, etc.) and a display unit (display, touch panel, etc.). Including, for example, a personal computer, a smart phone, and a tablet. The viewer can request the video distribution server 92 to play back video content via the input unit of the viewer terminal 91. In addition, the video content can be viewed via the display unit of the viewer terminal 91. Furthermore, the viewer can input visual information (for example, a comment) displayed superimposed on the video content via the input unit. Here, the “visual information” is information visually recognizable via the display unit, and is, for example, a character, a figure, a symbol, a combination thereof, or a combination of these and a color. Further, the image is not limited to a still image, and may be a moving image. For example, (1) text information (such as “w” or “8”) meaning a predetermined act such as “laughter” or “applause”, (2) “laughter” or “applause” other than text information Bit information on the computer to identify and identify a predetermined action, (3) Emoticon, pictograph, etc. that are not normal text information. For example, common pictograms between mobile phones of different carriers (see Reference 1), (4) ASCII art, etc., which are generally like pictures using text information and text information layout information (Reference 2) (5) Net slang corresponding to the above (1) to (4). For example, there is a net slang such as “I grew up” for text information “www...
(Reference 1) “docomo / au common pictograms”, NTT DOCOMO, Inc., [online], [Search February 9, 2015], Internet <URL: https://www.nttdocomo.co.jp/service / developer / smart_phone / make_contents / pictograph />
(Reference 2) "ASCII art", [online], February 2, 2015, Wikipedia, [Search February 9, 2015], Internet <URL: http://en.wikipedia.org/wiki/% E3% 82% A2% E3% 82% B9% E3% 82% AD% E3% 83% BC% E3% 82% A2% E3% 83% BC% E3% 83% 88>

<Video distribution server 92>
The moving image distribution server 92 receives moving images from the moving image database and the video camera, and distributes the moving images stored in the moving image database or the moving images recorded by the video camera in real time in response to requests from the viewer terminals 91 and 100. To do. In addition to moving images recorded by a video camera, CG synthesized and edited in real time or CG synthesized from motion capture may be distributed in real time. In the present embodiment, the moving image means video content provided with an audio signal synchronized with the time axis. The moving image database stores visual information added to the moving image together with the moving image. Furthermore, metadata is added to the visual information. The metadata includes the input time of visual information, the size of visual information, its color, its appearance method, its moving speed, moving position, and the like. For example, the size, color, appearance method, moving speed, moving position, and the like may be selectable by the input person of the visual information, and the viewer terminals 91 and 100 transmit the metadata together with the visual information as a moving image. Stored in the database with the video.

<Viewer terminal 100>
The viewer terminal 100 is operated by a person who views video content (for example, a moving image) (for example, a moving image viewer). The viewer terminal 100 includes a display unit (display, touch panel, etc.) 110, a sound collection unit (microphone, etc.) 120, a recognition unit 130, and a display size acquisition unit 140, and includes, for example, a personal computer, a smart phone, a tablet, and the like. . The viewer can request the video distribution server 92 to play back video content via the input unit (sound collection unit 120, keyboard, mouse, touch panel, etc.) of the viewer terminal 100. Also, the video content can be viewed via the display unit 110 of the viewer terminal 100. The viewer of the viewer terminal 100 can input visual information (for example, a comment) displayed superimposed on the video content via the input unit.

  The viewer terminal 100 displays the video content via the display unit 110 (S110A). Note that the video content is distributed from the moving image distribution server 92. The viewer terminal 100 collects the sound generated by the viewer of the video content corresponding to the video content as the sound signal x (t) via the sound collection unit 120 (S120). Note that t is an index representing time.

<Recognition unit 130>
The recognizing unit 130 receives the collected sound signal x (t) including at least a sound other than the utterance, and corresponds to the sound other than the utterance based on the sound signal x (t) and outputs the sound other than the utterance. Predetermined visual information v (t) that is not a character string written as it is and is visually recognizable via the display unit or display unit 110 of the viewer terminal 91 is obtained (S130), and the video distribution server is obtained via the communication line. 92.

  The moving image distribution server 92 receives the visual information v (t) and distributes it superimposed on the moving image. The visual information v (t) added to the moving image is stored in the moving image database together with the moving image. The display unit or display unit 110 of the viewer terminal 91 receives and displays the video content on which the visual information v (t) is superimposed (S110B). In addition, at the time of playback when visual information v (t) is transmitted, only the video is distributed without superimposing visual information v (t), and at the subsequent playback, the video with visual information superimposed is distributed Also good.

  Here, “sound other than speech” means sound other than “speech produced as a result of language as speech,” such as laughter and applause.

  “Predetermined visual information” corresponds to sounds other than the above-mentioned utterances (for example, applause sound, laughter), and in this embodiment, text information “8” representing applause sound as “predetermined visual information” It is assumed that text information “w” indicating laughter is used.

  The “character string in which sounds other than utterances are expressed as they are” is basically text information obtained by inputting sounds other than utterances into a conventional speech recognition apparatus. Since the conventional speech recognition apparatus recognizes “speech”, that is, “speech produced as a result of speech as a language” and “sound other than speech” as a recognition target, it is appropriate to recognize The recognition result cannot be obtained. For example, if the sound other than utterance is applause, even if applause sound is input to a conventional speech recognition device, it is unlikely that text information such as “flapping” will be obtained, and may not be subject to speech recognition as noise Is expensive. If it is desired to obtain the text information “flickering” as a result of speech recognition using a conventional speech recognition apparatus, it is necessary for a person to pronounce “cracking”. Similarly, when the sound other than the utterance is a laughing voice, even if the laughing voice is input to the conventional voice recognition device, it is unlikely that the text information “wahaha” can be obtained and is not subject to voice recognition as noise, or There is a possibility that a voice recognition result that cannot be judged as a laughing voice is obtained. Therefore, the recognizing unit 130 determines predetermined visual information (for example, text information “applause sound indicating applause sound) different from text information obtained by inputting sounds other than utterances into a conventional speech recognition apparatus for sounds other than utterances. 8 ”and text information“ w ”representing laughter).

  Hereinafter, a method of recognizing “sound other than speech” and acquiring visual information as a recognition result will be described. In the present embodiment, the predetermined visual information is obtained so as to be superimposed on the video content.

  The recognizing unit 130 obtains an index (eg, volume, power, energy) representing the magnitude from the received sound signal x (t), and based on the magnitude relationship between the magnitude index and a predetermined threshold, When it is determined whether or not some sound can be picked up instead of silence, predetermined visual information is acquired by the following method. For example, when the loudness indicator is a value that increases as the sound increases (for example, volume, power, energy), it is determined that there is no sound when the loudness indicator is less than a predetermined threshold. Sometimes it is determined that some sound could be picked up. In addition, when the index indicating the magnitude is smaller as the sound signal is larger, it is determined that there is no sound when the index indicating the magnitude is larger than a predetermined threshold, and any sound can be collected when the index is smaller than the threshold. It is determined that In the present embodiment, it is determined that any sound can be collected when the volume of the sound signal x (t) is equal to or higher than a predetermined threshold.

(Acquisition method 1)
The recognition unit 130 obtains predetermined visual information from one or more types of visual information already superimposed on the video content. For example, at a time when the volume of the sound signal x (t) is equal to or higher than a predetermined threshold, one type of visual information is selected from one or more types of visual information already superimposed on the video content, and the predetermined visual information is selected. And For example, (1-1) the predetermined visual information is selected as the type of visual information having the largest number of superimposed information. In addition, (1-2) predetermined visual information is selected at random according to the ratio of the number superimposed. (1-3) Randomly select predetermined visual information from one or more types of superimposed visual information.

  For example, when the video content with visual information as shown in FIG. 3 is received at the time when the volume of the sound signal x (t) exceeds a predetermined threshold, text information “8” representing applause sound and laughter Two types of visual information, the text information “w” to be represented, are already superimposed on the video content, and the number of the respective superimposed pieces of visual information is four and two. In this embodiment, certain visual information (for example, text information “8”) and repetition of the visual information (for example, text information “888”) are handled as the same type of visual information. However, it may be handled as different types of visual information. Either one of the two types of visual information is selected to obtain predetermined visual information. In the case of (1-1), the type of visual information with the largest number of superimposed information is text information “8” representing applause sound, and is thus obtained as predetermined visual information. In the case of (1-2), the ratio of the number of text information “8” superimposed on applause sound is 4/6, the ratio of the number of text information “w” superimposed on laughter Is 2/6, and predetermined visual information is selected at random according to this ratio. For example, text information “8” representing applause sound with probability of 4/6 is selected as predetermined visual information, and text information “w” representing laughter with probability of 2/6 is selected as predetermined visual information. . In the case of (1-3), text information “8” representing applause sound with a probability of 1/2 is selected as predetermined visual information, and text information “w” representing laughter with a probability of 1/2 is predetermined. Select as visual information.

  For example, instead of “at the time when the volume of the sound signal x (t) is equal to or higher than a predetermined threshold”, “by the time when the volume of the sound signal x (t) is equal to or higher than the predetermined threshold” One type of visual information may be selected from one or more types of visual information that has already been superimposed on the predetermined visual information. For example, when the volume of the sound signal x (t) exceeds a predetermined threshold, one type of visual information is selected from one or more types of visual information obtained before that, and the predetermined visual information is selected. Also good. As a selection method, the methods (1-1) to (1-3) may be used.

(Acquisition method 2)
The recognizing unit 130 obtains predetermined visual information from one or more types of visual information prepared in advance for superimposing and displaying the video content. For example, a visual information database as shown in FIG. 4 is prepared in advance, and (2) predetermined visual information is randomly selected from one or more types of visual information. In addition, when it is desired to select only a predetermined action, for example, information meaning “laughter” as the predetermined visual information, only information meaning “laughter” is stored in the visual information database, for example, “w” “(laughter). ) "" :-) "" (^ o ^) "etc. should be prepared.

(Acquisition method 3)
(Acquisition method 2) may be combined with processing for recognizing what kind of sound the sound signal x (t) is.

  (3-1) For example, if the voice signal x (t) is subjected to VAD (voice activity detection) and determined to be a voice segment, `` sound other than speech '' becomes `` laughter '' It determines with corresponding, and the visual information corresponding to a laughter is selected as predetermined visual information. If it is determined as a non-voice section, it is determined that “sound other than speech” corresponds to “applause sound”, and visual information corresponding to applause is selected as predetermined visual information. In this case, when there are two or more types of actions that the predetermined visual information means, the visual information database stores whether the visual information corresponds to a voice section or a non-voice section. . In addition, when there are a plurality of types of visual information corresponding to a voice segment and a non-speech segment, predetermined visual information is selected at random.

  (3-2) For example, a voice feature is extracted in advance from “sound other than speech”, and a similarity with the voice feature extracted from the sound signal x (t) is obtained, and the similarity is equal to or greater than a predetermined threshold value. In this case, the visual information corresponding to the “sound other than speech” is selected as predetermined visual information. In addition, when there are a plurality of types of visual information corresponding to laughter, applause, etc., predetermined visual information may be selected at random. In the conventional speech recognition apparatus, the speech feature amount is extracted from the utterance. In the present embodiment, the speech feature amount is extracted from “sound other than the utterance”. In this case, “sound other than speech” means a predetermined action (laughing or applause) of the viewer (corresponding to the predetermined action) and does not include background noise or the like.

(Acquisition method 4)
(Acquisition method 1) may be combined with (acquisition method 2) or (acquisition method 3).

  The recognizing unit 130 (4-a) has one type of visual information already superimposed at the time of the video content corresponding to the sound signal collection time, or (4-b) When the ratio of one type is extremely high, the type of visual information is obtained as predetermined visual information.

  On the other hand, in cases other than (4-a) and (4-b), predetermined visual information is obtained from a plurality of types of visual information prepared in advance for superimposition and display on video content.

  For example, at the time when the volume of the sound signal x (t) is equal to or greater than a predetermined threshold (or “by the time when the volume of the sound signal x (t) is equal to or greater than the predetermined threshold”), it is already superimposed on the video content. Determine whether the visual information is one type or more. In the case of one type, the visual information is obtained as predetermined visual information. When there are two or more types, the ratio of the type of visual information with the largest number of superimposed information is obtained, and when the ratio is larger than a predetermined threshold (for example, 0.5), the visual information is selected as the predetermined visual information. To do. When the ratio of the type of visual information having the largest number of superimposed information is equal to or less than a predetermined threshold value, the predetermined visual information is selected by the method (acquisition method 2) or (acquisition method 3).

By such an acquisition method, it is possible to convert sounds other than speech into visual information that occupies most of the visual information currently displayed on the display unit and display it on the screen.
(Acquisition method 5)
As a combination of (acquisition method 1) and (acquisition method 2) or (acquisition method 3), the following methods are also conceivable.

  When the proportion of one type of visual information in the visual information already superimposed is high at the time of the video content corresponding to the sound signal collection time of (5) the sound signal, the recognition unit 130 determines that type of visual information. Is obtained as predetermined visual information, and in cases other than (5), the predetermined visual information should be obtained from a plurality of types of visual information prepared in advance for display in a superimposed manner on video content. By giving priority, predetermined visual information is obtained.

  For example, at the time when the volume of the sound signal x (t) is equal to or greater than a predetermined threshold (or “by the time when the volume of the sound signal x (t) is equal to or greater than the predetermined threshold”), it is already superimposed on the video content. Each ratio is calculated for each type of visual information, and when the ratio is greater than a predetermined threshold a (for example, a> 0.5), the visual corresponding to the ratio is determined with a predetermined probability b (0.5 <b <1). Information is selected as predetermined visual information, and predetermined visual information is selected by the method (acquisition method 2) or (acquisition method 3) with the probability of (1-b). On the other hand, when the ratio is equal to or less than the predetermined threshold a, the predetermined visual information is selected by the method of (Acquisition method 2) or (Acquisition method 3) with the predetermined probability c (0.5 <c <1), Visual information corresponding to the ratio is selected as predetermined visual information with a probability of (1-c).

<Display size acquisition unit 140>
The display size acquisition unit 140 receives the collected sound signal x (t) including at least a sound other than the utterance, obtains an index (for example, volume, power, energy) indicating the magnitude, and an index indicating the magnitude It is determined whether or not there is silence based on the magnitude relationship between and a predetermined threshold. For example, when the loudness indicator is a value that increases as the sound increases (for example, volume, power, energy), it is determined that there is no sound when the loudness indicator is less than a predetermined threshold. Sometimes it is determined that some sound could be picked up. In addition, when the index indicating the magnitude is smaller as the sound signal is larger, it is determined that there is no sound when the index indicating the magnitude is larger than a predetermined threshold, and any sound can be collected when the index is smaller than the threshold. It is determined that Furthermore, when it is determined that some sound can be picked up instead of silence, the display size acquisition unit 140 displays information corresponding to the index indicating the magnitude, and the predetermined visual information is displayed on the display unit or display of the viewer terminal 91. It is obtained as size information s (t) when displayed via the unit 110 (S140), and transmitted to the moving image distribution server 92 via the communication line. For example, the information s (t) is acquired so that the larger the sound signal x (t) is, the larger the sound signal x (t) is.

  With such a configuration, the character size can be changed and displayed on the display unit even in accordance with the volume, and the viewer's atmosphere can be conveyed in more detail.

<Effect>
With the above configuration, visual information can be displayed superimposed on video content at an appropriate timing without clicking a comment posting button or pressing an enter key.

<Modification>
In this embodiment, the display unit 110 displays visual information superimposed on the video content, but a display unit that displays only the video content may be separately provided.

  A conventional speech recognition apparatus may be incorporated in the viewer terminal 100 of the present embodiment. For example, a conventional speech recognition device may be incorporated in the preceding stage of the recognition unit 130 and the recognition unit 130 may perform the recognition process only when appropriate speech recognition cannot be performed.

  Although the display size acquisition unit 140 is provided in the present embodiment, it is not always necessary. If the display size acquisition unit 140 is not provided, a default value may be set in advance as information representing the size of visual information. The size of the visual information may be changeable from an input unit (not shown) by the operation of the viewer.

  In the present embodiment, the recognition unit 130 is incorporated in the viewer terminal 100, but may be configured as an independent recognition device. In addition, the recognition unit 130 may be configured to be incorporated in the video distribution server 92 or the viewer terminal 91 on the side of reproducing a video other than the viewer terminal 100. In that case, it is necessary to transmit the sound signal x (t) to a device in which the recognition unit 130 is incorporated. In consideration of the data transmission amount, a configuration in which the recognition unit 130 is incorporated in the viewer terminal 100 and the visual information v (t) is transmitted as in the present embodiment is desirable.

  In the present embodiment, video content is presented to the viewer on the display unit 110, but other content may be presented. The terminal only needs to be able to present content to the target person by some kind of stimulation, and may present video content by sound stimulation and light stimulation as in the present embodiment, or sound content (radio by only sound stimulation). Broadcasts, etc.), other stimuli (chemical substances, temperature, pressure) that can be received by other sensory organs (tactile sensation, olfactory sensation, taste sensation) of the subject, or each stimulus Content may be presented by a combination of Even in that case, the display unit 110 is used to display predetermined visual information. In addition, content presented by stimuli (light, sound, chemical substances, temperature, pressure, etc.) that can be received by the subject's sensory organs, or a combination thereof, is also referred to as “media content”. A configuration for presenting a subject to a target person is referred to as a presentation unit. The sound collection unit 120 collects sound generated by the target person who feels the stimulus from the media content in response to the media content.

  The viewer terminal 100 may include a display length acquisition unit (not shown). The display length acquisition unit receives the collected sound signal x (t) including at least the sound other than the utterance, obtains an index indicating the magnitude (for example, moving average of the volume), and obtains an index indicating the magnitude and a predetermined value. Whether or not there is silence is determined based on the magnitude relationship with the threshold value. For example, when the loudness indicator is a value that increases as the sound increases (for example, volume, power, energy), it is determined that there is no sound when the loudness indicator is less than a predetermined threshold. Sometimes it is determined that some sound could be picked up. In addition, when the index indicating the magnitude is smaller as the sound signal is larger, it is determined that there is no sound when the index indicating the magnitude is larger than a predetermined threshold, and any sound can be collected when the index is smaller than the threshold. It is determined that When the display length acquisition unit determines that it has been able to pick up some sound, not silence, information corresponding to the duration of the sound is displayed as predetermined visual information on the display unit or display unit 110 of the viewer terminal 91. Is obtained as the number of repetitions when being displayed, and is output to the recognition unit 130. For example, the number of repetitions increases as the duration of the sound signal x (t) increases. The recognizing unit 130 outputs the obtained predetermined visual information v (t) to the moving image distribution server 92 as predetermined visual information again according to the number of repetitions. For example, when the predetermined visual information v (t) is “w” and the number of repetitions is 5, the predetermined visual information v (t) is “wwwww” or a net slang that has the same meaning as this. A certain “Kusasei Eita” or the like is transmitted to the moving image distribution server via the communication line. With such a configuration, the viewer's atmosphere can be conveyed in detail. The recognition unit 130 outputs the predetermined visual information v (t) after obtaining the number of repetitions. Therefore, the visual information v (t) is converted into a moving image during reproduction when the visual information v (t) is transmitted. In the case of superimposing on the sound, there is a difference between the generation of sound other than the utterance and the output of the predetermined visual information v (t). In order to eliminate this shift, the predetermined visual information v (t) may be transmitted to the moving image distribution server 92 without being repeated at the time of reproduction at the time of transmission. Then, the moving image distribution server 92 receives the visual information v (t) and distributes it superimposed on the moving image without repeating. After obtaining the number of repetitions, the recognizing unit 130 outputs the obtained predetermined visual information v (t) to the moving image distribution server 92 as predetermined visual information again according to the number of repetitions. The moving image distribution server 92 stores visual information v (t) repeated together with the moving image in the moving image database. With such a configuration, the moving image distribution server 92 can distribute a moving image in which repeated visual information is superimposed on a reproduction request made after storing the repeated visual information v (t).

  In the present embodiment, the recognition unit 130 obtains an index representing the magnitude from the received sound signal x (t), and determines whether there is silence based on the magnitude relationship between the magnitude index and a predetermined threshold. Although it is determined, an existing VAD (voice activity detection) may be provided before the recognition unit 130 to determine whether the sound signal x (t) is silent by VAD. Further, when it is determined that some sound has been collected, the sound signal x (t) is input to the recognition unit 130. The recognition unit 130 uses the sound signal x (t) to generate predetermined visual information. Obtain v (t).

<Other variations>
The present invention is not limited to the above-described embodiments and modifications. For example, the various processes described above are not only executed in time series according to the description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. In addition, it can change suitably in the range which does not deviate from the meaning of this invention.

<Program and recording medium>
In addition, various processing functions in each device described in the above embodiments and modifications may be realized by a computer. In that case, the processing contents of the functions that each device should have are described by a program. Then, by executing this program on a computer, various processing functions in each of the above devices are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its storage unit. When executing the process, this computer reads the program stored in its own storage unit and executes the process according to the read program. As another embodiment of this program, a computer may read a program directly from a portable recording medium and execute processing according to the program. Further, each time a program is transferred from the server computer to the computer, processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program includes information provided for processing by the electronic computer and equivalent to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In addition, although each device is configured by executing a predetermined program on a computer, at least a part of these processing contents may be realized by hardware.

Claims (8)

A recognition unit that obtains predetermined visual information that corresponds to a sound other than the utterance and is not a character string that directly expresses the sound other than the utterance based on a sound signal including at least the sound other than the utterance collected. Including,
Recognition device. The recognition device according to claim 1,
The predetermined visual information is obtained for superimposition and display on video content,
The recognition unit
Obtaining the predetermined visual information from at least one of one or more types of visual information already superimposed on video content and one or more types of visual information prepared in advance for display superimposed on video content;
Recognition device. The recognition device according to claim 2,
The recognition unit
When there are a plurality of types of visual information already superimposed at the time of the video content corresponding to the sound collection time of the sound signal, the type of visual information having the largest number of superimposed information is selected as the predetermined visual information. Get as,
Recognition device. The recognition device according to claim 2,
The recognizing unit (1-a) when there is one type of visual information already superimposed at the time of the video content corresponding to the sound collection time of the sound signal, or (1-b) there are a plurality of types However, when the ratio of one type is extremely high, the visual information of the type is obtained as the predetermined visual information. (2) In other cases, prepared in advance to be displayed superimposed on the video content. The predetermined visual information is obtained from a plurality of types of visual information.
Recognition device. The recognition device according to claim 2,
The recognizing unit (1) when the proportion of one type of visual information in the visual information already superimposed at the time of the video content corresponding to the sound collection time of the sound signal is high, Priority is given to obtaining information as the predetermined visual information. (2) In other cases, the predetermined visual information is obtained from a plurality of types of visual information prepared in advance to be displayed superimposed on video content. Priority is given to obtaining the above-mentioned predetermined visual information,
Recognition device. The recognition apparatus according to any one of claims 1 to 5,
A display size acquisition unit that obtains information corresponding to an index that represents a loudness other than the utterance as loudness information when the predetermined visual information is displayed via the display unit;
Recognition device. A first display for displaying video content;
A sound collection unit that collects sound emitted in response to the video content and converts the sound into the sound signal;
A recognition device according to any one of claims 1 to 6;
A second display unit that displays the predetermined visual information obtained by the recognition device superimposed on video content,
Video content presentation system. A presentation unit for presenting media content;
A sound collection unit that collects sound emitted in response to the media content and converts the sound into the sound signal;
A recognition device according to any one of claims 1 to 6;
A display unit that displays the predetermined visual information obtained by the recognition device superimposed on video content corresponding to the media content,
Video content presentation system.

Images