JP4529632B2 - Content processing method and content processing apparatus - Google Patents

Description

  The present invention relates to a method and apparatus for processing content such as video and audio.

  When playing back content, it is considered to play or present other content together.

  For example, Patent Literature 1 (Japanese Patent Laid-Open No. 8-76779) refers to music information of a karaoke song, and controls lighting and air conditioning in order to raise the atmosphere of the place according to the content of the song. It is shown.

  Japanese Patent Application Laid-Open No. 2003-507808 discloses a description method for describing the contents and structure of video / audio data.

  Further, Patent Document 3 (Japanese Patent Laid-Open No. 2001-333410) shows that the user can view the content under the optimum condition and the user's preference by using the metadata describing the optimum viewing condition of the content. The data indicates that the content to be provided to the user is determined.

  Patent Document 4 (Japanese Patent Laid-Open No. 2002-158948) discloses that a plurality of video / audio data in which the same scene is recorded are reproduced in association with each other.

  As described above, it is considered to control lighting and air conditioning according to content contents in order to increase the atmosphere of the place when playing music content, and to select content according to user preference It is considered.

The prior art documents listed above are as follows.
JP-A-8-76779 Special table 2003-507808 gazette JP 2001-333410 A JP 2002-158948 A

  However, as shown in Patent Document 1, even when the atmosphere and presence are enhanced, the optimum atmosphere and presence are often different for the user.

  In particular, content such as sports broadcasts can be enjoyed as long as the atmosphere is on the stand of the supporting team, but it is fun if the atmosphere is on the other team's stand. Often not felt.

  For this reason, it is difficult to create an optimal atmosphere and presence only from the content.

  Even if the song is the same, depending on the mood of the user, there are times when you want to listen by clapping, etc., and when you want to listen carefully and quietly. If an atmosphere or a sense of presence is created, an atmosphere or a sense of presence corresponding to the user's current mood cannot be created.

  In view of this, the present invention is capable of creating an optimal atmosphere and a realistic sensation according to not only the content but also the user's mood at that time.

The content processing method of the present invention includes:
Sub-content to be played back together with the main content from the main content content metadata that characterizes the content of the main content and user status metadata that is data indicating the user profile and the physical or mental state of the user at that time A recommendation information generation step for generating recommendation information for recommending,
A feature information generating step for generating feature information indicating the feature of the sub-content from the sub-content content metadata characterizing the content of the sub-content;
A sub-content selection step of selecting, from among a plurality of sub-contents, a sub-content whose feature information is closest to the recommendation information as a sub-content to be reproduced together with the main content;
It is those with a.

In the content processing method described above, when content such as music or video is played back or recorded, not only the content of the main content that is the content, but also other content that is optimal for the current situation of the user is used as the sub-content. Since it is selected and reproduced or recorded together with the main content, it is possible to create an optimal atmosphere and a sense of realism not only according to the content (main content) but also according to the user's mood at that time.

  As described above, according to the present invention, it is possible to create an optimal atmosphere and realistic sensation according to not only the contents but also the user's mood at that time.

[1. Embodiment for Reproducing Content: FIGS. 1 to 4]
An embodiment in the case of reproducing content will be described below.

  The main content and the sub-content are named for convenience as described above, and there is no distinction that the main content is more important than the sub-content, and that the amount of data is large. .

(1-1. First example of reproduction: FIGS. 1 to 3)
FIG. 1 shows an example of a content processing apparatus according to the present invention. When main content recorded on a recording medium is reproduced, sub-content according to the content of the main content and the user's situation is selected and reproduced together with the main content. It is.

  The content processing device of this example includes a disc playback unit 1 as a content playback device, drives the spindle motor 3 and the optical pickup 4 under the control of the system controller 11, and stores the main content recorded on the optical disc 2. Read and play.

  In this example, the main content is video data such as movies and dramas, subtitle data, and audio data recorded by the MPEG (Moving Picture Experts Group) system.

  Specifically, as shown in FIG. 2, the data stream of the main content is obtained by adding video packets, subtitle packets, and audio packets to the header, and each packet includes video data for the header. Subtitle data or audio data is added, and the audio data of the audio packet consists of audio data of each audio channel.

  On the optical disc 2, main content content metadata characterizing the content of the main content can be recorded together with the main content data. When the main content content metadata is recorded on the optical disc 2, the main content content metadata is described in a format compliant with the MPEG7 standard or other formats, and is recorded for each GOP (Group Of Pictures) of the MPEG stream or in the MPEG stream. Multiplex for each picture.

  Main content content metadata that characterizes the content of main content is generally (1) information indicating the genre of content such as music, movies, dramas, documentaries, sports, and (2) if it is music, Information indicating the classification of scenes, such as information indicating that it is between songs, information indicating the succession of a story if it is a movie or drama, information indicating that it is a scoring scene if it is a sport, (3 In the case of sound (sound) such as music, information indicating the loudness of the sound, and in the case of video, information indicating the characteristics of the signal, such as information indicating the brightness of the screen.

  Information indicating the copyright relationship of the copyrighted work defined in MPEG21 can also be used for generating recommendation information, which will be described later, as information characterizing the content of the content, such as conditions relating to secondary processing of the content.

  In the following, the main content content metadata is recorded on the optical disc 2 together with the main content data as described above, and the main content content metadata is read from the main content data read from the optical disc 2 in the content processing device (content reproduction device) as described later. Together with the generation of.

  Data read from the optical disk 2 by the optical pickup 4 is supplied to the demodulation circuit 13 through the RF amplifier 12, demodulated by the demodulation circuit 13, and error-corrected.

  Further, the data after demodulation error correction is supplied to the demultiplexer 14, and video data, subtitle data and audio data constituting the main content, and main content content metadata are recorded on the optical disc 2 from the demultiplexer 14. In some cases, the main content content metadata is obtained separately.

  The separated video data, subtitle data, audio data, and main content content metadata are decoded by the video decoder 15, subtitle decoder 16, audio decoder 17, and metadata decoder 18, respectively, according to the decoding time stamp. The decoded video data, subtitle data, and audio data are converted into a video signal, a subtitle signal, and an audio signal, respectively, according to the presentation time stamp.

  The video signal from the video decoder 15 and the subtitle signal from the subtitle decoder 16 are supplied to the subtitle superimpose circuit 23 through the video reproduction circuit 21 and the subtitle reproduction circuit 22, respectively. The subtitle signal is superimposed on the screen, and the video with the subtitle superimposed is displayed on the display screen of the video display device 24 such as a liquid crystal display device.

  The audio data from the audio decoder 17 is selected as sub-content by the sub-content selection unit 46 as will be described later, and is supplied to the audio data mixing circuit 27 together with the audio data decoded by the audio decoder 47, and the audio data mixing circuit 27 1, the audio data as a part of the main content and the audio data as the sub content are mixed, and the mixed audio is output from the sound output device 28 such as a speaker or a headphone.

  On the other hand, in order to generate user situation metadata characterizing the user situation, key input information from the key input unit 31, biological information from the biological sensor 33, and information read from the external storage medium 35 are respectively The user status metadata generation unit 41 is supplied via the interfaces 32, 34 and 36.

  The key input unit 31 includes an infrared light emitting unit provided on the panel unit of the content processing apparatus main body or constituting a remote controller of the content processing apparatus and an infrared light receiving unit provided on the panel unit of the content processing apparatus main body. The user inputs the user's situation through the key input unit 31 in advance or at the time of content reproduction.

  The biological sensor 33 detects a user's physical or mental state. The sensor such as a pulse sensor or a perspiration sensor, a camera that detects a user's body movement (part or whole body movement), and a user's For example, a microphone that captures the voice.

  The external storage medium 35 is a removable storage medium such as a memory card in which information indicating a user's situation is written.

  The user's situation is directly: (a) the user's favorite music genre or artist, user profile such as a favorite team or player, and (b) the physical or mental state of the user at that time, concrete Specifically, it can be roughly classified into the pulse rate, the amount of sweating, the temperature of the fingertip, the hearing age, the visual age, the presence or absence of body movement, the size, etc., but it can be detected from the information in (b). The feelings of music that can be done, the degree of concentration in the story, and the feelings of “fun” and “want to relax” are also included in the physical or mental state of (b).

  In addition, the viewing environment of the user at that time, for example, background noise or brightness in the viewing room, also affects the physical or mental state of the user. Include.

  The user profile (a) is input by the user through the key input unit 31 or written in the external storage medium 35. The physical or mental state of the user at that time in (b) is detected directly by the biosensor 33 or by analyzing the output signal of the biosensor 33. In addition, the user's viewing environment at that time may be input by the user using the key input unit 31 or written in the external storage medium 35 as in the case of the user profile.

  In the user situation metadata generation unit 41, the information indicating the user situation as described above is comprehensively analyzed, and user situation metadata characterizing the user situation is generated. The generated user situation metadata is sent to the recommendation information generation unit 42.

  On the other hand, as described above, when the main content content metadata is recorded on the optical disc 2 and decoded by the metadata decoder 18, the main content content metadata from the metadata decoder 18 is used as the recommendation information generating unit 42. Is sent out.

  If no metadata is recorded on the optical disc 2, the video data from the video decoder 15, the subtitle data from the subtitle decoder 16, and the audio data from the audio decoder 17 are sent to the main content content metadata generation unit 43. The main content content metadata is generated by the main content content metadata generation unit 43 and sent to the recommendation information generation unit 42.

  Specifically, the main content content metadata generation unit 43 analyzes the video data, caption data, and caption data, and displays “dark scene”, “entire screen is red scene”, “entire screen is blue scene”, “dark noise”. "High-level scene", "Scene with wide dynamic range", "Scene with high level of low frequency component", "Scene with high level of high frequency component", "Subtitle is blue", "Yellow subtitle on white background" Main content content metadata such as “” is generated.

  The recommendation information generation unit 42 should combine the main content from the user status metadata from the user status metadata generation unit 41 and the main content content metadata from the metadata decoder 18 or the main content content metadata generation unit 43. Recommendation information for recommending sub-content is generated.

  On the other hand, in this example, a plurality of audio data (audio files) as sub-contents are recorded in advance in the external storage medium 37, and the external storage medium 37 is loaded into the content processing apparatus, so that the external storage medium 37 The audio data as the sub content is read and supplied to the sub content content metadata generation unit 44 through the interface 38. The external storage medium 37 in which the audio data as the sub-content is recorded may be the same storage medium as the external storage medium 35 in which the user profile and the like are recorded.

  The sub-content content metadata generation unit 44 analyzes the audio data as the sub-content and generates sub-content content metadata that characterizes the content of the sub-content.

  The sub content content metadata generated by the sub content content metadata generation unit 44 is sent to the feature information generation unit 45, and the feature information generation unit 45 generates feature information indicating the characteristics of the audio data as the sub content. Is done.

  Specifically, the feature information is generated by expressing the result of sub-content analysis on a multidimensional space. For example, in the case of content recorded with applause in a concert hall with a lot of reverberation, the dimensions are “giving a live feeling”, “applause by the audience”, and “no shout”.

  The feature information generated by the feature information generation unit 45 is sent to the sub-content selection unit 46, and the sub-content selection unit 46 generates the recommendation information generated by the recommendation information generation unit 42 and the feature information generation unit 45. From the feature information, the audio data whose feature information is closest to the recommendation information is selected as sub-contents.

  Specifically, the sub-content selection unit 46 displays the feature information generated by the feature information generation unit 45 for a plurality of audio files such as a plurality of songs recorded in the external storage medium 37, and the user status at that time and Compared with the recommendation information generated by the recommendation information generation unit 42 based on the content of the main content, the sub-content (audio file) whose feature information is closest to the recommendation information is detected, and the identification code, song title, etc. of the sub-content are detected. Identification information is notified to the system controller 11. In response to this, the system controller 11 reads the sub-content (audio file) having the identification information from the external storage medium 37 and sends it to the sub-content selection unit 46 through the interface 38.

  In selecting the sub-content, the dimension that is not important among the dimensions of the recommendation information is configured not to be used for determining the distance (difference) between the recommendation information and the feature information.

  The audio data selected by the sub-content selection unit 46 is decoded by the audio decoder 47, supplied to the audio data mixing circuit 27, mixed with the audio data decoded by the audio decoder 17, and mixed from the sound output device. Later audio is output.

  In the above example, the main content content metadata specifically includes “music live”, “during singing by singer A”, “at the end of performance song B”, “baseball broadcast”, “during C team attack” , "Scoring scene", "Bright screen", "Quiet scene", "Speaking scene", "Scene with sound effects", "Subtitles are displayed rolling from right to left Metadata such as “status” is obtained in association with the presentation time of the main content.

  As the recommendation information, specifically, for example, the main content content metadata is “pop singer” and “live concert”, and the user situation metadata is “favorite singer”, “favorite song” and “fun mood” ”Is generated,“ Energize live concert atmosphere ”and“ Make you feel participating ”.

  As sub-contents, for example, when the recommended information is information such as “enliven the atmosphere of a live concert”, “feel like participating”, and “end of a performance song”, “provide a live feeling” ”,“ Applause by audience ”and“ Recording applause in a concert hall rich in reverberation ”having feature information of“ no applause ”are selected.

  Therefore, in the above-described content processing method (content reproduction method), it is possible to create an optimal atmosphere and realistic sensation according to not only the content of the main content but also the user's mood at that time.

  FIG. 3 shows an example of reproduction processing performed by the content processing apparatus (content reproduction apparatus) in the example of FIG. 1 under the control of the system controller 11.

  In this example, the reproduction is started by the user's reproduction start operation. First, in step 51, the optical disc 2 and the optical pickup 4 are driven, and the multiplexed data of the main content is read and demodulated.

  Next, in step 52, the video data, subtitle data, audio data, and main content content metadata (or video data, subtitle data, and audio data if main content content metadata is not recorded on the optical disc 2) are separated, Further, the process proceeds to step 53, where necessary packets and audio channels are selected according to the user instruction and the playback environment of the content processing apparatus.

  Next, in step 54, the packet header is viewed and each data is sent to the corresponding decoder, and the process proceeds to step 55 where each data is decoded by each decoder. If the main content content metadata is not recorded on the optical disc 2, the main content content metadata is generated after step 55.

  Next, in step 56, user status metadata is generated, and further, recommendation information is generated from the user status metadata and main content content metadata.

  Next, in step 57, feature information is generated, and the process further proceeds to step 58, where the feature information is compared with recommendation information, and audio data as sub-contents is selected.

  Next, in step 59, the main content video, subtitles and audio and the sub-content audio are presented in combination. That is, video and subtitles are displayed by the video display device 24, and audio is output by the acoustic output device 28. However, the contents related to the packets not selected in step 53 are different.

  The example of FIG. 1 is a case where main content content metadata is recorded on the optical disc 2, or a case where main content content metadata is generated by the main content content metadata generation unit 43. The main content content metadata is a content processing device. It is acquired from the outside of the content processing apparatus by connecting to a server such as a content provider through a line connection unit included in the network, or acquired by a storage medium such as a memory card different from the optical disk 2 on which the main content is recorded. Also good.

  The example of FIG. 1 is a case where sub-contents are recorded on the external storage medium 37. The sub-contents are obtained by connecting to a server such as a content provider by a line connection unit provided in the content processing apparatus. You may make it acquire from the outside of a processing apparatus.

  In that case, sub-content content metadata and further feature information is generated in an external server and transmitted to the content processing device on the user side. The content processing device converts the feature information and recommendation information into recommendation information. The system may be configured to select the sub-content having the closest feature information.

  Further, by transmitting the main content content metadata and user status metadata or recommendation information from the content processing device on the user side to the external server, or the main content content metadata is provided in the external server as described above, Alternatively, when the external server generates the main content content metadata, the user status metadata is transmitted from the content processing device on the user side to the external server, so that the external server uses the recommendation information and the feature information. The system may be configured to select the sub-content having the feature information closest to the recommendation information and transmit the sub-content to the user-side content processing apparatus.

(1-2. Second example of reproduction: FIG. 4)
FIG. 4 shows an example in which a broadcast is received, main content is acquired and played back.

  In this example, the digital broadcast receiving unit 61 receives digital broadcasts such as BS digital broadcasts, CS digital broadcasts, and terrestrial digital broadcasts as main contents. The received main content data is demodulated by the demodulation circuit 63 and separated into each data by the demultiplexer 14 as in the example of FIG.

  Furthermore, in this example, the content processing device (broadcast reception / playback device) is configured to be able to acquire sub-contents by connecting to a content provider server such as a broadcast source by the line connection unit 67. The acquired sub content data is sent to the sub content content metadata generation unit 44 and the sub content selection unit 46 through the interface 68. The rest is the same as the example of FIG.

  Broadcasting may be live broadcasts, and not only the atmosphere and presence, but also immediacy is required. In this example, when the main content broadcast is a game relay such as baseball or soccer, sub-contents of each team You can get contents such as the latest state of the support seat.

  Specifically, for example, the recommendation information is “a match between baseball team A and team B”, “attack of player C during attack of team A”, “batting rate of player C is 0.33”, and “user is player When it is called “C fan”, the sub-contents have the characteristic information “support for the stand on the team A side” and “support for the player C”. Is selected and played in combination with the main content.

  The example of FIG. 4 is a case where the sub-content is audio data as in the example of FIG. 1, but the sub-content may be video data, video data, and audio data. When the main content and the sub-content both include video, the sub-content video is displayed in a two-screen display together with the main content video, or is displayed in place of the main content video, or alternately with the main content video. The content processing apparatus is configured.

  Also in this example, as described above with reference to the example of FIG. 1, the sub-content can be selected in an external server.

[2. Embodiment for Recording Content: FIGS. 5 to 8]
The present invention can also be applied when recording content.

  As an example in FIG. 5, an image (image data) obtained by photographing a subject with an imaging device is used as main content, and an image (image data) according to the content of the main content and the user's situation is used as sub content. The case of selecting and recording in combination with the main content is shown.

  The content processing system of this example is configured by an imaging device 70 on the user side and a server system on the service providing side, and is used in a theme park as shown in FIG. 6, for example.

  The theme park of FIG. 6 is provided with a plurality of facilities 91 to 97 having individual themes in a large site 90, using facilities desired by the user, taking pictures in or near the facilities, and taking video Is something that can be done.

  The imaging device 70 in FIG. 5 is a digital video camera, a digital still camera, a mobile phone terminal with a camera, or the like, and the server system on the service providing side is one in or outside the site 90 or a facility that is a hot spot. It is provided for each of 91 to 97, and includes a transmission / reception antenna 81, a service providing server 82, and a content database 83.

  The content database 83 stores a large number of image files and audio files that are sub-contents. The image in this case is an animation image, a character image, or the like, and the sound is music or a voice of a character. A plurality of images suitable for each facility 91 to 97 are prepared. Further, the above-described sub-content content metadata or feature information is added to each image file and audio file.

  In the imaging device 70, a subject is photographed by the camera unit 71, and photographed image data is obtained as main content from the camera unit 71. The captured image data is processed by the image processing unit 72 and recorded on the recording medium 74 by the writing / reading control unit 73. The recording medium 74 is, for example, a removable recording medium attached to the imaging device 70, but may be a recording medium such as a semiconductor memory built in the imaging device 70.

  The captured image data recorded on the recording medium 74 is read from the recording medium 74 by the writing / reading control unit 73, processed by the image processing unit 72, and the captured image is displayed on the display unit 75 such as a liquid crystal display unit. .

  Further, at the time of shooting or immediately after shooting, the shot image data is sent to the main content content metadata generation unit 43 through the image processing unit 72, and the main content content metadata generation unit 43 generates the main content content metadata. 42.

  On the other hand, at the time of shooting or immediately after shooting, as in the example of FIG. 1 or 4, in the user situation metadata generation unit 41, the key input information input from the key input unit 31 via the interface 32 and the interface from the biometric sensor 33. From the biometric information input via 34 and the information read from the external storage medium 35 and input via the interface 36, user status metadata characterizing the user status is generated, and the recommended information generation unit 42 Is sent out.

  In this example, the imaging device 70 includes a GPS (Global Positioning System) antenna 78 and a GPS positioning unit 79, receives radio waves from GPS satellites, processes the data, and measures the position of the imaging device 70. The position information of the measurement result is sent to the recommendation information generation unit 42 at the time of shooting or immediately after shooting.

  The GPS positioning information in this case can be main content content metadata that characterizes the content of the captured image that is the main content, and can be a determination material for selecting the sub content.

  For example, when the user takes a picture in front of the facility 91 in FIG. 6, the facility 91 is shown in the photographed image, and the GPS positioning information indicates that the picture is taken near the facility 91. It becomes possible to select an image or sound suitable for the facility 91.

  The recommendation information generation unit 42 is the main content from the user situation metadata from the user situation metadata generation unit 41, the main content content metadata from the main content content metadata generation unit 43, and the GPS positioning information from the GPS positioning unit 79. Recommendation information for recommending sub-contents to be combined with the photographed image is generated.

  In this example, the recommendation information is transmitted from the transmission / reception unit 76 through the antenna 77 to the server system on the service providing side.

  In the server system on the service providing side, the recommendation information is received by the service providing server 82 via the antenna 81. In this example, the feature information is obtained from a large number of image files stored in the content database 83 as sub-contents. The image file whose information is closest to the received recommendation information is selected and transmitted from the service providing server 82 to the imaging device 70 through the antenna 81.

  In the imaging device 70, the provided image data is received by the transmission / reception unit 76 through the antenna 77 and sent to the image processing unit 72.

  In the image processing unit 72, the provided image as the sub-content is combined with the captured image as the main content. As a combination mode, as shown in FIG. 7, the sub content image (provided image) Ps is combined with the main content image (captured image) Pm according to user settings, and the combined image is recorded as one image file on the recording medium 74. The imaging device 70 can select one of a synthesis mode recorded on the recording medium 74 and a link mode in which the main content image and the sub-content image are linked by a related identifier and recorded on the recording medium 74 as separate image files. Configure.

  The above is a case where the provided image as the sub-content is combined with the captured image as the main content, but the imaging device can select either the mode in which the sub-content is combined or the mode in which the sub-content is not combined depending on the user setting. 70 may be configured.

  The above is the case where the sub-content is also an image, but as described above, the sub-content may be sound such as music or voice of a character. In that case, as shown in FIG. 8, the image file Fp as the main content and the audio file Fa as the sub-content are linked by the related identifier and recorded on the recording medium 74.

It is a figure which shows an example of the content processing apparatus of this invention. It is a figure which shows an example of the MPEG stream recorded on an optical disk. It is a figure which shows an example of the reproduction | regeneration processing in the content processing apparatus of FIG. It is a figure which shows the other example of the content processing apparatus of this invention. It is a figure which shows an example of the content processing system in the case of recording a content. It is a figure which shows an example of the facility where the content processing system of FIG. 5 is used. It is a figure which shows the case where a main content image and a sub content image are synthesize | combined. It is a figure which shows the case where a main content image file and a sub content audio | voice file are linked.

Explanation of symbols

  Since all the main parts are described in the figure, they are omitted here.

Claims (18)

And the main content content metadata characterizing the content of the main content, and a user profile Oyo Biyu chromatography The status metadata, and recommendation information generation step of generating recommendation information for recommending sub content to be reproduced along with the main content ,
A feature information generating step for generating feature information indicating the feature of the sub-content from the sub-content content metadata characterizing the content of the sub-content;
A sub-content selection step of selecting, from among a plurality of sub-contents, a sub-content whose feature information is closest to the recommendation information as a sub-content to be reproduced together with the main content;
And the user situation metadata is data indicating physical or mental state of the user at the time when the recommendation information is generated . The content processing method according to claim 1,
A content processing method for reproducing together the main content and the sub-content selected in the sub-content selection step as sub-content to be reproduced together with the main content. The content processing method according to claim 1,
A content processing method in which the main content and the sub-content selected in the sub-content selection step as sub-content to be reproduced together with the main content are combined and recorded, or recorded in association with each other. The content processing method according to claim 1,
A content processing method for analyzing the main content and generating the main content content metadata in a content processing apparatus on a user side. The content processing method according to claim 1,
The content processing method for fetching the main content content metadata into the user-side content processing apparatus from outside the user-side content processing apparatus. The content processing method according to claim 1,
A content processing method for analyzing the sub-content and generating the sub-content content metadata in a user-side content processing apparatus. The content processing method according to claim 1,
The content processing method according to claim 1, wherein the sub-content content metadata is taken into the user-side content processing apparatus from outside the user-side content processing apparatus. The content processing method according to claim 1,
A content processing method for executing the recommendation information generation step, the feature information generation step, and the sub-content selection step in a user-side content processing apparatus. The content processing method according to claim 1,
In the content processing device on the user side, the recommendation information generation step is executed, and the recommendation information generated in the recommendation information generation step is transmitted to a service providing side device outside the content processing device on the user side, A content processing method for executing the feature information generation step and the sub-content selection step in the service-providing device and transmitting the sub-content selected in the sub-content selection step to the content processing device on the user side . The content processing method according to claim 1,
In the service providing device external to the user-side content processing apparatus, the main content content metadata and the user situation metadata or the user situation metadata are acquired from the user-side content processing apparatus, and the recommendation information A content processing method that executes a generation step, the feature information generation step, and the sub-content selection step, and transmits the sub-content selected in the sub-content selection step to the content processing apparatus on the user side. And the main content content metadata characterizing the content of the main content, and a user profile Oyo Biyu chromatography The status metadata, and recommendation information generation means for generating recommendation information for recommending sub content to be reproduced along with the main content ,
Feature information generating means for generating feature information indicating the feature of the sub-content from the sub-content content metadata characterizing the content of the sub-content;
Sub-content selection means for selecting, from among a plurality of sub-contents, a sub-content whose feature information is closest to the recommendation information as a sub-content to be reproduced together with the main content;
And the user situation metadata is data indicating a physical or mental state of the user at the time when the recommendation information is generated . The content processing apparatus according to claim 11, wherein
A content processing apparatus that reproduces together the main content and the sub-content selected by the sub-content selection unit as the sub-content to be reproduced together with the main content. The content processing apparatus according to claim 11, wherein
A content processing apparatus that synthesizes and records the main content and the sub-content selected by the sub-content selection unit as sub-content to be reproduced together with the main content, or records the sub-content in association with each other. The content processing apparatus according to claim 11, wherein
A content processing apparatus comprising main content content metadata generating means for analyzing the main content and generating the main content content metadata. The content processing apparatus according to claim 11, wherein
A content processing apparatus comprising main content content metadata acquisition means for fetching the main content content metadata from outside the content processing apparatus into the content processing apparatus. The content processing apparatus according to claim 11, wherein
A content processing apparatus comprising sub-content content metadata generating means for analyzing the sub-content and generating the sub-content content metadata. The content processing apparatus according to claim 11, wherein
A content processing apparatus comprising sub-content content metadata acquisition means for fetching the sub-content content metadata from outside the content processing apparatus into the content processing apparatus. And the main content content metadata characterizing the content of the main content, and a user profile Oyo Biyu chromatography The status metadata, and recommendation information generation means for generating recommendation information for recommending sub content to be reproduced along with the main content ,
Transmitting means for transmitting the recommendation information generated by the recommendation information generating means to the device on the service providing side;
In the service providing device, the feature information indicating the feature of the content is selected as the content closest to the recommendation information transmitted by the transmitting unit, and the receiving unit receives the transmitted sub-content.
And the user situation metadata is data indicating a physical or mental state of the user at the time when the recommendation information is generated .

Images