JP4776179B2 - Timing determining apparatus and timing determining method - Google Patents

Description

  The present invention relates to a timing determination apparatus and a timing determination method for determining a timing for providing a service.

When providing a specific service to a user, it is necessary to provide the service at an appropriate timing in accordance with the user's preference and convenience. For this reason, for example, when the user moves with time, a technique for adjusting the timing of providing a communication service based on the position information of the user has been disclosed (see, for example, Patent Document 1).
JP 2003-85382 A

  As described above, there is a technology that provides a service based on location information. However, when a user receives and plays back video or music, the user can select an appropriate one according to the content of the video or music. You may wish to enjoy a good service.

 In view of the above problems, an object of the present invention is to provide a timing determination device and a timing determination method that appropriately determine the timing for providing a service based on the content of a medium being played.

  In order to achieve the above object, the first feature of the present invention is: (a) a feature amount of media, which is encoded data that can be reproduced, and fitness indicating a degree suitable for providing a specific service; (B) fitness storage means for accumulating correspondences, and (c) providing specific services according to the fitness values stored in the fitness storage means. The gist of the present invention is that it is a timing determination device including timing determination means for determining the timing to perform.

  According to the timing determination device according to the first feature, by comparing the feature amount of the media with the adaptability, it is possible to appropriately determine the timing for providing the service based on the content of the media being played.

  The timing determination device according to the first feature may further include fitness prediction means for predicting future fitness based on past fitness stored in the fitness storage and media feature amount. According to this timing determination device, it is possible to appropriately determine the timing for providing a service even for media for which the fitness cannot be calculated in advance.

  In addition, the fitness determining means in the timing determining device according to the first feature includes a feature quantity extracting means for extracting a media feature quantity, a media playback means for playing back the media, and a timing for providing a specific service for the media. May include an adaptability acquisition unit that acquires the evaluation result in the above, a correspondence creation unit that associates the feature amount of the media reproduced from the media playback unit with the evaluation result acquired from the adaptability acquisition unit. According to this timing determination device, it is possible to accurately associate media feature amounts with fitness.

  The second feature of the present invention is the step of (a) determining a correspondence relationship between a feature amount of media, which is encoded data that can be reproduced, and fitness indicating a degree suitable for providing a specific service. And (b) a step of accumulating the correspondence, and (c) a step of determining a timing for providing a specific service according to the value of the fitness stored in the fitness storage means. It is a summary.

  According to the timing determination method according to the second feature, it is possible to appropriately determine the timing for providing the service based on the content of the media being played by comparing the feature amount of the media with the fitness.

 According to the present invention, it is possible to provide a timing determination apparatus and a timing determination method that appropriately determine the timing for providing a service based on the content of a medium being played.

  Next, embodiments of the present invention will be described with reference to the drawings. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals. However, it should be noted that the drawings are schematic.

(First embodiment)
In the first embodiment, the timing of service movement is determined in a service that moves media being played on a communication terminal (media receiving apparatus) to another communication terminal (media receiving apparatus). In the first embodiment, a case will be described in which feature amounts for a medium being played are already accumulated in a timing determination device (static timing determination method).

  As shown in FIG. 1, the timing determination system according to the first embodiment includes a timing determination device 10, a media transmission device 20, and media reception devices 30 and 31. Here, it is assumed that a service for moving media currently being played by the media receiver 30 to the media receiver 31 at an appropriate timing is performed. For example, a mobile phone or a PHS is used as the media receivers 30 and 31, but any device having a media playback function and a media reception function, which will be described in detail later, may be used.

  The media transmission device 20 includes a media storage unit 251 that stores media, and a media transmission unit 201 that transmits media to the timing determination device 10 and the media reception devices 30 and 31. Here, “media” refers to reproducible encoded data such as video data and audio data.

  The media reception devices 30 and 31 are media reception units 301 and 311 that receive media from the media transmission device 20, media playback units 302 and 312 that reproduce media, and connection between the timing determination device 10 and the media transmission device 20. Trigger receiving units 303 and 313 for receiving triggers (commands) for performing connection / disconnection are provided. Here, the “connection” is a logical communication path established between devices. For example, the connection is identified by a transmission source IP address and a transmission source port number, and a transmission destination IP address and a transmission destination port number.

  As illustrated in FIG. 1, the timing determination device 10 includes a feature amount extraction unit 103, an fitness determination unit 107, an fitness storage unit 151, a timing determination unit 101, and a trigger transmission unit 102.

  The feature amount extraction unit 103 extracts the feature amount of the media that is received from the media transmission device 20 and is currently played back by the media reception device 30. The feature amount is divided into a low-level feature amount calculated from a measured value of the media and a high-level feature amount that captures a semantic aspect of the media. Low-level feature amounts include, for example, when the media is video data, the entire frame or subject color, texture, shape, motion (eg, motion vector), region, etc., and when the media is music data, Size, pitch, spectrum analysis signal (for example, inverse Fourier transform of logarithm of short-time amplitude spectrum). As the high-level feature quantity, for example, a semantic aspect of MPEG-7 can be used (see ISO / IEC 15938-5 “Part5-Multimedia Description Scheme”). Semantic aspects are related to related information in the media, can describe the relationship between people and objects appearing in the video, and even if it is difficult to search with low-level features, abstract search at the semantic level is possible And For example, an abstract search “a picture of a happy couple giving a ceremony in a beautiful church” becomes possible. The feature amount will be described in detail later.

  The fitness determining unit 107 determines the correspondence between the feature amount of the media stored in the media storage unit 251 and the fitness, and stores this correspondence in the fitness storage unit 151. Here, the “adaptability” indicates a degree suitable for providing a specific service. Specific services include terminal-to-terminal service movement, advertisement insertion service, actuator-linked service, and the like. A method for determining a specific fitness will be described in detail later. In the first embodiment, it is assumed that the fitness determination process is performed in advance before the media is transmitted to the media reception device 30.

  The fitness storage unit 151 stores the relationship between the feature amount of media and the fitness in association with each other.

  The timing determination unit 101 compares the fitness value stored in the fitness storage unit 151 with the feature value of the media currently being played back in the media reception device 30 extracted from the feature value extraction unit 103. Determine when to provide a specific service. Specifically, the timing determination unit 101 may determine the timing by determining whether or not the fitness stored in the fitness storage unit 151 exceeds a certain value (threshold). For example, when the fitness value is represented by a numerical value from 0 to 1, it is determined that a value exceeding 0.8 is a suitable timing for providing the service, and in other cases, the service is provided. It is determined that it is inappropriate to do so.

  Further, the timing determination unit 101 transmits a trigger to the trigger reception units 303 and 313 of the media reception devices 30 and 31 via the trigger transmission unit 102 at a timing determined to be appropriate. Here, the timing for providing the service needs to be determined in consideration of the time at which the trigger was previously transmitted. For example, every time the fitness level rises or falls in the vicinity of the threshold value, a service is exchanged between the media receiving devices 30 and 31 (chattering) if a trigger is transmitted. In order to prevent this, it is necessary that a certain period of time is not considered suitable for service provision even if the threshold is exceeded, or that a certain period of time is deemed inappropriate for service provision even if the threshold is exceeded. Although this fixed time varies depending on the target service, it is preferable that the service movement is about 2 to 5 seconds.

  The trigger transmission unit 102 transmits a command to disconnect the connection with the media transmission device 20 to the trigger reception unit 303 of the media reception device 30 according to the determination of the timing determination unit 101, and the media to the trigger reception unit 3113 of the media reception device 31. A command to connect a connection with the transmission device 20 is transmitted. As a protocol between the trigger transmitting unit 102 and the trigger receiving units 303 and 313, the Internet layer 7 (for example, HTTP, SIP, FTP) can be used. For example, a “connection command” or a “disconnect command” transmitted from the timing determination device 10 to the media reception devices 30 and 31 may be included in the payload.

  Further, the timing determination device 10 according to the first embodiment includes a processing control device (CPU), and includes a fitness determination unit 107, a feature amount extraction unit 103, a timing determination unit 101, and the like as modules. it can. These modules can be realized by executing a dedicated program for using a predetermined program language in a general-purpose computer such as a personal computer.

  The fitness storage unit 151 is a recording medium that stores the correspondence between feature amounts and fitness. Examples of the recording medium include RAM, ROM, hard disk, flexible disk, compact disk, IC chip, and cassette tape. According to such a recording medium, the fitness data can be easily stored, transported, sold, and the like.

  Although not shown, the timing determination device 10 includes a program holding unit that stores a program for causing a processing control device (CPU) to execute fitness determination processing, feature amount extraction processing, timing determination processing, and the like. Also good. The program holding unit is a recording medium such as a RAM, a ROM, a hard disk, a flexible disk, a compact disk, an IC chip, and a cassette tape. According to such a recording medium, the recognition program can be easily stored, transported, sold, and the like.

  Next, the timing determination method according to the first embodiment will be described with reference to FIG.

  (A) First, in step S101, the media reception device 30 transmits a connection request to the media transmission device 20. In step S102, a connection is established between the media transmission device 20 and the media reception device 30. Here, the connection is a logical communication path identified by the IP address and port number in the media transmission device 20 and the IP address and port number in the media reception device 30. After the connection is established, the media is transmitted from the media transmission device 20 to the media reception device 30, and the media reception device 30 reproduces the media.

  (B) Next, in step S103, the feature amount extraction unit 103 of the timing determination device 10 extracts the feature amount of the media transmitted from the media transmission device 20 to the media reception device 30. In step S <b> 104, the timing determination unit 101 of the timing determination apparatus 10 acquires the service provision fitness using the correspondence between the feature amount and the fitness stored in the fitness storage 151, and moves the service. Decide when to do it. For example, the timing determination unit 101 determines whether the fitness level exceeds a certain value.

  (C) When the fitness satisfies the condition, the timing determination device 10 transmits a connection disconnection command to the media reception device 30 via the trigger transmission unit 102 in step S105. When receiving the connection disconnection command from the timing determination device 10, the media reception device 30 transmits a connection disconnection request to the media transmission device 20 in step S106. Thereafter, in step S107, the connection between the media transmission device 20 and the media reception device 30 is disconnected.

  (D) Next, in step S <b> 108, the timing determination device 10 transmits a connection connection command to the media reception device 31 via the trigger transmission unit 102. Upon receiving the connection connection command from the timing determination device 10, the media reception device 31 transmits a connection connection request to the media transmission device 20 in step S109. Thereafter, in step S110, the connection between the media transmission device 20 and the media reception device 31 is connected. After the connection is established, the media is transmitted from the media transmission device 20 to the media reception device 31, and the media reception device 31 reproduces the media.

  As described above, according to the timing determination device 10 and the timing determination method according to the first embodiment, as shown in FIG. 3, when moving the medium being played to another terminal device, a highly adaptable timing is obtained. Can move the service.

  Next, the above-described feature amount will be described in more detail.

  First, a case where a motion vector is used as a feature amount of video data will be described. There are two methods for obtaining motion vectors: block matching method and gradient method (Hiroki Imamura, Yukiko Kenmochi, Ichiko Kotani, “Optical flow estimation considering changes in lighting conditions using 3D voting space”, Electronic Information Communication) Journal of academic papers, Vol.J85-D-II No.1 pp.12-22 2002). The block matching method can detect a motion with high accuracy but has a large amount of processing. The gradient method has a relatively small amount of processing but has a low accuracy of detecting a motion. However, when a codec whose motion vector has been calculated in advance is used (for example, MPEG), processing for acquiring the feature amount from the inside of the medium is performed.

  When the block matching method is used, the sum of absolute values of motion vectors in each block can be handled as a video feature amount. Specifically, as shown in FIG. 4A, a video (frame) is divided into a plurality of blocks. Then, in which direction each block in a certain frame has moved in the next frame, specifically, as shown in FIG. 4B, in each block, up and down at a certain time interval (for example, 100 ms). Calculate the amount of motion and the amount of left and right motion. At this time, the feature value is a set of feature values for all blocks.

  Further, a case where a pulse train is used as a feature amount of acoustic data will be described. In order to calculate the feature quantity of the pulse train, as shown in FIG. 5, a certain time interval (for example, 100 ms) is cut out as a frame. Then, the feature amount is calculated from the amplitude and the number of waves in the frame.

  The above-described calculation of the feature amount using the motion vector or the pulse train is an example, and it goes without saying that the feature amount may be calculated using other data.

  Next, the method for determining the fitness described in the fitness determining unit 107 will be described in more detail. Examples of the fitness determination method include a method using function approximation and a method using subjective evaluation.

First, a method using function approximation will be described. As for the adaptability when providing a service, for example, when moving a service, consider a case where the adaptability is higher when the motion vector of the video is larger and the sound is louder. Here, if the feature quantity of the motion vector mv (moving vector) of the video at a certain time t is C _mv and the feature quantity of the acoustic magnitude sv (sound vector) is C _sv , the fitness F (t) at the time t Is
F (t) = C _mv (t) + C _sv (t) ...... Formula (1)
It becomes. Here, the feature amount C _x for a certain characteristic x may be normalized to a range of up to 1 from minimum zero. The time corresponds to each frame of video and each sample of sound. Further, when the sound volume is larger and the fitness is lower, the reciprocal may be used as follows.

F (t) = C _mv (t) + 1 / C _sv (t) ...... Formula (2)
In addition, the relationship between the previous fitness F (t) and the feature value C _x (t) related to the characteristic x is related only to information related to a certain time t, but a past feature value (for example, a feature one time before) It is also possible to formulate using the quantity C _x (t−1)) and the past fitness (for example, fitness F (t−1) one time before). For example, when using the feature amount C _x (t−1) one hour before, the following is performed.

  Here, * represents all feature quantities.

Equation (3) is an example. When calculating the current fitness F (t) regardless of the past fitness F (t-1), an equation that does not use F (t-1) is used. When calculating the current fitness F (t) regardless of the past feature quantity C _x (t−1), an expression that does not use C _x (t−1) may be used. Moreover, you may combine the result by a some defining formula. For example, an arithmetic average may be used as follows.

F (t) = {F ₁ (t) + F ₂ (t)} / 2 …… Equation (4)
Here, for example, Formula (3) can be used for F ₁ (t).

In addition, when the media used for determining the relationship between the feature quantity and the fitness level is the same as the media used for timing determination, the future feature quantity (for example, the feature quantity C _x (t + 1) one time ahead) It is also possible to formulate using future fitness (for example, fitness F (t + 1) one time ahead). For example, when the feature amount C _x (t + 1) one hour before is used, it is as follows.

  Next, as a method for determining fitness, a method using subjective evaluation will be described. As shown in FIG. 6, the subjective evaluation device 50 (adaptation level determination unit 107) used for subjective evaluation includes a feature amount extraction unit 501, a media playback unit 502, an fitness level acquisition unit 503, and a correspondence relationship creation unit 504. Is provided. The feature amount extraction unit 501 extracts the feature amount of the media acquired from the media storage unit 251. The media playback unit 502 plays back the media and presents it to the user 40. The fitness level acquisition unit 503 acquires the evaluation result of the user 40 regarding the timing of providing a specific service for the reproduced media. For example, when performing a service for moving a medium being played on a communication terminal to another communication terminal, the user 40 evaluates a timing suitable for moving the service while the medium is being played. At this time, for example, the user 40 evaluates the designated timing in five stages, and inputs the numbers from 1 to 5 to the fitness level acquisition unit 503 as evaluation values. The correspondence creation unit 504 associates the feature amount of the media played from the media playback unit 502 with the user evaluation value acquired from the fitness acquisition unit 503, and stores them in the fitness storage unit 151.

  Here, there are a single stimulation method and a double stimulation method as methods standardized by the ITU-R recommendation (ITU-R Recommendation BT.500-7, “Methodology for the subjective assessment”). of the quality of television pictures, "p. 582-604, 1995.). The single stimulus method is a method of prompting evaluation by giving one stimulus for each trial (for example, timing of moving a service). The double stimulation method is a method for promoting evaluation by comparing a reference stimulus with a comparison stimulus for each trial. The double stimulation method has higher accuracy than the single stimulation method, but has a problem that the burden on the user is increased because the test time is long.

  The evaluation values include quality values and deterioration values standardized in the ITU-R recommendation (ITU-R Recommendation BT.500-7, “Methodology for the subjective assessment of the quality of television pictures,” p. 582-604, 1995.). The quality value is a five-level scale (5: good, 4: slightly good, 3: the same, 2: slightly bad, 1: bad) representing the degree of goodness. The deterioration value is a five-level scale (5: cannot be detected, 4: can be detected but is not worried, 3: worried a little, 2: worried, 1: very worried). These evaluation values may be normalized to a value from 0 to 1 when stored in the fitness storage unit 151.

  In addition, when it is difficult to obtain feature amounts for all images and sounds, the correspondence between feature amounts and fitness may be obtained in consideration of the types of images and sound types. For example, when the video data is classified into sports video, news video, drama video, etc., and the fitness is obtained for a new medium, the fitness may be obtained based on the fitness for the same type of data.

  For example, for a certain drama image, there is an image in which the correspondence between a certain feature amount a and the fitness b is obvious. In a case where the fitness b is obtained for an image showing a feature a ′ similar to the feature a, the fitness b can be obtained from the fitness b. For example, when it is desirable to increase the fitness level when the feature amount increases, the fitness level b ′ = b (a ′ / a), and when it is desirable to decrease the fitness level when the feature amount increases, the fitness level b ′. = B (a / a ').

  Next, the subjective evaluation method (fitness determination method) according to the first embodiment will be described with reference to FIG. Here, a single stimulus quality measure method will be described as an example.

  (A) First, in step S201, the feature amount extraction unit 501 acquires media from the media storage unit 251 and extracts feature amounts. Next, in step S <b> 202, the media playback unit 502 presents an evaluation stimulus regarding the media to the user 40. The user 40 evaluates how much the evaluation stimulus is felt by using the five quality values. For example, when requesting the user to evaluate the service movement timing, the service movement that moves the service between terminals in 0 seconds at a plurality of playback times in media (eg, 10 seconds, 20 seconds, and 30 seconds) is referred to as an evaluation stimulus. To do.

  (B) Next, in step S203, the fitness level acquisition unit 503 acquires the evaluation result of the user as the fitness level. The correspondence creation unit 504 acquires the fitness obtained by the fitness acquisition unit 503 and the feature quantity extracted by the feature quantity extraction unit 501 (step S204). In step S205, the correspondence between the feature quantity and the fitness is obtained. Create a relationship.

  (C) Next, in step S206, it is determined whether the trial has been repeated as many times as necessary. If the required number of times has not been repeated, the process returns to step S202. On the other hand, if the required number of repetitions has been repeated, the process proceeds to step S207, and the fitness storage unit 151 associates and stores the media feature quantity with the user evaluation value (fitness).

  If the order of the evaluation stimuli to be given is inferred by the user, the sense cannot be correctly evaluated due to the influence of preconceptions. Therefore, the evaluation stimuli in each trial may be selected at random and presented to the user.

  Moreover, although the fitness determination part 107 shown in FIG. 6 was demonstrated as an apparatus (subjective evaluation apparatus 50) used for the method using subjective evaluation, you may use it as an apparatus used for the method using function approximation. In this case, the fitness level acquisition unit 503 acquires the evaluation result based on the function approximation for the timing of providing a specific service for the played media. Then, the correspondence creation unit 504 associates the feature amount of the media reproduced from the media reproducing unit 502 with the evaluation value based on the function approximation obtained from the fitness obtaining unit 503 and accumulates it in the fitness accumulating unit 151.

  According to the timing determination device 10 and the timing determination method according to the first embodiment, the timing of providing a service is appropriately determined based on the content of the media being played by comparing the feature amount of the media with the fitness. Can be determined. Specifically, media being played on a communication terminal can be moved to another communication terminal device at an appropriate timing.

  In addition, the fitness determining unit 107 uses the method using function approximation or the method using subjective evaluation, so that the feature amount of the media reproduced from the media reproducing unit 502 and the evaluation acquired from the fitness acquiring unit 503 are used. The result can be associated. For this reason, it is possible to accurately associate the feature amount of the media with the fitness.

(Second Embodiment)
In the first embodiment, the feature amount for the medium being reproduced has already been accumulated (static timing determination method). However, in the second embodiment, the feature amount has not been accumulated, and the media is reproduced. At the same time, the case where it is necessary to predict the feature amount (dynamic timing determination method) will be described. In the second embodiment, as in the first embodiment, in the service of moving the media being played on the communication terminal (media receiving apparatus) to another communication terminal (media receiving apparatus) Determine the timing of movement.

  As shown in FIG. 8, the timing determination system according to the second embodiment includes a timing determination device 10, a media transmission device 20, and media reception devices 30 and 31. Here, it is assumed that a service for moving media currently being played by the media receiver 30 to the media receiver 31 at an appropriate timing is performed. For example, a mobile phone or a PHS is used as the media receivers 30 and 31, but any device having a media playback function and a media reception function, which will be described in detail later, may be used.

  Since the media transmission device 20 is the same as that of the first embodiment except that the media transmission unit 201 transmits media only to the timing determination device 10, description thereof is omitted here.

  In the media receiving devices 30 and 31, media receiving units 301 and 311 receive media from the timing determining device 10, and trigger receiving units 303 and 313 receive triggers (commands) for connecting / disconnecting connections with the timing determining device 10. Since it is the same as that of 1st Embodiment except receiving from the timing determination apparatus 10, description is abbreviate | omitted here.

  As illustrated in FIG. 8, the timing determination device 10 includes a feature amount extraction unit 103, an fitness determination unit 107, an fitness storage unit 151, a timing determination unit 101, a trigger transmission unit 102, and a media reception unit 105. And a media transmission unit 106 and a fitness prediction unit 104.

  The media receiver 105 receives media from the media transmitter 20. The media transmission unit 106 transmits media to the media reception devices 30 and 31.

  The fitness prediction unit 104 predicts future fitness from the past fitness and the feature amount accumulated in the fitness accumulation unit 151. For example, assuming that the current fitness level is on a straight line between the fitness level 1 hour before and the fitness level 2 hours before, the prediction can be made by the line segment ratio. The fitness prediction method will be described in detail later.

  In addition, the feature amount extraction unit 103, the fitness determination unit 107, the fitness storage unit 151, the timing determination unit 101, and the trigger transmission unit 102 are the same as those in the first embodiment, and thus description thereof is omitted here. .

  Next, a timing determination method according to the second embodiment will be described with reference to FIG.

  (A) First, in step S301, the timing determination device 10 transmits a connection request to the media transmission device 20. In step S302, a connection between the timing determination device 10 and the media transmission device 20 is established.

  (B) Next, in step S303, the media reception device 30 transmits a connection request to the timing determination device 10. In step S304, a connection is established between the timing determination device 10 and the media reception device 30. After the connection is established, the media is transmitted from the timing determination device 10 to the media reception device 30, and the media reception device 30 reproduces the media.

(C) Next, in step S <b> 305, the feature amount extraction unit 103 of the timing determination device 10 extracts the feature amount of the media transmitted to the media reception device 30. In step S <b> 306, the fitness prediction unit 104 of the timing determination device 10 predicts a future fitness from the past fitness stored in the fitness storage unit 151. As a fitness prediction method, as shown in FIG. 10 (a), it is assumed that the current fitness is on a straight line between the fitness of 1 hour before and the fitness of 2 hours before, and prediction is performed by the line segment ratio. There is a way to do it. From fitness F (t-1) at time t-1 and fitness F (T-2) at time t-2, fitness F (t) at time t is
F (t) = 2F (t-1)-F (t-2) ...... Formula (6)
And can be calculated. As shown in FIG. 10 (b), there is also a method of prediction using the least square method and an exponential function (refer to Omura Taira, “Experiments and Evaluations-Data Analysis Torano Maki!”, Nikka Giren, 2003). . If it is difficult to accurately represent a phenomenon in which the fitness increases or decreases by the line segment ratio or the least square method, prediction may be performed using a polynomial as shown in FIG. it can.

  (D) Next, in step S307, the timing determination unit 101 of the timing determination apparatus 10 acquires the fitness level of service provision using the correspondence relationship between the fitness level and the feature amount predicted by the fitness level prediction unit 104. And determine the timing of service movement. For example, the timing determination unit 101 determines whether the fitness level exceeds a certain value.

  (E) When the fitness satisfies the condition, the timing determination device 10 transmits a connection disconnection command to the media reception device 30 via the trigger transmission unit 102 in step S308. When receiving a connection disconnection command from the timing determination device 10, the media reception device 30 transmits a connection disconnection request to the timing determination device 10 in step S309. Thereafter, in step S310, the connection between the timing determination device 10 and the media reception device 30 is disconnected. Further, when receiving a connection disconnection command from the timing determination device 10, the media transmission device 20 transmits a connection disconnection request to the timing determination device 10 in step S 311. Thereafter, in step S312, the connection between the timing determination device 10 and the media transmission device 20 is disconnected.

  (F) Next, when receiving a connection connection command from the timing determination device 10, the media transmission device 20 transmits a connection connection request to the timing determination device 10 in step S314. Thereafter, in step S315, the connection between the timing determination device 10 and the media transmission device 20 is connected. In step S 313, the timing determination device 10 transmits a connection connection command to the media reception device 31 via the trigger transmission unit 102. Upon receiving the connection connection command from the timing determination device 10, the media reception device 31 transmits a connection connection request to the timing determination device 10 in step S316. Thereafter, in step S317, the connection between the timing determination device 10 and the media reception device 31 is connected. After the connection is established, the media is transmitted from the media transmission device 20 to the media reception device 31, and the media reception device 31 reproduces the media.

  According to the timing determination device 10 and the timing determination method according to the second embodiment, the future fitness can be predicted from the past fitness stored in the fitness storage 151 and the feature amount of the media. For this reason, it is possible to appropriately determine the timing for providing the service even for media for which the fitness cannot be calculated in advance. Specifically, media being played on a communication terminal can be moved to another communication terminal device at an appropriate timing.

(Third embodiment)
In the first and second embodiments, the timing for moving the media being played on the communication terminal (media receiving device) to another communication terminal (media receiving device) is determined. In the third embodiment, The timing for inserting an advertisement into the media being played back by the communication terminal (media receiving device) is determined.

  It is conceivable to insert sponsor information as content provider information in the middle of the streaming video. The user typically does not want to continue to view streaming information and view information about the sponsor. At this time, if an advertisement is inserted in an inappropriate part of the streaming video, the user's concentration may be distracted, which may cause discomfort (Noriko Nagata, Sanae Wake, Mieko Osuga, Seiji Iguchi) , "Effect of TV commercial insertion timing on attention concentration", IEICE Tech. Reports, HCS 2003-10, 2003).

  The timing determination system according to the third embodiment includes a timing determination device 10, a media transmission device 20, and a media reception device 30, as shown in FIG. Here, it is assumed that a service for inserting an advertisement at an appropriate timing is performed in the middle of media (streaming video) currently being played by the media receiver 30. For example, a mobile phone or a PHS is used as the media receiver 31, but any device having a media playback function and a media reception function, which will be described in detail later, may be used.

  The media transmission device 20 includes a media storage unit 251 that stores media, a media transmission unit 201 that transmits media to the timing determination device 10 and the media reception device 30, and an advertisement storage unit 252 that stores advertisements.

  The media transmission unit 201 transmits the media stored in the media storage unit 251 or the advertisement stored in the advertisement storage unit 252 in response to a connection command from the media reception device 30.

  The media receiving device 30 includes a media receiving unit 301 that receives media from the media transmitting device 20, a media playback unit 302 that plays media, and a trigger that connects and disconnects the connection from the timing determination device 10 to the media transmitting device 20. A trigger receiving unit 303 for receiving (command). The media receiving unit 311 receives media or advertisements according to the trigger.

  As illustrated in FIG. 11, the timing determination device 10 includes a feature amount extraction unit 103, an fitness storage unit 151, a timing determination unit 101, and a trigger transmission unit 102.

  Since the timing determination apparatus 10 is the same as that of the first embodiment except that the trigger transmission unit 102 transmits a trigger (command) at the timing of inserting an advertisement, description thereof is omitted here.

  Further, the fitness determining unit 107 described in the first embodiment is not shown here, but also in the timing determining device 10 according to the third embodiment, the fitness determining unit 107 according to need. Is provided.

  Next, a timing determination method according to the third embodiment will be described with reference to FIG.

  (A) First, in step S401, the media reception device 30 transmits a connection request to the media transmission device 20. In step S402, a connection is established between the media transmission device 20 and the media reception device 30. After the connection is established, the media is transmitted from the media transmission device 20 to the media reception device 30, and the media reception device 30 reproduces the media.

  (B) Next, in step S403, the feature amount extraction unit 103 of the timing determination device 10 extracts the feature amount of the media transmitted from the media transmission device 20 to the media reception device 30. In step S <b> 404, the timing determination unit 101 of the timing determination apparatus 10 acquires the service provision fitness using the correspondence relationship between the feature value and the fitness stored in the fitness storage 151, and inserts the advertisement. Decide when to do it. For example, the timing determination unit 101 determines whether the fitness level exceeds a certain value.

  (C) When the fitness satisfies the condition, the timing determination device 10 transmits a connection disconnection command to the media reception device 30 via the trigger transmission unit 102 in step S405. When receiving the connection disconnection command from the timing determination device 10, the media reception device 30 transmits a connection disconnection request to the media transmission device 20 in step S406. Thereafter, in step S407, the connection between the media transmission device 20 and the media reception device 30 is disconnected.

  (D) Next, in step S <b> 408, the timing determination device 10 transmits an advertisement connection command to the media reception device 30 via the trigger transmission unit 102. When receiving the connection connection command from the timing determination device 10, the media reception device 30 transmits an advertisement connection request to the media transmission device 20 in step S409. Thereafter, in step S410, the connection between the media transmission device 20 and the media reception device 30 is connected. Then, the media receiving device 30 reproduces the advertisement.

  (E) Next, when a predetermined time (for example, 30 seconds) elapses after the advertisement is reproduced, the timing determination device 10 transmits an advertisement disconnection command to the media reception device 30 in step S411. When receiving the advertisement disconnection command from the timing determination device 10, the media reception device 30 transmits an advertisement disconnection request to the media transmission device 20 in step S 412. Thereafter, in step S413, the connection between the media transmission device 20 and the media reception device 30 is disconnected. Here, although the timing determination device 10 transmits a disconnection command, the media reception device 30 does not wait for a command from the timing determination device 10 and, for example, when a certain time has passed, the connection with the media transmission device 20 It is good also as cutting.

  Thus, according to the timing determination device 10 and the timing determination method according to the third embodiment, as shown in FIG. 13, when inserting an advertisement, the advertisement can be inserted at a highly adaptable timing. .

(Fourth embodiment)
In the fourth embodiment, a system for determining a timing for interlocking an actuator device in accordance with reproduction of media in a media receiving device will be described.

  In the future ubiquitous society, it is assumed that a plurality of devices cooperate to provide services. For example, when playing video or sound such as on a television, the presence of a higher level of realism can be achieved by linking the actuator devices in the room (for example, lights and air conditioners) according to the video and sound features. it can. An example in which the timing determination device 10 is used for such a service will be described.

  As shown in FIG. 14, the timing determination system according to the fourth embodiment includes a timing determination device 10, a media transmission device 20, a media reception device 30, and an actuator device 33. Here, it is assumed that a service for operating the actuator device at an appropriate timing in the middle of the media (streaming video) currently being played by the media receiving device 30 is provided.

  The media transmission device 20 includes a media storage unit 251 that stores media, and a media transmission unit 201 that transmits media to the timing determination device 10 and the media reception device 30.

  The media receiver 30 includes a media receiver 301 that receives media from the media transmitter 20 and a media playback unit 302 that plays back the media.

  As illustrated in FIG. 14, the timing determination device 10 includes a feature amount extraction unit 103, an fitness storage unit 151, a timing determination unit 101, and a trigger transmission unit 102.

  The trigger transmission unit 102 transmits a trigger (command) to the actuator device 33 at the timing of turning on / off the illumination and raising / lowering the room temperature.

  In addition, the fitness storage unit 151 stores a correspondence relationship between the feature amount and the motion fitness of the actuator device 33.

  Since the feature quantity extraction unit 103 and the timing determination unit 101 are the same as those in the first embodiment, description thereof is omitted here.

  Further, the fitness determining unit 107 described in the first embodiment is not shown here, but also in the timing determining device 10 according to the fourth embodiment, the fitness determining unit 107 according to need. Is provided.

  The actuator device 33 includes: a trigger receiving unit 331 that receives a trigger (command) for turning on / off the lighting and raising / lowering the room temperature from the timing determining device 10; an illumination adjusting unit 332 that controls turning on / off the illumination; And a room temperature adjustment unit 333 that controls the rise and fall of the room temperature.

  Next, a timing determination method according to the fourth embodiment will be described with reference to FIG.

  (A) First, in step S501, the media reception device 30 transmits a connection request to the media transmission device 20. In step S502, a connection is established between the media transmission device 20 and the media reception device 30. After the connection is established, the media is transmitted from the media transmission device 20 to the media reception device 30, and the media reception device 30 reproduces the media.

  (B) Next, in step S503, the feature amount extraction unit 103 of the timing determination device 10 extracts the feature amount of the media transmitted from the media transmission device 20 to the media reception device 30. In step S <b> 504, the timing determination unit 101 of the timing determination device 10 acquires the service provision fitness using the correspondence relationship between the feature value and the fitness stored in the fitness storage 151, and turns on the illumination. Determine the timing.

  (C) The timing determination device 10 transmits an illumination lighting command to the actuator device 33 via the trigger transmission unit 102 in step S505 when the fitness satisfies the condition for lighting the illumination. . When receiving the illumination lighting command from the timing determination device 10, the actuator device 33 lights the illumination in step S506, and transmits an illumination lighting completion signal to the timing determination device 10 in step S507.

  (D) Next, when the fitness satisfies the condition for turning off the illumination, the timing determining device 10 instructs the actuator device 33 to turn off the illumination via the trigger transmission unit 102 in step S508. Send. When receiving the illumination turn-off command from the timing determination device 10, the actuator device 33 turns off the illumination in step S 509, and transmits an illumination turn-off completion signal to the timing determination device 10 in step S 510.

  (E) Next, the timing determination device 10 instructs the actuator device 33 to increase the room temperature via the trigger transmission unit 102 in step S511 when the condition for the fitness to increase the room temperature is satisfied. Send. Next, when the actuator device 33 receives the room temperature increase command from the timing determination device 10, the actuator device 33 increases the room temperature in step S 512, and transmits a room temperature increase completion signal to the timing determination device 10 in step S 513.

  (F) Next, when the fitness satisfies the condition for lowering the room temperature, in step S514, the timing determination device 10 instructs the actuator device 33 to lower the room temperature via the trigger transmission unit 102. Send. Next, when the actuator device 33 receives a room temperature lowering instruction from the timing determining device 10, the actuator device 33 lowers the room temperature in step S 515, and transmits a room temperature lowering completion signal to the timing determining device 10 in step S 516.

  Further, whether or not the above-described fitness satisfies each condition is determined by referring to an adaptation pattern table shown in FIG. 16, for example. In FIG. 16, feature amounts for temperature adjustment are associated with each other. Pattern 1 indicates that the temperature is controlled to be high (here, 27 degrees) when the medium is described as climax as metadata (high-level feature value). Pattern 2 indicates that when the motion vector of the video is equal to or less than the threshold, it is determined that the motion is small, and the temperature is controlled to be slightly lower (here, 25 degrees). Pattern 3 indicates that when the motion vector of the video is equal to or greater than the threshold (or when there is no sound), it is determined that the motion is intense and the temperature is controlled to be slightly higher (here, 26 degrees). Such an adaptation pattern table is accumulated in the fitness accumulation unit 151, and the actuator device 33 is controlled based on this pattern.

  According to the timing determination device 10 and the timing determination method according to the fourth feature, for example, when the video screen approaches a mountain, the light is changed drastically, and when the scene is not a mountain, the light is changed slowly. it can. In addition, for example, when a terrible video is being played, the room air can be lowered, and when a calm video is being played, the room air can be raised.

  In this way, by linking the actuator device 33 in accordance with the image, it is possible to further enhance the sense of reality.

(Fifth embodiment)
In the fourth embodiment, there is an actuator device separately from the media receiving device, and the actuator device is operated according to the fitness. In the fifth embodiment, the media receiving device has an actuator function. Will be described.

  Currently, a plurality of actuator devices are mounted on portable terminals such as notebook PCs, PDAs, and cellular phones. For example, a display, a speaker, a vibrator, and the like can be cited, and by linking these actuator devices, it is possible to further enhance the realism of the communication service. Examples of communication services include streaming video distribution, voice phone calls, and video phone calls. An example in which the timing determination device 10 is used for such a service will be described.

  As illustrated in FIG. 17, the timing determination system according to the fifth embodiment includes a timing determination device 10, a media transmission device 20, and a media reception device 34. Here, it is assumed that a service for operating the actuator function in the media receiving device 34 is performed at an appropriate timing in the middle of the media (streaming video) currently being played by the media receiving device 30.

  The media transmission device 20 includes a media storage unit 251 that stores media, and a media transmission unit 201 that transmits media to the timing determination device 10 and the media reception device 30.

  The media reception device 34 includes a media reception unit 301 that receives media from the media transmission device 20, a media playback unit 302 that plays media, a trigger reception unit 331 that receives a trigger (command) for vibration, and a media reception device And a vibration adjusting unit 332 that vibrates 34.

  As illustrated in FIG. 17, the timing determination device 10 includes a feature amount extraction unit 103, an fitness storage unit 151, a timing determination unit 101, and a trigger transmission unit 102.

  The trigger transmission unit 102 transmits a trigger (command) to the media reception device 34 at a timing to start / stop vibration.

  In addition, the fitness storage unit 151 stores the correspondence between the feature amount and the operation fitness of the media reception device 34.

  Since the feature quantity extraction unit 103 and the timing determination unit 101 are the same as those in the first embodiment, description thereof is omitted here.

  Further, the fitness determining unit 107 described in the first embodiment is not shown here, but the fitness determining unit 107 according to the fifth embodiment is also necessary in the timing determining device 10 according to the fifth embodiment. Is provided.

  Next, a timing determination method according to the fifth embodiment will be described with reference to FIG.

  (A) First, in step S601, the media reception device 34 transmits a connection request to the media transmission device 20. In step S 602, a connection is established between the media transmission device 20 and the media reception device 34. After the connection is established, the media is transmitted from the media transmission device 20 to the media reception device 34, and the media reception device 34 reproduces the media.

  (B) Next, in step S603, the feature amount extraction unit 103 of the timing determination device 10 extracts the feature amount of the media transmitted from the media transmission device 20 to the media reception device 34. In step S604, the timing determination unit 101 of the timing determination device 10 acquires the service provision fitness using the correspondence between the feature quantity and the fitness stored in the fitness storage 151, and vibrates. To decide.

  (C) The timing determination device 10 transmits a vibration start command to the media reception device 34 via the trigger transmission unit 102 in step S605 when the fitness satisfies the condition for starting vibration. To do. When receiving the vibration start command from the timing determination device 10, the media reception device 34 starts vibration in step S 606, and transmits a vibration start completion signal to the timing determination device 10 in step S 607.

  (D) Next, when the fitness satisfies the condition for stopping the vibration, the timing determination device 10 stops the vibration with respect to the media reception device 34 via the trigger transmission unit 102 in step S608. Send instructions. When receiving the vibration stop command from the timing determination device 10, the media reception device 34 stops the vibration in step S 609 and transmits a vibration stop completion signal to the timing determination device 10 in step S 610.

  Further, whether or not the above-described fitness satisfies the vibration start or stop condition is determined by referring to an adaptation pattern table shown in FIG. 19, for example. In FIG. 19, feature quantities for vibration adjustment are associated with each other. Pattern 1 indicates that the vibrator is not activated when the sound volume is equal to or less than the threshold value as the acoustic feature amount. Pattern 2 activates a vibrator when the motion vector is greater than or equal to a threshold value (eg, a screen with intense motion) as the video feature amount, or when the sound volume is greater than or equal to the threshold value (eg: a scene that swells) as the acoustic feature amount. It is shown that. Pattern 3 indicates that the vibrator is activated when the acoustic feature quantity includes a frequency equal to or higher than the threshold value. Such an adaptation pattern table is stored in the fitness storage unit 151, and the media receiver 34 is controlled based on this pattern.

  According to the timing determination device 10 and the timing determination method according to the fifth feature, for example, if the movement of the scene shows a certain amount of movement, the vibrator can be activated. Further, the vibrator can be activated according to the volume of voice.

  In this way, the presence of the media receiver 34 is interlocked with the video, so that a sense of reality can be further enhanced.

(Other embodiments)
Although the present invention has been described according to the above-described embodiments, it should not be understood that the descriptions and drawings constituting a part of this disclosure limit the present invention. From this disclosure, various alternative embodiments, examples and operational techniques will be apparent to those skilled in the art.

 For example, in the first to fifth embodiments, it has been described that the connection request and the disconnection request are transmitted from the media reception device 30, 31, 34 to the media transmission device 20, but the media reception device 30, 31 is transmitted from the media transmission device 20. , 34 may send a connection request and a disconnection request.

 In FIG. 2, the timing determination device 10 has been described as transmitting a connection command after transmitting a disconnect command (steps S105 and S108). However, the disconnection command may be transmitted after transmitting the connection command. The same applies to FIG.

 Further, although it has been described that the fitness determination unit 107, the feature amount extraction unit 103, and the timing determination unit 101 may be provided as modules in one CPU, they may be provided in different CPUs and different devices. In that case, a plurality of devices are connected by a bus or the like.

  As described above, the present invention naturally includes various embodiments not described herein. Therefore, the technical scope of the present invention is defined only by the invention specifying matters according to the scope of claims reasonable from the above description.

1 is a configuration block diagram of a timing determination device according to a first embodiment. It is a flowchart which shows the timing determination method which concerns on 1st Embodiment. It is a figure explaining the service movement timing which concerns on 1st Embodiment. It is a figure explaining the feature-value of the video data which concerns on the 1st-5th embodiment. It is a figure explaining the feature-value of the acoustic data which concerns on the 1st-5th embodiment. It is a block diagram of the subjectivity evaluation apparatus (fitness determination unit) according to the first to fifth embodiments. It is a flowchart which shows the subjective evaluation method (fitness determination method) which concerns on 1st-5th embodiment. It is a block diagram of the configuration of the timing determination device according to the second embodiment. It is a flowchart which shows the timing determination method which concerns on 2nd Embodiment. It is a figure explaining the fitness prediction method which concerns on 2nd Embodiment. It is a structure block diagram of the timing determination apparatus which concerns on 3rd Embodiment. It is a flowchart which shows the timing determination method which concerns on 3rd Embodiment. It is a figure explaining the advertisement insertion timing which concerns on 3rd Embodiment. It is a structure block diagram of the timing determination apparatus which concerns on 4th Embodiment. It is a flowchart which shows the timing determination method which concerns on 4th Embodiment. It is an example of the fitness pattern table which the fitness storage part which concerns on 4th Embodiment accumulate | stores. It is a block diagram of the configuration of the timing determination device according to the fifth embodiment. It is a flowchart which shows the timing determination method which concerns on 5th Embodiment. It is an example of the fitness pattern table | surface which the fitness storage part which concerns on 5th Embodiment accumulate | stores.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Timing determination apparatus 20 ... Media transmission apparatus 30, 31, 34 ... Media reception apparatus 33 ... Actuator apparatus 40 ... User 50 ... Subjective evaluation apparatus 101 ... Timing determination part 102 ... Trigger transmission part 103 ... Feature-value extraction part 104 ... Adaptation Degree prediction unit 105 ... Media reception unit 106 ... Media transmission unit 107 ... Adaptability determination unit 151 ... Adaptivity accumulation unit 201 ... Media transmission unit 251 ... Media accumulation unit 252 ... Advertisement accumulation unit 301, 311 ... Media reception unit 302, 312 ... Media playback unit 303, 313 ... Trigger reception unit 331 ... Trigger reception unit 332 ... Vibration adjustment unit 332 ... Lighting adjustment unit 333 ... Room temperature adjustment unit 501 ... Feature quantity extraction unit 502 ... Media playback unit 503 ... Adaptation degree acquisition unit 504 ... Correspondence creation department

Claims (6)

Fitness determining means for determining a correspondence relationship between a feature amount of media, which is encoded data that can be reproduced, and fitness indicating a degree suitable for providing a specific service;
Fitness storage means for storing the correspondence relationship;
Timing determining means for determining a timing at which the fitness value accumulated in the fitness accumulating means exceeds a threshold as timing for providing the specific service;
A trigger transmitter for transmitting a predetermined trigger at the timing determined by the timing determination means;
The specific service is a service movement for moving the encoded data from the first media receiving apparatus reproducing the encoded data transmitted from the media transmitting apparatus to the second media receiving apparatus,
The trigger transmission unit transmits a trigger for disconnecting the connection with the media transmission device to the first media reception device, and connects to the media transmission device with respect to the second media reception device. A timing determination apparatus characterized by transmitting a trigger for connecting the two. Fitness determining means for determining a correspondence relationship between a feature amount of media, which is encoded data that can be reproduced, and fitness indicating a degree suitable for providing a specific service;
Fitness storage means for storing the correspondence relationship;
Timing determining means for determining a timing at which the fitness value accumulated in the fitness accumulating means exceeds a threshold as timing for providing the specific service;
A trigger transmitter for transmitting a predetermined trigger at the timing determined by the timing determination means;
The specific service is an advertisement insertion service that inserts an advertisement into the encoded data that is transmitted from a media transmission device and reproduced by a media reception device;
The trigger transmission unit transmits a first trigger and a second trigger to the media reception device,
The first trigger is a trigger for transmitting a stop of transmission of the encoded data to the media transmission device,
The timing determination device, wherein the second trigger is a trigger for requesting the media transmission device to transmit advertisement data to the media reception device. And fitness in the past stored in the fitness storing means, the timing determination device according to claim 1 or 2, further comprising a fitness predicting means for predicting future fitness from the feature of the media . The fitness determining means includes
Feature amount extraction means for extracting the feature amount of the media;
Media playback means for playing back the media;
Fitness acquisition means for acquiring an evaluation result at a timing of providing the specific service for the media;
3. The timing determination device according to claim 1, further comprising: a correspondence creation unit that associates the feature amount of the media played from the media playback unit with the evaluation result acquired from the fitness level acquisition unit. . Determining a correspondence relationship between a feature amount of media, which is encoded data that can be reproduced, and fitness indicating a degree suitable for providing a specific service;
Accumulating the correspondences;
Determining a timing at which the accumulated fitness value exceeds a threshold as a timing for providing the specific service;
Transmitting a predetermined trigger at the determined timing, and
The specific service is a service movement for moving the encoded data from the first media receiving apparatus reproducing the encoded data transmitted from the media transmitting apparatus to the second media receiving apparatus,
In the step of transmitting the trigger, a trigger for disconnecting the connection with the media transmission device is transmitted to the first media reception device, and the media transmission device is transmitted to the second media reception device. A timing determination method characterized by transmitting a trigger for connecting the connection. Determining a correspondence relationship between a feature amount of media, which is encoded data that can be reproduced, and fitness indicating a degree suitable for providing a specific service;
Accumulating the correspondences;
Determining a timing at which the accumulated fitness value exceeds a threshold as a timing for providing the specific service;
Transmitting a predetermined trigger at the determined timing, and
The specific service is an advertisement insertion service that inserts an advertisement into the encoded data that is transmitted from a media transmission device and reproduced by a media reception device;
In the step of transmitting the trigger, a first trigger and a second trigger are transmitted to the media receiving device,
The first trigger is a trigger for transmitting a stop of transmission of the encoded data to the media transmission device,
The timing determination method, wherein the second trigger is a trigger for requesting the media transmission device to transmit advertisement data to the media reception device.

Images