WO2017038480A1 - Content processing device, content processing method, and program - Google Patents

Content processing device, content processing method, and program Download PDF

Info

Publication number
WO2017038480A1
WO2017038480A1 PCT/JP2016/074104 JP2016074104W WO2017038480A1 WO 2017038480 A1 WO2017038480 A1 WO 2017038480A1 JP 2016074104 W JP2016074104 W JP 2016074104W WO 2017038480 A1 WO2017038480 A1 WO 2017038480A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
output
buffer
skip
repeat
Prior art date
Application number
PCT/JP2016/074104
Other languages
French (fr)
Japanese (ja)
Inventor
真一 長尾
Original Assignee
ソニーセミコンダクタソリューションズ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ソニーセミコンダクタソリューションズ株式会社 filed Critical ソニーセミコンダクタソリューションズ株式会社
Publication of WO2017038480A1 publication Critical patent/WO2017038480A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk

Definitions

  • the present technology relates to a content processing device, a content processing method, and a program, and in particular, for example, a content processing device, a content processing method, and a content processing device that can prevent a user who views content from feeling uncomfortable.
  • a content processing device a content processing method, and a content processing device that can prevent a user who views content from feeling uncomfortable.
  • the program in particular, for example, a content processing device, a content processing method, and a content processing device that can prevent a user who views content from feeling uncomfortable.
  • a content processing apparatus that processes MPEG-encoded content skips output of a target frame, which is a frame to be output from among video frames included in the content, or repeatedly outputs a target frame. Is done.
  • skipping the output of the target frame is also called skipping
  • repeating the output of the target frame is also called repeating
  • Patent Document 1 proposes an MPEG decoder that displays a video frame at an appropriate timing by detecting a shift in the output timing of the video frame with respect to the system clock and performing skipping or repeating.
  • the present technology has been made in view of such a situation, and can prevent a user who views content from feeling uncomfortable.
  • the content processing apparatus or the program according to the present technology includes a timing control unit that controls an output timing for outputting content according to a detection result of a user's line of sight, or such a content processing apparatus. It is a program for causing a computer to function.
  • the content processing method of the present technology is a content processing method including controlling an output timing of outputting content according to a detection result of a user's line of sight.
  • the output timing for outputting the content is controlled according to the detection result of the user's line of sight.
  • the content processing device may be an independent device, or may be an internal block constituting one device.
  • the program can be provided by being transmitted through a transmission medium or by being recorded on a recording medium.
  • FIG. 10 is a flowchart for explaining an example of processing of the timing control unit 24 when skip and repeat control is performed so as to prevent overflow and underflow of the video ES buffer 13; It is a flowchart explaining the example of a process of the timing control part 24 when performing control of skip and repeat so that the difference of the output time of each video and audio which should be output simultaneously becomes small.
  • FIG. 18 is a block diagram illustrating a configuration example of an embodiment of a computer to which the present technology is applied.
  • FIG. 1 is a block diagram illustrating a configuration example of an embodiment of a content processing apparatus to which the present technology is applied.
  • a display device capable of displaying video such as a TV (television receiver), a glasses-type wearable device, a smartphone, or the like.
  • a content processing apparatus includes a TS (Transport Stream) buffer 11, a DEMUX 12, a video ES (Elementary Stream) buffer 13, an audio ES buffer 14, a caption buffer 15, a video decoder 16, an audio decoder 17, a caption decoder 18, and a buffer. 19, 20, and 21, a line-of-sight detection unit 22, a buffer accumulation amount monitoring unit 23, and a timing control unit 24.
  • the TS buffer 11 plays, for example, a tuner, a stream server on a network such as the Internet, and storage media (for example, a Bru-ray (registered trademark) disk, a DVD (Digital Versatile Disc), a hard disk, a memory card, etc.)
  • storage media for example, a Bru-ray (registered trademark) disk, a DVD (Digital Versatile Disc), a hard disk, a memory card, etc.
  • a content stream including video, audio, and captions as needed is supplied from a playback device or the like.
  • the TS buffer 11 temporarily stores the content stream supplied thereto and supplies it to the DEMUX 12.
  • an MPEG (Moving Picture Experts Group) stream can be adopted as the content stream.
  • MPEG streams include an MPEG1 stream, MPEG2-PS (Program Stream), and MPEG2-TS.
  • the DEMUX 12 demultiplexes the content stream from the TS buffer 11 into video, audio, caption, etc. (data) as necessary. Further, the DEMUX 12 supplies video, audio, and caption obtained by demultiplexing to the video ES buffer 13, audio ES buffer 14, and caption buffer 15, respectively.
  • the DEMUX 12 includes a demultiplexing unit 31, a video PES (Packetized Elementary Stream) buffer 32, an audio PES buffer 33, a caption PES buffer 34, PES header detection units 35, 36, and 37, and a time information extraction unit 38.
  • a demultiplexing unit 31 a video PES (Packetized Elementary Stream) buffer 32, an audio PES buffer 33, a caption PES buffer 34, PES header detection units 35, 36, and 37, and a time information extraction unit 38.
  • the demultiplexer 31 separates video, audio, captions, and the like from the content stream supplied from the TS buffer 11 to the DEMUX 12.
  • the content stream is separated into a video PES, an audio PES, and the like.
  • the content stream is, for example, MPEG2-TS, the content stream is separated into a video PES, an audio PES, a caption PES, a section, and the like.
  • the video PES, audio PES, and caption PES separated from the content stream in the demultiplexing unit 31 are supplied to the video PES buffer 32, the audio PES buffer 33, and the caption PES buffer 34, respectively.
  • the video PES buffer 32 temporarily stores the video PES supplied from the demultiplexing unit 31.
  • the audio PES buffer 33 temporarily stores the audio PES supplied from the demultiplexing unit 31.
  • the caption PES buffer 34 temporarily stores the caption PES supplied from the demultiplexing unit 31.
  • the PES header detection unit 35 sequentially reads video PES from the video PES buffer 32 and analyzes the PES header of the video PES to extract necessary information. Further, the PES header detection unit 35 extracts the video ES that is data of the video PES from the video PES read from the video PES buffer 32 and supplies the video ES to the video ES buffer 13.
  • the PES header detection unit 36 sequentially reads out the audio PES from the audio PES buffer 33 and analyzes the PES header of the audio PES, thereby extracting necessary information. Further, the PES header detection unit 36 extracts the audio ES that is data of the audio PES from the audio PES read from the audio PES buffer 33 and supplies the audio ES to the audio ES buffer 14.
  • the PES header detection unit 37 sequentially reads the caption PES from the caption PES buffer 34 and analyzes the PES header of the caption PES to extract necessary information. Further, the PES header detection unit 37 extracts a caption, which is data of the caption PES, from the caption PES read from the caption PES buffer 34 and supplies the caption PES to the caption buffer 14.
  • the time information extraction unit 38 uses time information (time stamp (PCR (PCR (PCR) (PCR) ()) to output (display) video, audio, and caption from the information extracted from the PES header by the PES header detection units 35 to 37.
  • Program Clock Reference) and PTS (Presentation Time Stamp)) are extracted and supplied to the timing control unit 24.
  • the video ES buffer 13 functions as a video buffer that stores the video ES supplied from the PES header detection unit 35.
  • the audio ES buffer 14 stores the audio ES supplied from the PES header detection unit 36.
  • the caption buffer 15 stores the caption supplied from the PES header detection unit 37.
  • the video decoder 16 reads and decodes the video ES stored in the video ES buffer 13 and supplies a video frame obtained by the decoding to the buffer 19.
  • the audio decoder 17 reads out and decodes the audio ES stored in the audio ES buffer 14 and supplies the audio obtained by the decoding to the buffer 20.
  • the caption decoder 18 reads and decodes the caption stored in the caption buffer 15, and supplies the caption 21 obtained by the decoding to the buffer 21.
  • the buffer 19 stores video frames supplied from the video decoder 16.
  • the video frames stored in the buffer 19 are read out under the control of the timing control unit 24, supplied to a display device (not shown), and displayed.
  • Buffer 20 stores the audio supplied from audio decoder 17.
  • the audio stored in the buffer 20 is read according to the control of the timing control unit 24, supplied to an audio output device such as a speaker or headphones (not shown), and emitted.
  • the buffer 21 stores the caption supplied from the caption decoder 18.
  • the caption stored in the buffer 21 is read according to the control of the timing control unit 24, supplied to a display device on which video is displayed, and displayed.
  • the line-of-sight detection unit 22 detects the line of sight of a user who views a video or the like displayed on the display device, and supplies (notifies) detection result information representing the detection result of the line of sight to the timing control unit 24.
  • the line-of-sight detection unit 22 detects a user's line of sight by performing image processing of a user's image captured by the camera and the user's image captured by the camera, for example, and a display device that displays the video And an image processing device that identifies whether the device is suitable for the computer.
  • a camera mounted on the smartphone can be used as the camera constituting the line-of-sight detection unit 22.
  • the content processing apparatus of FIG. 1 is applied to, for example, a glasses-type wearable device, when the wearable device is equipped with a sensor that detects the user's line of sight, the sensor is connected to the line-of-sight detection unit 22. Can be used as a camera.
  • timing control unit 24 can acquire the detection result information by some method, it is not necessary to provide the line-of-sight detection unit 22 in the content processing apparatus of FIG.
  • the detection result information is obtained by photographing the user with a surveillance camera installed in the room where the TV is placed.
  • the detection result information is provided to the timing control unit 24, it is not necessary to provide the line-of-sight detection unit 22 in the content processing apparatus.
  • the detection result information that the line-of-sight detection unit 22 supplies to the timing control unit 24 is, for example, whether the user's line of sight is directed to the display device on which the video is displayed, that is, the user is displayed on the display device. Indicates whether you are watching a video.
  • the buffer accumulation amount monitoring unit 23 monitors the buffer accumulation amount (video ES) of the video ES buffer 13 and supplies it to the timing control unit 24.
  • the timing control unit 24 controls an output timing for outputting (reading) video from the buffer 19, an output timing for outputting audio from the buffer 20, and an output timing for outputting captions from the buffer 21.
  • the timing control unit 24 controls the output timing according to the time information supplied from the time information extraction unit 38. Furthermore, the timing control unit 24 controls the output timing according to the detection result information supplied from the line-of-sight detection unit 22 and the buffer accumulation amount supplied from the buffer accumulation amount monitoring unit 23 as necessary.
  • the timing control unit 24 uses the video time information supplied from the time information extraction unit 38 and, for example, the earliest output time represented by the time information among the video frames stored in the buffer 19 (The frame that precedes in time is identified as the frame of interest to be displayed next.
  • the timing control unit 24 controls the output timing of the frame of interest according to the detection result information from the line-of-sight detection unit 22 and the buffer accumulation amount from the buffer accumulation amount monitoring unit 23.
  • timing control unit 24 As control of the output timing of the frame of interest by the timing control unit 24, adjustment of the time at which the frame of interest is output from the buffer 19, and skip and repeat control of output of the frame of interest are performed.
  • the output of the target frame is skipped, and the frame to be output immediately after the target frame is output from the buffer 19 as a new target frame.
  • a new frame is generated by combining a plurality of frames of the target frame and one or more frames to be output after the target frame.
  • One frame is output from the buffer 19 as a new frame of interest.
  • the timing control unit 24 controls skip and repeat according to the detection result information from the line-of-sight detection unit 22 and the buffer accumulation amount from the buffer accumulation amount monitoring unit 23.
  • the timing control unit 24 performs skip control. In this case, since the output of the frame is skipped when the buffer accumulation amount is equal to or greater than the skip threshold, the overflow of the video ES buffer 13 can be prevented.
  • the timing control unit 24 performs repeat control. In this case, since the output of the frame is repeated when the buffer accumulation amount is equal to or less than the repeat threshold, underflow of the video ES buffer 13 can be prevented.
  • the skip threshold and the repeat threshold have a relationship of skip threshold> repeat threshold.
  • the timing control unit 24 performs skip and repeat control according to the buffer accumulation amount.
  • the video displayed on the display device becomes a jerky motion video. Therefore, the user who views the video may feel uncomfortable.
  • the timing control unit 24 skips according to the detection result information from the line-of-sight detection unit 22 so that the user can easily skip or repeat when the user does not watch the video displayed on the display device. And repeat control.
  • the timing control unit 24 sets the skip threshold value and the repeat threshold value to default values.
  • the timing control unit 24 sets the skip threshold value to be smaller than the default value and sets the repeat threshold value to be larger than the default value. To do.
  • the skip threshold is set smaller than the default value
  • the repeat threshold is set larger than the default value.
  • the skip threshold and the repeat threshold are default values, when the possibility of skip or repeat is high in the near future, when the user does not watch the video, skip or Repeat is performed.
  • controlling the output timing of outputting the content (video) according to the detection result of the user's line of sight is particularly useful for the content as described above.
  • content that includes video and audio that are to be output at the same time is output in the timing control unit 24.
  • control of video output timing that is, skip and repeat control, is performed so that the difference between the output times of the video and audio to be output simultaneously is reduced. Can do.
  • the predetermined threshold value may satisfy a content stream standard (for example, MPEG) or may not satisfy the value.
  • the TS buffer 11 temporarily stores the content stream supplied thereto and supplies it to the DEMUX 12.
  • the DEMUX 12 demultiplexes the content stream from the TS buffer 11 into video, audio, caption, etc. (data) as necessary. Further, the DEMUX 12 supplies the video ES, the audio ES, and the caption obtained by the demultiplexing to the video ES buffer 13, the audio ES buffer 14, and the caption buffer 15, respectively, and stores them.
  • the DEMUX 12 (the time information extracting unit 38) extracts time information from the content stream from the TS buffer 11 and supplies the time information to the timing control unit 24.
  • the video decoder 16 reads and decodes the video ES stored in the video ES buffer 13, and supplies the video frame obtained by the decoding to the buffer 19 for storage.
  • the audio decoder 17 reads and decodes the audio ES stored in the audio ES buffer 14 and supplies the audio obtained by the decoding to the buffer 20 for storage.
  • the caption decoder 18 reads and decodes the caption stored in the caption buffer 15, and supplies the caption obtained by the decoding to the buffer 21 for storage.
  • the video frames stored in the buffer 19 and the captions stored in the buffer 21 are read according to the control of the timing control unit 24, supplied to the display device, and displayed.
  • the audio stored in the buffer 20 is read according to the control of the timing control unit 24, supplied to the audio output device, and emitted.
  • the line-of-sight detection unit 22 detects the user's line of sight, and supplies detection result information representing the detection result of the line of sight to the timing control unit 24.
  • the buffer accumulation amount monitoring unit 23 monitors the buffer accumulation amount of the video ES buffer 13 and supplies it to the timing control unit 24.
  • the timing control unit 24 corresponds to the time information supplied from the DEMUX 12 (the time information extraction unit 38), the detection result information supplied from the line-of-sight detection unit 22, and the buffer storage amount supplied from the buffer storage amount monitoring unit 23. Control the output timing of video, audio and caption from each of the buffers 19 to 21.
  • the output timing control by the timing control unit 24 includes skip and repeat control.
  • Skip and repeat control can be performed so as to prevent overflow and underflow of the video ES buffer 13, and so as to reduce the difference between the output times of video and audio to be output simultaneously. Further, the skip and repeat control can be performed so as to prevent the video ES buffer 13 from overflowing and underflowing, and to reduce the difference between the output times of the video and audio to be output simultaneously.
  • a plurality of video ES buffers 13 to caption buffers 15 and video PS buffers 32 to caption PES buffers 34 can be provided.
  • FIG. 2 is a flowchart for explaining an example of processing of the timing control unit 24 when skip and repeat control is performed so as to prevent overflow and underflow of the video ES buffer 13.
  • step S11 the timing control unit 24 acquires the buffer accumulation amount of the video ES buffer 13 supplied from the buffer accumulation amount monitoring unit 23, and the process proceeds to step S12.
  • step S12 the timing control unit 24 determines whether or not there is a notification that the user is not watching the video, that is, the detection result information supplied from the line-of-sight detection unit 22 indicates that the user is not watching the video. Determine whether or not.
  • step S12 If it is determined in step S12 that the detection result information does not indicate that the user does not watch the video, that is, if the user is watching the video, the process proceeds to step S13.
  • step S13 the timing control unit 24 sets each of the skip threshold and the repeat threshold to default values, and the process proceeds to step S14.
  • the default value of the skip threshold is, for example, 90% of the storage capacity of the video ES buffer 13 (hereinafter also referred to as buffer capacity), and the default value of the repeat threshold is For example, the capacity is 10%.
  • step S14 the timing control unit 24 determines whether or not the buffer accumulation amount is equal to or less than the repeat threshold value.
  • step S14 When it is determined in step S14 that the buffer accumulation amount is equal to or less than the repeat threshold, that is, in this case, the buffer accumulation amount is equal to or less than 10% of the buffer capacity that is the default value of the repeat threshold, and the video ES buffer When 13 underflows may occur, the process proceeds to step S15.
  • step S15 the timing control unit 24 performs repeat control of the output of the frame of interest from the buffer 19, and the process ends.
  • the frame of interest is repeatedly output from the buffer 19 by the repeat control of the output of the frame of interest. Thereby, the underflow of the video ES buffer 13 is prevented.
  • step S14 If it is determined in step S14 that the buffer accumulation amount is not equal to or less than the repeat threshold, the process proceeds to step S16, and the timing control unit 24 determines whether the buffer accumulation amount is equal to or greater than the skip threshold. .
  • step S16 When it is determined in step S16 that the buffer accumulation amount is equal to or larger than the skip threshold, that is, in this case, the buffer accumulation amount is 90% or more of the buffer capacity that is the default value of the skip threshold, and the video ES buffer When there is a possibility of 13 overflows, the process proceeds to step S17.
  • step S17 the timing control unit 24 performs control of skipping output of the frame of interest from the buffer 19, and the process ends.
  • the skip of the output of the target frame for example, the output of the target frame from the buffer 19 is skipped, and the frame to be displayed next to the target frame is output. Thereby, overflow of the video ES buffer 13 is prevented.
  • step S16 when it is determined that the buffer accumulation amount is not equal to or greater than the skip threshold value, that is, in this case, the buffer accumulation amount is not 90% or more of the buffer capacity that is the default value of the skip threshold value, and If the repeat threshold is not less than the default value of 10% and there is a low possibility that overflow and underflow of the video ES buffer 13 will occur, the process proceeds to step S18.
  • step S18 the timing control unit 24 outputs the frame of interest from the buffer 19, and the process ends.
  • step S12 determines whether the detection result information indicates that the user does not watch the video. If it is determined in step S12 that the detection result information indicates that the user does not watch the video, the process proceeds to step S19, and the timing control unit 24 is likely to be skipped or repeated.
  • Each of the skip threshold and the repeat threshold is set so that
  • the timing control unit 24 sets the skip threshold to a default value that is smaller than 90% of the buffer capacity, for example, 80% of the buffer capacity. Furthermore, the timing control unit 24 sets the repeat threshold value to a default value that is larger than 10% of the buffer capacity, for example, 20% of the buffer capacity.
  • step S19 After each of the skip threshold and the repeat threshold is set in step S19 so that skip and repeat are easily performed, the process proceeds to step S14, and the same process as described above is performed. Is done.
  • the skip threshold and the repeat threshold are set so that the skip and repeat are easily performed.
  • skip and repeat threshold are default values, skip and repeat may be performed in advance if the user is not watching the video when there is a high possibility of skip or repeat in the near future. Is done.
  • the user since the user does not notice the awkward movement of the video caused by skipping or repeating, the user can be prevented from feeling uncomfortable by watching the video with the awkward movement.
  • FIG. 3 is a flowchart illustrating an example of processing of the timing control unit 24 when the skip and repeat control is performed so that the difference between the output times of the video and audio to be output simultaneously becomes small.
  • step S ⁇ b> 31 the timing control unit 24 determines the difference between the output (display) time indicated by the time information of the target frame stored in the buffer 19 and the output time indicated by the time information of the target audio frame stored in the buffer 20 ( Hereinafter, the output time difference is also calculated, and the process proceeds to step S32.
  • the audio frame of interest means an audio frame of audio to be output from the audio frames stored in the buffer 20 (audio at the earliest output time indicated by the time information).
  • step S32 the timing control unit 24 determines whether the output time difference is equal to or greater than a predetermined threshold value.
  • step S32 If it is determined in step S32 that the output time difference is equal to or greater than the predetermined threshold, the process proceeds to step S33.
  • step S33 the timing control unit 24 increments the number of detections (a variable for counting) unsync_count in which the output time difference is continuously detected to be equal to or greater than a predetermined threshold by 1, and the process proceeds to step S35. Proceed to
  • step S32 If it is determined in step S32 that the output time difference is not equal to or greater than the predetermined threshold, the process proceeds to step S34.
  • step S34 the timing control unit 24 resets the detection count unsync_count to 0, and the process proceeds to step S35.
  • step S35 as in step S12 of FIG. 2, the timing control unit 24 determines whether there is a notification that the user is not watching the video, that is, the detection result information supplied from the line-of-sight detection unit 22 Determines whether or not indicates that the video is not being watched.
  • step S35 If it is determined in step S35 that the detection result information does not indicate that the user does not watch the video, that is, if the user does not watch the video, the process proceeds to step S36.
  • step S36 the timing control unit 24 sets the threshold value resync_count of the number of detections unsync_count to a default value, and the process proceeds to step S37.
  • step S37 the timing control unit 24 determines whether or not the number of detections unsync_count is greater than the threshold value resync_count.
  • step S37 if it is determined that the number of detections unsync_count is greater than the threshold value resync_count, that is, if the output time difference is equal to or greater than the predetermined threshold value, the process continues as follows: Proceed to step S38.
  • step S38 the timing control unit 24 performs AV synchronization control and ends the process.
  • output skip and repeat control of the frame of interest are performed so that the output time difference becomes smaller than a predetermined threshold.
  • step S37 determines whether the number of detections unsync_count is greater than the threshold resync_count, that is, even if the output time difference is not equal to or greater than the predetermined threshold. If it is not consecutive that the output time difference is equal to or greater than the predetermined threshold value more than the threshold resync_count, the process proceeds to step S39.
  • step S39 the timing control unit 24 outputs the frame of interest from the buffer 19, and the process ends.
  • step S35 if it is determined in step S35 that the detection result information indicates that the user does not watch the video, the process proceeds to step S40, and the timing control unit 24 is likely to be skipped or repeated.
  • the threshold resync_count is set so that
  • the timing control unit 24 sets the threshold resync_count to a default value, for example, 0.8 times.
  • step S40 After each of the skip threshold and the repeat threshold is set in step S40 so that skip and repeat are easily performed, the process proceeds to step S37, and the same process as described above is performed. Is done.
  • the threshold resync_count is set so that the user does not watch the video, and therefore, skipping and repeating are easy to be performed.
  • the threshold value resync_count is a default value, skipping and repeating are performed in advance when the user is not watching the video when there is a high possibility of skipping and repeating in the near future.
  • a threshold resync_count of the number of detections unsync_count is set depending on whether the user is watching video, and when the number of detections unsync_count is larger than the threshold resync_count, that is, the output time difference is equal to or greater than a predetermined threshold
  • skip and repeat control is performed when the number of consecutive times is greater than the threshold resync_count, but skip and repeat control is simply based on the magnitude relationship between the output time difference and the predetermined threshold. Can be done accordingly.
  • a predetermined threshold value for the output time difference is set depending on whether the user is watching the video, and when the output time difference is equal to or greater than the predetermined threshold value, skip and repeat control can be performed.
  • the predetermined threshold is set to a default value
  • the predetermined threshold is set to a value smaller than the default value
  • the threshold value resync_count of the number of detections unsync_count depends on whether the user is watching the video as shown in FIG. As in the case of setting, skip and repeat control can be easily performed when the user is not watching the video.
  • an MPEG stream is used as the content stream.
  • the content stream is not limited to the MPEG stream.
  • a series of processing of the timing control unit 24 can be performed by hardware or can be performed by software.
  • a program constituting the software is installed in a computer such as a microcomputer.
  • FIG. 4 shows a configuration example of an embodiment of a computer in which a program for executing the series of processes described above is installed.
  • the program can be recorded in advance in a hard disk 105 or a ROM 103 as a recording medium built in the computer.
  • the program can be stored (recorded) in the removable recording medium 111.
  • a removable recording medium 111 can be provided as so-called package software.
  • examples of the removable recording medium 111 include a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disc, a DVD (Digital Versatile Disc), a magnetic disc, and a semiconductor memory.
  • the program can be installed on the computer from the removable recording medium 111 as described above, or can be downloaded to the computer via the communication network or the broadcast network and installed on the built-in hard disk 105. That is, the program is transferred from a download site to a computer wirelessly via a digital satellite broadcasting artificial satellite, or wired to a computer via a network such as a LAN (Local Area Network) or the Internet. be able to.
  • a network such as a LAN (Local Area Network) or the Internet.
  • the computer includes a CPU (Central Processing Unit) 102, and an input / output interface 110 is connected to the CPU 102 via the bus 101.
  • CPU Central Processing Unit
  • the CPU 102 executes a program stored in a ROM (Read Only Memory) 103 accordingly. .
  • the CPU 102 loads a program stored in the hard disk 105 into a RAM (Random Access Memory) 104 and executes it.
  • the CPU 102 performs processing according to the flowchart described above or processing performed by the configuration of the block diagram described above. Then, the CPU 102 outputs the processing result as necessary, for example, via the input / output interface 110, from the output unit 106, transmitted from the communication unit 108, and further recorded in the hard disk 105.
  • the input unit 107 includes a keyboard, a mouse, a microphone, and the like.
  • the output unit 106 includes an LCD (Liquid Crystal Display), a speaker, and the like.
  • the processing performed by the computer according to the program does not necessarily have to be performed in chronological order in the order described as the flowchart. That is, the processing performed by the computer according to the program includes processing executed in parallel or individually (for example, parallel processing or object processing).
  • the program may be processed by one computer (processor), or may be distributedly processed by a plurality of computers. Furthermore, the program may be transferred to a remote computer and executed.
  • the system means a set of a plurality of components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing. Accordingly, a plurality of devices housed in separate housings and connected via a network and a single device housing a plurality of modules in one housing are all systems. .
  • the present technology can take a cloud computing configuration in which one function is shared by a plurality of devices via a network and is jointly processed.
  • each step described in the above flowchart can be executed by one device or can be shared by a plurality of devices.
  • the plurality of processes included in the one step can be executed by being shared by a plurality of apparatuses in addition to being executed by one apparatus.
  • this technique can take the following structures.
  • a content processing apparatus including a timing control unit that controls output timing of outputting content according to a detection result of a user's line of sight.
  • the content includes a video, The content processing apparatus according to ⁇ 1>, wherein the timing control unit performs skip or repeat control of the video output as control of the output timing.
  • the timing control unit controls the skip or repeat according to a detection result of the user's line of sight so that the skip or repeat is easily performed when the user is not watching the video. 2>.
  • the video is output after being stored in a video buffer
  • the timing controller is The skip control is performed when the accumulation amount of the video stored in the video buffer is equal to or greater than a skip threshold value for performing the skip, and the accumulation amount of the video stored in the video buffer performs the repeat. If the repeat threshold is not greater than the repeat control,
  • the content processing device according to ⁇ 3>, wherein when the user is not watching the video, the skip threshold is decreased and the repeat threshold is increased.
  • the content further includes audio
  • the timing controller is When the difference between the output times at which the video and the audio to be output are to be output is equal to or greater than a predetermined threshold, the skip or repeat control is performed so that the difference between the output times becomes small.
  • the content processing apparatus according to ⁇ 3>, wherein when the user does not watch the video, the skip or repeat is controlled so that the skip or repeat is easily performed.
  • ⁇ 6> In the video skip, the output of the video is skipped, or one video is generated from the plurality of videos.
  • the content processing device according to any one of ⁇ 2> to ⁇ 5>.
  • ⁇ 7> The content processing apparatus according to any one of ⁇ 1> to ⁇ 6>, further including a line-of-sight detection unit that detects a user's line of sight.
  • the video is a video having a frame rate that does not match the display rate of a display device that displays the video, or a video that is output at an output rate that does not match the transmission rate of the video.
  • ⁇ 2> to ⁇ 4> The content processing apparatus according to any one of 4).
  • ⁇ 9> The content processing apparatus according to ⁇ 5>, wherein the content is content that is output with the video and audio to be output simultaneously shifted.
  • a content processing method including controlling output timing of outputting content according to a detection result of a user's line of sight.
  • ⁇ 11> A program for causing a computer to function as a timing control unit that controls output timing of outputting content in accordance with a detection result of a user's line of sight.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Databases & Information Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Television Signal Processing For Recording (AREA)

Abstract

This technology relates to a content processing device, a content processing method, and a program which can prevent a user viewing a content from feeling uncomfortable. A timing control unit controls an output timing at which a content is outputted according to a detection result of a user's line of sight. This technology is applicable, for example, to a television receiver (TV) for displaying a content.

Description

コンテンツ処理装置、コンテンツ処理方法、及び、プログラムContent processing apparatus, content processing method, and program
　本技術は、コンテンツ処理装置、コンテンツ処理方法、及び、プログラムに関し、特に、例えば、コンテンツを視聴するユーザが違和感を感じることを抑制することができるようにするコンテンツ処理装置、コンテンツ処理方法、及び、プログラムに関する。 The present technology relates to a content processing device, a content processing method, and a program, and in particular, for example, a content processing device, a content processing method, and a content processing device that can prevent a user who views content from feeling uncomfortable. Regarding the program.
　例えば、MPEG符号化されたコンテンツを処理するコンテンツ処理装置では、コンテンツに含まれるビデオのフレームのうちの、これから出力しようとするフレームである注目フレームの出力を飛ばすことや、注目フレームの出力を繰り返すことが行われる。 For example, a content processing apparatus that processes MPEG-encoded content skips output of a target frame, which is a frame to be output from among video frames included in the content, or repeatedly outputs a target frame. Is done.
　ここで、注目フレームの出力を飛ばすことを、スキップともいい、注目フレームの出力を繰り返すことを、リピートともいう。 Here, skipping the output of the target frame is also called skipping, and repeating the output of the target frame is also called repeating.
　例えば、特許文献１では、システムクロックに対するビデオのフレームの出力タイミングのずれを検出し、スキップやリピートを行うことで、ビデオのフレームを、適切なタイミングで表示するMPEGデコーダが提案されている。 For example, Patent Document 1 proposes an MPEG decoder that displays a video frame at an appropriate timing by detecting a shift in the output timing of the video frame with respect to the system clock and performing skipping or repeating.
特開2007-235986号公報JP 2007-235986 JP
　ビデオのフレームの出力のスキップやリピートが行われた場合、表示装置に表示されるビデオは、いわゆるカクカクしたぎこちない動きの表示になり、そのビデオを視聴するユーザに違和感を感じさせる。 When video frame output is skipped or repeated, the video displayed on the display device becomes a display of so-called awkward and awkward movements, which makes the user who views the video feel uncomfortable.
　本技術は、このような状況に鑑みてなされたものであり、コンテンツを視聴するユーザが違和感を感じることを抑制することができるようにするものである。 The present technology has been made in view of such a situation, and can prevent a user who views content from feeling uncomfortable.
　本技術のコンテンツ処理装置、又は、プログラムは、ユーザの視線の検出結果に応じて、コンテンツを出力する出力タイミングを制御するタイミング制御部を備えるコンテンツ処理装置、又は、そのようなコンテンツ処理装置として、コンピュータを機能させるためのプログラムである。 The content processing apparatus or the program according to the present technology includes a timing control unit that controls an output timing for outputting content according to a detection result of a user's line of sight, or such a content processing apparatus. It is a program for causing a computer to function.
　本技術のコンテンツ処理方法は、ユーザの視線の検出結果に応じて、コンテンツを出力する出力タイミングを制御することを含むコンテンツ処理方法である。 The content processing method of the present technology is a content processing method including controlling an output timing of outputting content according to a detection result of a user's line of sight.
　本技術のコンテンツ処理装置、コンテンツ処理方法、及び、プログラムにおいては、ユーザの視線の検出結果に応じて、コンテンツを出力する出力タイミングが制御される。 In the content processing apparatus, the content processing method, and the program of the present technology, the output timing for outputting the content is controlled according to the detection result of the user's line of sight.
　コンテンツ処理装置は、独立した装置であっても良いし、１つの装置を構成している内部ブロックであっても良い。 The content processing device may be an independent device, or may be an internal block constituting one device.
　また、プログラムは、伝送媒体を介して伝送することにより、又は、記録媒体に記録して、提供することができる。 Further, the program can be provided by being transmitted through a transmission medium or by being recorded on a recording medium.
　本技術によれば、コンテンツを視聴するユーザが違和感を感じることを抑制することができる。 According to the present technology, it is possible to prevent the user who views the content from feeling uncomfortable.
　なお、ここに記載された効果は必ずしも限定されるものではなく、本開示中に記載されたいずれかの効果であってもよい。 It should be noted that the effects described here are not necessarily limited, and may be any of the effects described in the present disclosure.
本技術を適用したコンテンツ処理装置の一実施の形態の構成例を示すブロック図である。It is a block diagram which shows the structural example of one Embodiment of the content processing apparatus to which this technique is applied. スキップ及びリピートの制御を、ビデオESバッファ１３のオーバーフロー及びアンダーフローを防止するように行う場合の、タイミング制御部２４の処理の例を説明するフローチャートである。10 is a flowchart for explaining an example of processing of the timing control unit 24 when skip and repeat control is performed so as to prevent overflow and underflow of the video ES buffer 13; スキップ及びリピートの制御を、同時に出力すべきビデオ及びオーディオそれぞれの出力時刻の差分が小さくなるように行う場合の、タイミング制御部２４の処理の例を説明するフローチャートである。It is a flowchart explaining the example of a process of the timing control part 24 when performing control of skip and repeat so that the difference of the output time of each video and audio which should be output simultaneously becomes small. 本技術を適用したコンピュータの一実施の形態の構成例を示すブロック図である。And FIG. 18 is a block diagram illustrating a configuration example of an embodiment of a computer to which the present technology is applied.
　＜本技術を適用したコンテンツ処理装置の一実施の形態＞ <An embodiment of a content processing apparatus to which the present technology is applied>
　図１は、本技術を適用したコンテンツ処理装置の一実施の形態の構成例を示すブロック図である。 FIG. 1 is a block diagram illustrating a configuration example of an embodiment of a content processing apparatus to which the present technology is applied.
　図１のコンテンツ処理装置は、例えば、TV（テレビジョン受像機）や、眼鏡型のウエアラブルデバイス、スマートフォンその他の、ビデオを表示することが可能な表示装置に適用することができる。 1 can be applied to a display device capable of displaying video, such as a TV (television receiver), a glasses-type wearable device, a smartphone, or the like.
　図１において、コンテンツ処理装置は、TS(Transport Stream)バッファ１１、DEMUX１２、ビデオES(Elementary Stream)バッファ１３、オーディオESバッファ１４、キャプションバッファ１５、ビデオデコーダ１６、オーディオデコーダ１７、キャプションデコーダ１８、バッファ１９，２０、及び、２１、視線検出部２２、バッファ蓄積量監視部２３、並びに、タイミング制御部２４を有する。 In FIG. 1, a content processing apparatus includes a TS (Transport Stream) buffer 11, a DEMUX 12, a video ES (Elementary Stream) buffer 13, an audio ES buffer 14, a caption buffer 15, a video decoder 16, an audio decoder 17, a caption decoder 18, and a buffer. 19, 20, and 21, a line-of-sight detection unit 22, a buffer accumulation amount monitoring unit 23, and a timing control unit 24.
　TSバッファ１１には、例えば、チューナや、インターネット等のネットワーク上のストリームサーバ、蓄積メディア（例えば、Bru-ray（登録商標）ディスクや、DVD(Digital Versatile Disc)、ハードディスク、メモリカード等）を再生する再生装置等から、ビデオや、オーディオ、キャプションを必要に応じて含むコンテンツのストリームが供給される。 The TS buffer 11 plays, for example, a tuner, a stream server on a network such as the Internet, and storage media (for example, a Bru-ray (registered trademark) disk, a DVD (Digital Versatile Disc), a hard disk, a memory card, etc.) A content stream including video, audio, and captions as needed is supplied from a playback device or the like.
　TSバッファ１１は、そこに供給されるコンテンツのストリームを一時記憶し、DEMUX１２に供給する。 The TS buffer 11 temporarily stores the content stream supplied thereto and supplies it to the DEMUX 12.
　ここで、コンテンツのストリームとしては、例えば、MPEG(Moving Picture Experts Group)のストリームを採用することができる。MPEGのストリームとしては、例えば、MPEG1ストリームや、MPEG2-PS(Program Stream)、MPEG2-TSがある。 Here, for example, an MPEG (Moving Picture Experts Group) stream can be adopted as the content stream. Examples of MPEG streams include an MPEG1 stream, MPEG2-PS (Program Stream), and MPEG2-TS.
　DEMUX１２は、TSバッファ１１からのコンテンツのストリームを、必要に応じて、ビデオや、オーディオ、キャプション等（のデータ）に逆多重化する。さらに、DEMUX１２は、逆多重化によって得られるビデオ、オーディオ、及び、キャプションを、ビデオESバッファ１３、オーディオESバッファ１４、及び、キャプションバッファ１５に、それぞれ供給する。 The DEMUX 12 demultiplexes the content stream from the TS buffer 11 into video, audio, caption, etc. (data) as necessary. Further, the DEMUX 12 supplies video, audio, and caption obtained by demultiplexing to the video ES buffer 13, audio ES buffer 14, and caption buffer 15, respectively.
　すなわち、DEMUX１２は、多重分離部３１、ビデオPES(Packetized Elementary Stream)バッファ３２、オーディオPESバッファ３３、キャプションPESバッファ３４、PESヘッダ検出部３５，３６、及び、３７、並びに、時刻情報抽出部３８を有する。 In other words, the DEMUX 12 includes a demultiplexing unit 31, a video PES (Packetized Elementary Stream) buffer 32, an audio PES buffer 33, a caption PES buffer 34, PES header detection units 35, 36, and 37, and a time information extraction unit 38. Have.
　多重分離部３１は、TSバッファ１１からDEMUX１２に供給されるコンテンツのストリームから、ビデオや、オーディオ、キャプション等を分離する。 The demultiplexer 31 separates video, audio, captions, and the like from the content stream supplied from the TS buffer 11 to the DEMUX 12.
　コンテンツのストリームが、例えば、MPEG1ストリーム又はMPEG2-PSである場合には、そのコンテンツのストリームは、ビデオPES及びオーディオPES等に分離される。また、コンテンツのストリームが、例えば、MPEG2-TSである場合には、そのコンテンツのストリームは、ビデオPES、オーディオPES、キャプションPES、及び、セクション等に分離される。 When the content stream is, for example, an MPEG1 stream or MPEG2-PS, the content stream is separated into a video PES, an audio PES, and the like. When the content stream is, for example, MPEG2-TS, the content stream is separated into a video PES, an audio PES, a caption PES, a section, and the like.
　多重分離部３１において、コンテンツのストリームから分離されたビデオPES、オーディオPES、及び、キャプションPESは、ビデオPESバッファ３２、オーディオPESバッファ３３、及び、キャプションPESバッファ３４に、それぞれ供給される。 The video PES, audio PES, and caption PES separated from the content stream in the demultiplexing unit 31 are supplied to the video PES buffer 32, the audio PES buffer 33, and the caption PES buffer 34, respectively.
　ビデオPESバッファ３２は、多重分離部３１から供給されるビデオPESを一時記憶する。オーディオPESバッファ３３は、多重分離部３１から供給されるオーディオPESを一時記憶する。キャプションPESバッファ３４は、多重分離部３１から供給されるキャプションPESを一時記憶する。 The video PES buffer 32 temporarily stores the video PES supplied from the demultiplexing unit 31. The audio PES buffer 33 temporarily stores the audio PES supplied from the demultiplexing unit 31. The caption PES buffer 34 temporarily stores the caption PES supplied from the demultiplexing unit 31.
　PESヘッダ検出部３５は、ビデオPESバッファ３２からビデオPESを順次読み出し、そのビデオPESのPESヘッダを解析することで、必要な情報を抽出する。さらに、PESヘッダ検出部３５は、ビデオPESバッファ３２から読み出したビデオPESから、そのビデオPESのデータであるビデオESを抽出し、ビデオESバッファ１３に供給する。 The PES header detection unit 35 sequentially reads video PES from the video PES buffer 32 and analyzes the PES header of the video PES to extract necessary information. Further, the PES header detection unit 35 extracts the video ES that is data of the video PES from the video PES read from the video PES buffer 32 and supplies the video ES to the video ES buffer 13.
　PESヘッダ検出部３６は、オーディオPESバッファ３３からオーディオPESを順次読み出し、そのオーディオPESのPESヘッダを解析することで、必要な情報を抽出する。さらに、PESヘッダ検出部３６は、オーディオPESバッファ３３から読み出したオーディオPESから、そのオーディオPESのデータであるオーディオESを抽出し、オーディオESバッファ１４に供給する。 The PES header detection unit 36 sequentially reads out the audio PES from the audio PES buffer 33 and analyzes the PES header of the audio PES, thereby extracting necessary information. Further, the PES header detection unit 36 extracts the audio ES that is data of the audio PES from the audio PES read from the audio PES buffer 33 and supplies the audio ES to the audio ES buffer 14.
　PESヘッダ検出部３７は、キャプションPESバッファ３４からキャプションPESを順次読み出し、そのキャプションPESのPESヘッダを解析することで、必要な情報を抽出する。さらに、PESヘッダ検出部３７は、キャプションPESバッファ３４から読み出したキャプションPESから、そのキャプションPESのデータであるキャプションを抽出し、キャプションバッファ１４に供給する。 The PES header detection unit 37 sequentially reads the caption PES from the caption PES buffer 34 and analyzes the PES header of the caption PES to extract necessary information. Further, the PES header detection unit 37 extracts a caption, which is data of the caption PES, from the caption PES read from the caption PES buffer 34 and supplies the caption PES to the caption buffer 14.
　時刻情報抽出部３８は、PESヘッダ検出部３５ないし３７がPESヘッダから抽出した情報から、ビデオ、オーディオ、及び、キャプションを出力（表示）すべき出力時刻等を表す時刻情報（タイムスタンプ(PCR (Program Clock Reference)，PTS (Presentation Time Stamp)等)）を抽出し、タイミング制御部２４に供給する。 The time information extraction unit 38 uses time information (time stamp (PCR (PCR (PCR) (PCR) ()) to output (display) video, audio, and caption from the information extracted from the PES header by the PES header detection units 35 to 37. Program Clock Reference) and PTS (Presentation Time Stamp))) are extracted and supplied to the timing control unit 24.
　ビデオESバッファ１３は、PESヘッダ検出部３５から供給されるビデオESを記憶するビデオバッファとして機能する。オーディオESバッファ１４は、PESヘッダ検出部３６から供給されるオーディオESを記憶する。キャプションバッファ１５は、PESヘッダ検出部３７から供給されるキャプションを記憶する。 The video ES buffer 13 functions as a video buffer that stores the video ES supplied from the PES header detection unit 35. The audio ES buffer 14 stores the audio ES supplied from the PES header detection unit 36. The caption buffer 15 stores the caption supplied from the PES header detection unit 37.
　ビデオデコーダ１６は、ビデオESバッファ１３に記憶されたビデオESを読み出して復号し、その復号により得られるビデオのフレームを、バッファ１９に供給する。 The video decoder 16 reads and decodes the video ES stored in the video ES buffer 13 and supplies a video frame obtained by the decoding to the buffer 19.
　オーディオデコーダ１７は、オーディオESバッファ１４に記憶されたオーディオESを読み出して復号し、その復号により得られるオーディオを、バッファ２０に供給する。 The audio decoder 17 reads out and decodes the audio ES stored in the audio ES buffer 14 and supplies the audio obtained by the decoding to the buffer 20.
　キャプションデコーダ１８は、キャプションバッファ１５に記憶されたキャプションを読み出して復号し、その復号により得られるキャプションを、バッファ２１に供給する。 The caption decoder 18 reads and decodes the caption stored in the caption buffer 15, and supplies the caption 21 obtained by the decoding to the buffer 21.
　バッファ１９は、ビデオデコーダ１６から供給されるビデオのフレームを記憶する。バッファ１９に記憶されたビデオのフレームは、タイミング制御部２４の制御に従って読み出され、図示せぬ表示装置に供給されて表示される。 The buffer 19 stores video frames supplied from the video decoder 16. The video frames stored in the buffer 19 are read out under the control of the timing control unit 24, supplied to a display device (not shown), and displayed.
　バッファ２０は、オーディオデコーダ１７から供給されるオーディオを記憶する。バッファ２０に記憶されたオーディオは、タイミング制御部２４の制御に従って読み出され、図示せぬスピーカやヘッドフォン等のオーディオ出力装置に供給されて放音される。 Buffer 20 stores the audio supplied from audio decoder 17. The audio stored in the buffer 20 is read according to the control of the timing control unit 24, supplied to an audio output device such as a speaker or headphones (not shown), and emitted.
　バッファ２１は、キャプションデコーダ１８から供給されるキャプションを記憶する。バッファ２１に記憶されたキャプションは、タイミング制御部２４の制御に従って読み出され、ビデオが表示される表示装置に供給されて表示される。 The buffer 21 stores the caption supplied from the caption decoder 18. The caption stored in the buffer 21 is read according to the control of the timing control unit 24, supplied to a display device on which video is displayed, and displayed.
　視線検出部２２は、表示装置に表示されるビデオ等を視聴するユーザの視線を検出し、その視線の検出結果を表す検出結果情報を、タイミング制御部２４に供給（通知）する。 The line-of-sight detection unit 22 detects the line of sight of a user who views a video or the like displayed on the display device, and supplies (notifies) detection result information representing the detection result of the line of sight to the timing control unit 24.
　視線検出部２２は、例えば、ユーザを撮影するカメラと、そのカメラで撮影されるユーザの画像の画像処理を行うことで、ユーザの視線を検出し、その視線が、ビデオが表示される表示装置に向いているかどうかを特定する画像処理装置とで構成することができる。 The line-of-sight detection unit 22 detects a user's line of sight by performing image processing of a user's image captured by the camera and the user's image captured by the camera, for example, and a display device that displays the video And an image processing device that identifies whether the device is suitable for the computer.
　図１のコンテンツ処理装置が、例えば、スマートフォンに適用される場合、視線検出部２２を構成するカメラとしては、スマートフォンに実装されているカメラを用いることができる。さらに、図１のコンテンツ処理装置が、例えば、眼鏡型のウエアラブルデバイスに適用される場合に、そのウエアラブルデバイスがユーザの視線を検出するセンサを実装しているときには、そのセンサを、視線検出部２２を構成するカメラとして利用することができる。 When the content processing apparatus of FIG. 1 is applied to, for example, a smartphone, a camera mounted on the smartphone can be used as the camera constituting the line-of-sight detection unit 22. Furthermore, when the content processing apparatus of FIG. 1 is applied to, for example, a glasses-type wearable device, when the wearable device is equipped with a sensor that detects the user's line of sight, the sensor is connected to the line-of-sight detection unit 22. Can be used as a camera.
　なお、タイミング制御部２４が、何らかの方法で、検出結果情報を取得することができれば、図１のコンテンツ処理装置に、視線検出部２２を設ける必要はない。 Note that if the timing control unit 24 can acquire the detection result information by some method, it is not necessary to provide the line-of-sight detection unit 22 in the content processing apparatus of FIG.
　すなわち、例えば、図１のコンテンツ処理装置が、例えば、TVに適用されている場合において、そのTVが置かれている部屋に設置された監視カメラで、ユーザを撮影することにより検出結果情報が得られ、その検出結果情報が、タイミング制御部２４に提供されるときには、コンテンツ処理装置に、視線検出部２２を設ける必要はない。 That is, for example, when the content processing apparatus of FIG. 1 is applied to a TV, for example, the detection result information is obtained by photographing the user with a surveillance camera installed in the room where the TV is placed. When the detection result information is provided to the timing control unit 24, it is not necessary to provide the line-of-sight detection unit 22 in the content processing apparatus.
　ここで、視線検出部２２がタイミング制御部２４に供給する検出結果情報は、例えば、ユーザの視線が、ビデオが表示される表示装置に向いているかどうか、すなわち、ユーザが、表示装置に表示されたビデオを見ているかどうかを表す。 Here, the detection result information that the line-of-sight detection unit 22 supplies to the timing control unit 24 is, for example, whether the user's line of sight is directed to the display device on which the video is displayed, that is, the user is displayed on the display device. Indicates whether you are watching a video.
　バッファ蓄積量監視部２３は、ビデオESバッファ１３の（ビデオESの）バッファ蓄積量を監視し、タイミング制御部２４に供給する。 The buffer accumulation amount monitoring unit 23 monitors the buffer accumulation amount (video ES) of the video ES buffer 13 and supplies it to the timing control unit 24.
　タイミング制御部２４は、バッファ１９からビデオを出力する（読み出す）出力タイミング、バッファ２０からオーディオを出力する出力タイミング、及び、バッファ２１からキャプションを出力する出力タイミングを制御する。 The timing control unit 24 controls an output timing for outputting (reading) video from the buffer 19, an output timing for outputting audio from the buffer 20, and an output timing for outputting captions from the buffer 21.
　すなわち、タイミング制御部２４は、時刻情報抽出部３８から供給される時刻情報に応じて、出力タイミングの制御を行う。さらに、タイミング制御部２４は、必要に応じて、視線検出部２２から供給される検出結果情報や、バッファ蓄積量監視部２３から供給されるバッファ蓄積量に応じて、出力タイミングの制御を行う。 That is, the timing control unit 24 controls the output timing according to the time information supplied from the time information extraction unit 38. Furthermore, the timing control unit 24 controls the output timing according to the detection result information supplied from the line-of-sight detection unit 22 and the buffer accumulation amount supplied from the buffer accumulation amount monitoring unit 23 as necessary.
　例えば、タイミング制御部２４は、時刻情報抽出部３８から供給される、ビデオの時刻情報を用い、バッファ１９に記憶されたビデオのフレームのうちの、例えば、時刻情報が表す出力時刻が最も早い（時間的に先行する）フレームを、次に表示すべき注目フレームとして特定する。 For example, the timing control unit 24 uses the video time information supplied from the time information extraction unit 38 and, for example, the earliest output time represented by the time information among the video frames stored in the buffer 19 ( The frame that precedes in time is identified as the frame of interest to be displayed next.
　そして、タイミング制御部２４は、注目フレームの出力タイミングを、視線検出部２２からの検出結果情報や、バッファ蓄積量監視部２３からのバッファ蓄積量に応じて制御する。 The timing control unit 24 controls the output timing of the frame of interest according to the detection result information from the line-of-sight detection unit 22 and the buffer accumulation amount from the buffer accumulation amount monitoring unit 23.
　タイミング制御部２４による注目フレームの出力タイミングの制御としては、注目フレームをバッファ１９から出力する時刻の調整と、注目フレームの出力のスキップやリピートの制御とが行われる。 As control of the output timing of the frame of interest by the timing control unit 24, adjustment of the time at which the frame of interest is output from the buffer 19, and skip and repeat control of output of the frame of interest are performed.
　注目フレームの出力のスキップでは、注目フレームの出力が飛ばされ、その注目フレームの直後に出力されるべきフレームが、新たな注目フレームとして、バッファ１９から出力される。 In skipping the output of the target frame, the output of the target frame is skipped, and the frame to be output immediately after the target frame is output from the buffer 19 as a new target frame.
　あるいは、注目フレームの出力のスキップでは、注目フレームと、その注目フレームの後に出力されるべき１フレーム以上のフレームとの複数のフレームを合成すること等により新たな１フレームが生成され、その新たな１フレームが、新たな注目フレームとして、バッファ１９から出力される。 Alternatively, in skipping the output of the target frame, a new frame is generated by combining a plurality of frames of the target frame and one or more frames to be output after the target frame. One frame is output from the buffer 19 as a new frame of interest.
　本実施の形態では、以上のように、注目フレームの出力のスキップを行う方法として、注目フレームの出力を単に飛ばす方法と、注目フレームを含む複数のフレームから、新たな注目フレームを生成する方法とがある。 In the present embodiment, as described above, as a method of skipping the output of the target frame, a method of simply skipping the output of the target frame, and a method of generating a new target frame from a plurality of frames including the target frame, There is.
　注目フレームの出力のリピートでは、注目フレームとして特定されたフレームが、バッファ１９から出力された後、そのフレームが、再び、注目フレームとされ、バッファ１９から出力される。 In the repeat of the output of the target frame, after the frame specified as the target frame is output from the buffer 19, the frame is again set as the target frame and output from the buffer 19.
　タイミング制御部２４は、視線検出部２２からの検出結果情報と、バッファ蓄積量監視部２３からのバッファ蓄積量とに応じて、スキップ及びリピートの制御を行う。 The timing control unit 24 controls skip and repeat according to the detection result information from the line-of-sight detection unit 22 and the buffer accumulation amount from the buffer accumulation amount monitoring unit 23.
　すなわち、バッファ蓄積量が、スキップを行うスキップ閾値以上（又はより大）である場合、タイミング制御部２４は、スキップの制御を行う。この場合、バッファ蓄積量がスキップ閾値以上であるときに、フレームの出力のスキップが行われるので、ビデオESバッファ１３のオーバーフローを防止することができる。 That is, when the buffer accumulation amount is equal to or greater than (or larger than) the skip threshold for skipping, the timing control unit 24 performs skip control. In this case, since the output of the frame is skipped when the buffer accumulation amount is equal to or greater than the skip threshold, the overflow of the video ES buffer 13 can be prevented.
　また、バッファ蓄積量が、リピートを行うリピート閾値以下（又は未満）である場合、タイミング制御部２４は、リピートの制御を行う。この場合、バッファ蓄積量がリピート閾値以下であるときに、フレームの出力のリピートが行われるので、ビデオESバッファ１３のアンダーフローを防止することができる。 Further, when the buffer accumulation amount is equal to or less than (or less than) the repeat threshold value for performing repeat, the timing control unit 24 performs repeat control. In this case, since the output of the frame is repeated when the buffer accumulation amount is equal to or less than the repeat threshold, underflow of the video ES buffer 13 can be prevented.
　ここで、スキップ閾値及びリピート閾値は、スキップ閾値＞リピート閾値の関係がある。 Here, the skip threshold and the repeat threshold have a relationship of skip threshold> repeat threshold.
　以上のように、タイミング制御部２４は、バッファ蓄積量に応じて、スキップ及びリピートの制御を行うが、スキップやリピートが行われると、表示装置に表示されるビデオがカクカクしたぎこちない動きのビデオになり、そのビデオを視聴するユーザに違和感を感じさせることがある。 As described above, the timing control unit 24 performs skip and repeat control according to the buffer accumulation amount. However, when skip or repeat is performed, the video displayed on the display device becomes a jerky motion video. Therefore, the user who views the video may feel uncomfortable.
　そこで、タイミング制御部２４は、視線検出部２２からの検出結果情報に応じて、ユーザが、表示装置に表示されたビデオを見ていない場合に、スキップやリピートが行われやすくなるように、スキップ及びリピートを制御する。 Therefore, the timing control unit 24 skips according to the detection result information from the line-of-sight detection unit 22 so that the user can easily skip or repeat when the user does not watch the video displayed on the display device. And repeat control.
　具体的には、例えば、検出結果情報が、ユーザがビデオを見ている旨を表している場合、タイミング制御部２４は、スキップ閾値及びリピート閾値を、デフォルト値に設定する。 Specifically, for example, when the detection result information indicates that the user is watching the video, the timing control unit 24 sets the skip threshold value and the repeat threshold value to default values.
　一方、検出結果情報が、ユーザがビデオを見ていない旨を表している場合、タイミング制御部２４は、スキップ閾値をデフォルト値よりも小に設定するとともに、リピート閾値をデフォルト値よりも大に設定する。 On the other hand, when the detection result information indicates that the user does not watch the video, the timing control unit 24 sets the skip threshold value to be smaller than the default value and sets the repeat threshold value to be larger than the default value. To do.
　以上のように、検出結果情報が、ユーザがビデオを見ていない旨を表している場合に、スキップ閾値をデフォルト値よりも小に設定するとともに、リピート閾値をデフォルト値よりも大に設定することで、ユーザがビデオを見ていない場合に、スキップやリピートが行われやすくなる。 As described above, when the detection result information indicates that the user does not watch the video, the skip threshold is set smaller than the default value, and the repeat threshold is set larger than the default value. Thus, when the user is not watching the video, it becomes easy to skip or repeat.
　その結果、スキップ閾値及びリピート閾値が、デフォルト値であれば、近い将来に、スキップやリピートが行われる可能性が高い場合において、ユーザがビデオを見ていないときに、いわば先取り的に、スキップやリピートが行われる。 As a result, if the skip threshold and the repeat threshold are default values, when the possibility of skip or repeat is high in the near future, when the user does not watch the video, skip or Repeat is performed.
　これにより、ユーザがビデオを見ているときに、スキップやリピートが行われ、ユーザが、ぎこちない動きのビデオを見ることで、ユーザに違和感を感じさせることを防止することができる。 This makes it possible to prevent the user from feeling uncomfortable by watching the video with awkward movements when skipping or repeating is performed when the user is watching the video.
　すなわち、ユーザの視線の検出結果に応じて、コンテンツ（のビデオ）を出力する出力タイミングを制御することで、ユーザに違和感を感じさせることを防止することができる。 That is, it is possible to prevent the user from feeling uncomfortable by controlling the output timing of outputting the content (video) according to the detection result of the user's line of sight.
　ここで、例えば、ビデオを表示する表示装置の表示レートと一致していないフレームレートのビデオを含むコンテンツや、ビデオの伝送レートと一致していない出力レートで出力されるビデオを含むコンテンツについては、ビデオESバッファ１３のオーバーフローやアンダーフローが生じやすくなり、スキップやリピートが行われやすくなる。 Here, for example, for content that includes video with a frame rate that does not match the display rate of the display device that displays the video, or content that includes video that is output at an output rate that does not match the video transmission rate, Overflow and underflow of the video ES buffer 13 are likely to occur, and skipping and repeating are easily performed.
　したがって、ユーザの視線の検出結果に応じて、コンテンツ（のビデオ）を出力する出力タイミングを制御することは、上述のようなコンテンツに、特に有用である。 Therefore, controlling the output timing of outputting the content (video) according to the detection result of the user's line of sight is particularly useful for the content as described above.
　また、例えば、同時に出力すべきビデオとオーディオとがずれて出力されるコンテンツ（再生（復号）中に、ビデオとオーディオとが、徐々にずれていくコンテンツを含む）については、タイミング制御部２４において、時刻情報抽出部３８からの時刻情報に応じて、同時に出力すべきビデオ及びオーディオそれぞれの出力時刻の差分が小さくなるように、ビデオの出力タイミングの制御、すなわち、スキップやリピートの制御を行うことができる。 In addition, for example, content that includes video and audio that are to be output at the same time (including content in which video and audio are gradually shifted during playback (decoding)) is output in the timing control unit 24. In accordance with the time information from the time information extraction unit 38, control of video output timing, that is, skip and repeat control, is performed so that the difference between the output times of the video and audio to be output simultaneously is reduced. Can do.
　このように、同時に出力すべきビデオとオーディオとがずれて出力されるコンテンツについて、スキップやリピートが行われる場合も、そのスキップやリピートによって、ぎこちない動きになったビデオを、ユーザが見たときには、ユーザに違和感を感じさせる。 In this way, even when skip and repeat are performed for content that is output with the video and audio that should be output simultaneously shifted, when the user sees a video that has become awkward due to the skip or repeat, Make the user feel uncomfortable.
　そこで、同時に出力すべきビデオとオーディオとがずれて出力されるコンテンツについても、ユーザの視線の検出結果に応じて、コンテンツを出力する出力タイミングを制御することで、ユーザに違和感を感じさせることを防止することができる。 Therefore, it is possible to make the user feel uncomfortable by controlling the output timing of outputting the content according to the detection result of the user's line of sight for the content that is output with the video and audio to be output simultaneously shifted. Can be prevented.
　なお、同時に出力すべきビデオ及びオーディオそれぞれの出力時刻の差分が、所定の閾値以上（又はより大）である場合に、その差分が小さくなるように、スキップやリピートの制御を行うことで、同時に出力すべきビデオ及びオーディオそれぞれの出力時刻の差分は、所定の閾値未満にすることができる。 Note that when the difference between the output times of the video and audio to be output at the same time is equal to or greater than (or larger than) a predetermined threshold value, skip and repeat control are performed so that the difference becomes smaller. The difference between the output times of the video and audio to be output can be less than a predetermined threshold.
　所定の閾値は、コンテンツのストリームの規格（例えば、MPEG）を満たす値であっても良いし、満たさない値であっても良い。 The predetermined threshold value may satisfy a content stream standard (for example, MPEG) or may not satisfy the value.
　以上のように構成されるコンテンツ処理装置では、TSバッファ１１は、そこに供給されるコンテンツのストリームを一時記憶し、DEMUX１２に供給する。 In the content processing apparatus configured as described above, the TS buffer 11 temporarily stores the content stream supplied thereto and supplies it to the DEMUX 12.
　DEMUX１２は、TSバッファ１１からのコンテンツのストリームを、必要に応じて、ビデオや、オーディオ、キャプション等（のデータ）に逆多重化する。さらに、DEMUX１２は、逆多重化によって得られるビデオES、オーディオES、及び、キャプションを、ビデオESバッファ１３、オーディオESバッファ１４、及び、キャプションバッファ１５に、それぞれ供給して記憶させる。 The DEMUX 12 demultiplexes the content stream from the TS buffer 11 into video, audio, caption, etc. (data) as necessary. Further, the DEMUX 12 supplies the video ES, the audio ES, and the caption obtained by the demultiplexing to the video ES buffer 13, the audio ES buffer 14, and the caption buffer 15, respectively, and stores them.
　また、DEMUX１２（の時刻情報抽出部３８）は、TSバッファ１１からのコンテンツのストリームから時刻情報を抽出し、タイミング制御部２４に供給する。 The DEMUX 12 (the time information extracting unit 38) extracts time information from the content stream from the TS buffer 11 and supplies the time information to the timing control unit 24.
　ビデオデコーダ１６は、ビデオESバッファ１３に記憶されたビデオESを読み出して復号し、その復号により得られるビデオのフレームを、バッファ１９に供給して記憶させる。 The video decoder 16 reads and decodes the video ES stored in the video ES buffer 13, and supplies the video frame obtained by the decoding to the buffer 19 for storage.
　オーディオデコーダ１７は、オーディオESバッファ１４に記憶されたオーディオESを読み出して復号し、その復号により得られるオーディオを、バッファ２０に供給して記憶させる。 The audio decoder 17 reads and decodes the audio ES stored in the audio ES buffer 14 and supplies the audio obtained by the decoding to the buffer 20 for storage.
　キャプションデコーダ１８は、キャプションバッファ１５に記憶されたキャプションを読み出して復号し、その復号により得られるキャプションを、バッファ２１に供給して記憶させる。 The caption decoder 18 reads and decodes the caption stored in the caption buffer 15, and supplies the caption obtained by the decoding to the buffer 21 for storage.
　バッファ１９に記憶されたビデオのフレーム、及び、バッファ２１に記憶されたキャプションは、タイミング制御部２４の制御に従って読み出され、表示装置に供給されて表示される。 The video frames stored in the buffer 19 and the captions stored in the buffer 21 are read according to the control of the timing control unit 24, supplied to the display device, and displayed.
　バッファ２０に記憶されたオーディオは、タイミング制御部２４の制御に従って読み出され、オーディオ出力装置に供給されて放音される。 The audio stored in the buffer 20 is read according to the control of the timing control unit 24, supplied to the audio output device, and emitted.
　一方、視線検出部２２は、ユーザの視線を検出し、その視線の検出結果を表す検出結果情報を、タイミング制御部２４に供給する。 On the other hand, the line-of-sight detection unit 22 detects the user's line of sight, and supplies detection result information representing the detection result of the line of sight to the timing control unit 24.
　また、バッファ蓄積量監視部２３は、ビデオESバッファ１３のバッファ蓄積量を監視し、タイミング制御部２４に供給する。 Also, the buffer accumulation amount monitoring unit 23 monitors the buffer accumulation amount of the video ES buffer 13 and supplies it to the timing control unit 24.
　タイミング制御部２４は、DEMUX１２（の時刻情報抽出部３８）から供給される時刻情報、視線検出部２２から供給される検出結果情報、バッファ蓄積量監視部２３から供給されるバッファ蓄積量に応じて、バッファ１９ないし２１それぞれからのビデオ、オーディオ、及び、キャプションの出力タイミングの制御を行う。 The timing control unit 24 corresponds to the time information supplied from the DEMUX 12 (the time information extraction unit 38), the detection result information supplied from the line-of-sight detection unit 22, and the buffer storage amount supplied from the buffer storage amount monitoring unit 23. Control the output timing of video, audio and caption from each of the buffers 19 to 21.
　タイミング制御部２４による出力タイミングの制御には、スキップ及びリピートの制御が含まれる。 The output timing control by the timing control unit 24 includes skip and repeat control.
　スキップ及びリピートの制御は、ビデオESバッファ１３のオーバーフロー及びアンダーフローを防止するように行うことや、同時に出力すべきビデオ及びオーディオそれぞれの出力時刻の差分が小さくなるように行うことができる。さらに、スキップ及びリピートの制御は、ビデオESバッファ１３のオーバーフロー及びアンダーフローを防止し、かつ、同時に出力すべきビデオ及びオーディオそれぞれの出力時刻の差分が小さくなるように行うことができる。 Skip and repeat control can be performed so as to prevent overflow and underflow of the video ES buffer 13, and so as to reduce the difference between the output times of video and audio to be output simultaneously. Further, the skip and repeat control can be performed so as to prevent the video ES buffer 13 from overflowing and underflowing, and to reduce the difference between the output times of the video and audio to be output simultaneously.
　以下、スキップ及びリピートの制御を、ビデオESバッファ１３のオーバーフロー及びアンダーフローを防止するように行う場合と、同時に出力すべきビデオ及びオーディオそれぞれの出力時刻の差分が小さくなるように行う場合との、タイミング制御部２４の処理について説明する。 Hereinafter, skip and repeat control are performed so as to prevent overflow and underflow of the video ES buffer 13, and when the difference between the output times of the video and audio to be output simultaneously is reduced. The processing of the timing control unit 24 will be described.
　ここで、図１のコンテンツ処理装置においては、ビデオESバッファ１３ないしキャプションバッファ１５、並びに、ビデオPSバッファ３２ないしキャプションPESバッファ３４のそれぞれは、複数設けることができる。 Here, in the content processing apparatus shown in FIG. 1, a plurality of video ES buffers 13 to caption buffers 15 and video PS buffers 32 to caption PES buffers 34 can be provided.
　＜スキップ及びリピートの制御を、ビデオESバッファ１３のオーバーフロー及びアンダーフローを防止するように行う場合の、タイミング制御部２４の処理の例＞ <Example of processing of the timing control unit 24 when skip and repeat control are performed to prevent overflow and underflow of the video ES buffer 13>
　図２は、スキップ及びリピートの制御を、ビデオESバッファ１３のオーバーフロー及びアンダーフローを防止するように行う場合の、タイミング制御部２４の処理の例を説明するフローチャートである。 FIG. 2 is a flowchart for explaining an example of processing of the timing control unit 24 when skip and repeat control is performed so as to prevent overflow and underflow of the video ES buffer 13.
　ステップＳ１１において、タイミング制御部２４は、バッファ蓄積量監視部２３から供給されるビデオESバッファ１３のバッファ蓄積量を取得し、処理は、ステップＳ１２に進む。 In step S11, the timing control unit 24 acquires the buffer accumulation amount of the video ES buffer 13 supplied from the buffer accumulation amount monitoring unit 23, and the process proceeds to step S12.
　ステップＳ１２では、タイミング制御部２４は、ユーザがビデオを見ていない旨の通知があるかどうか、すなわち、視線検出部２２から供給される検出結果情報が、ユーザがビデオを見ていない旨を表しているかどうかを判定する。 In step S12, the timing control unit 24 determines whether or not there is a notification that the user is not watching the video, that is, the detection result information supplied from the line-of-sight detection unit 22 indicates that the user is not watching the video. Determine whether or not.
　ステップＳ１２において、検出結果情報が、ユーザがビデオを見ていない旨を表していないと判定された場合、すなわち、ユーザがビデオを見ている場合、処理は、ステップＳ１３に進む。ステップＳ１３では、タイミング制御部２４は、スキップ閾値及びリピート閾値のそれぞれを、デフォルト値に設定して、処理は、ステップＳ１４に進む。 If it is determined in step S12 that the detection result information does not indicate that the user does not watch the video, that is, if the user is watching the video, the process proceeds to step S13. In step S13, the timing control unit 24 sets each of the skip threshold and the repeat threshold to default values, and the process proceeds to step S14.
　ここで、図２では、スキップ閾値のデフォルト値は、ビデオESバッファ１３の記憶容量（以下、バッファ容量ともいう）の、例えば、90%の値になっており、リピート閾値のデフォルト値は、バッファ容量の、例えば、10%の値になっている。 Here, in FIG. 2, the default value of the skip threshold is, for example, 90% of the storage capacity of the video ES buffer 13 (hereinafter also referred to as buffer capacity), and the default value of the repeat threshold is For example, the capacity is 10%.
　ステップＳ１４では、タイミング制御部２４は、バッファ蓄積量が、リピート閾値以下であるかどうかを判定する。 In step S14, the timing control unit 24 determines whether or not the buffer accumulation amount is equal to or less than the repeat threshold value.
　ステップＳ１４において、バッファ蓄積量が、リピート閾値以下であると判定されたとき、すなわち、いまの場合、バッファ蓄積量が、リピート閾値のデフォルト値であるバッファ容量の10%以下であり、ビデオESバッファ１３のアンダーフローが生じるおそれがあるとき、処理は、ステップＳ１５に進む。 When it is determined in step S14 that the buffer accumulation amount is equal to or less than the repeat threshold, that is, in this case, the buffer accumulation amount is equal to or less than 10% of the buffer capacity that is the default value of the repeat threshold, and the video ES buffer When 13 underflows may occur, the process proceeds to step S15.
　ステップＳ１５では、タイミング制御部２４は、バッファ１９からの注目フレームの出力のリピートの制御を行い、処理は、終了する。 In step S15, the timing control unit 24 performs repeat control of the output of the frame of interest from the buffer 19, and the process ends.
　注目フレームの出力のリピートの制御により、バッファ１９から注目フレームが繰り返し出力される。これにより、ビデオESバッファ１３のアンダーフローが防止される。 注目 The frame of interest is repeatedly output from the buffer 19 by the repeat control of the output of the frame of interest. Thereby, the underflow of the video ES buffer 13 is prevented.
　また、ステップＳ１４において、バッファ蓄積量が、リピート閾値以下でないと判定されたとき、処理は、ステップＳ１６に進み、タイミング制御部２４は、バッファ蓄積量が、スキップ閾値以上であるかどうかを判定する。 If it is determined in step S14 that the buffer accumulation amount is not equal to or less than the repeat threshold, the process proceeds to step S16, and the timing control unit 24 determines whether the buffer accumulation amount is equal to or greater than the skip threshold. .
　ステップＳ１６において、バッファ蓄積量が、スキップ閾値以上であると判定されたとき、すなわち、いまの場合、バッファ蓄積量が、スキップ閾値のデフォルト値であるバッファ容量の90%以上であり、ビデオESバッファ１３のオーバーフローが生じるおそれがあるとき、処理は、ステップＳ１７に進む。 When it is determined in step S16 that the buffer accumulation amount is equal to or larger than the skip threshold, that is, in this case, the buffer accumulation amount is 90% or more of the buffer capacity that is the default value of the skip threshold, and the video ES buffer When there is a possibility of 13 overflows, the process proceeds to step S17.
　ステップＳ１７では、タイミング制御部２４は、バッファ１９からの注目フレームの出力のスキップの制御を行い、処理は、終了する。 In step S17, the timing control unit 24 performs control of skipping output of the frame of interest from the buffer 19, and the process ends.
　注目フレームの出力のスキップの制御により、例えば、バッファ１９からの注目フレームの出力を飛ばして、その注目フレームの次に表示すべきフレームの出力が行われる。これにより、ビデオESバッファ１３のオーバーフローが防止される。 By controlling the skip of the output of the target frame, for example, the output of the target frame from the buffer 19 is skipped, and the frame to be displayed next to the target frame is output. Thereby, overflow of the video ES buffer 13 is prevented.
　また、ステップＳ１６において、バッファ蓄積量が、スキップ閾値以上でないと判定されたとき、すなわち、いまの場合、バッファ蓄積量が、スキップ閾値のデフォルト値であるバッファ容量の90%以上でもなく、かつ、リピート閾値のデフォルト値である10%以下でもなく、ビデオESバッファ１３のオーバーフロー及びアンダーフローが生じるおそれが低い場合、処理は、ステップＳ１８に進む。 In step S16, when it is determined that the buffer accumulation amount is not equal to or greater than the skip threshold value, that is, in this case, the buffer accumulation amount is not 90% or more of the buffer capacity that is the default value of the skip threshold value, and If the repeat threshold is not less than the default value of 10% and there is a low possibility that overflow and underflow of the video ES buffer 13 will occur, the process proceeds to step S18.
　ステップＳ１８では、タイミング制御部２４は、バッファ１９から注目フレームを出力させ、処理は、終了する。 In step S18, the timing control unit 24 outputs the frame of interest from the buffer 19, and the process ends.
　一方、ステップＳ１２において、検出結果情報が、ユーザがビデオを見ていない旨を表していると判定された場合、処理は、ステップＳ１９に進み、タイミング制御部２４は、スキップやリピートが行われやすくなるように、スキップ閾値及びリピート閾値のそれぞれを設定する。 On the other hand, if it is determined in step S12 that the detection result information indicates that the user does not watch the video, the process proceeds to step S19, and the timing control unit 24 is likely to be skipped or repeated. Each of the skip threshold and the repeat threshold is set so that
　すなわち、タイミング制御部２４は、スキップ閾値を、デフォルト値である、バッファ容量の90%よりも小さい、例えば、バッファ容量の80%に設定する。さらに、タイミング制御部２４は、リピート閾値を、デフォルト値である、バッファ容量の10%よりも大きい、例えば、バッファ容量の20%に設定する。 That is, the timing control unit 24 sets the skip threshold to a default value that is smaller than 90% of the buffer capacity, for example, 80% of the buffer capacity. Furthermore, the timing control unit 24 sets the repeat threshold value to a default value that is larger than 10% of the buffer capacity, for example, 20% of the buffer capacity.
　以上のように、ステップＳ１９において、スキップやリピートが行われやすくなるように、スキップ閾値及びリピート閾値のそれぞれが設定された後、処理は、ステップＳ１４に進み、以下、上述の場合と同様の処理が行われる。 As described above, after each of the skip threshold and the repeat threshold is set in step S19 so that skip and repeat are easily performed, the process proceeds to step S14, and the same process as described above is performed. Is done.
　但し、いまの場合、ユーザがビデオを見ておらず、そのため、スキップやリピートが行われやすくなるように、スキップ閾値及びリピート閾値のそれぞれが設定されている。 However, in this case, the user does not watch the video, and therefore, the skip threshold and the repeat threshold are set so that the skip and repeat are easily performed.
　そのため、ステップＳ１５での注目フレームの出力のリピートの制御や、ステップＳ１７での注目フレームの出力のスキップの制御が行われやすくなる。 Therefore, it becomes easy to perform the repeat control of the output of the target frame in step S15 and the skip control of the output of the target frame in step S17.
　その結果、スキップ閾値及びリピート閾値が、デフォルト値であれば、近い将来に、スキップやリピートが行われる可能性が高い場合において、ユーザがビデオを見ていないときに、いわば先立って、スキップやリピートが行われる。 As a result, if the skip threshold and the repeat threshold are default values, skip and repeat may be performed in advance if the user is not watching the video when there is a high possibility of skip or repeat in the near future. Is done.
　したがって、ユーザは、スキップやリピートによって生じる、ビデオのぎこちない動きに気づかないので、ユーザが、ぎこちない動きのビデオを見ることで違和感を感じることを防止することができる。 Therefore, since the user does not notice the awkward movement of the video caused by skipping or repeating, the user can be prevented from feeling uncomfortable by watching the video with the awkward movement.
　なお、図２のフローチャートに従った処理は繰り返し行われる。 Note that the processing according to the flowchart of FIG. 2 is repeated.
　＜スキップ及びリピートの制御を、同時に出力すべきビデオ及びオーディオそれぞれの出力時刻の差分が小さくなるように行う場合の、タイミング制御部２４の処理の例＞ <Example of processing of the timing control unit 24 when the skip and repeat control is performed so that the difference between the output times of the video and audio to be output simultaneously becomes small>
　図３は、スキップ及びリピートの制御を、同時に出力すべきビデオ及びオーディオそれぞれの出力時刻の差分が小さくなるように行う場合の、タイミング制御部２４の処理の例を説明するフローチャートである。 FIG. 3 is a flowchart illustrating an example of processing of the timing control unit 24 when the skip and repeat control is performed so that the difference between the output times of the video and audio to be output simultaneously becomes small.
　ステップＳ３１において、タイミング制御部２４は、バッファ１９に記憶された注目フレームの時刻情報が表す出力（表示）時刻と、バッファ２０に記憶された注目オーディオフレームの時刻情報が表す出力時刻との差分（以下、出力時刻差分ともいう）を計算し、処理は、ステップＳ３２に進む。 In step S <b> 31, the timing control unit 24 determines the difference between the output (display) time indicated by the time information of the target frame stored in the buffer 19 and the output time indicated by the time information of the target audio frame stored in the buffer 20 ( Hereinafter, the output time difference is also calculated, and the process proceeds to step S32.
　ここで、注目オーディオフレームとは、バッファ２０に記憶されたオーディオのオーディオフレームのうちの、これから出力しようとするオーディオ（時刻情報が表す出力時刻が最も早い時刻のオーディオ）のオーディオフレームを意味する。 Here, the audio frame of interest means an audio frame of audio to be output from the audio frames stored in the buffer 20 (audio at the earliest output time indicated by the time information).
　ステップＳ３２では、タイミング制御部２４は、出力時刻差分が所定の閾値以上であるかどうかを判定する。 In step S32, the timing control unit 24 determines whether the output time difference is equal to or greater than a predetermined threshold value.
　ステップＳ３２において、出力時刻差分が所定の閾値以上であると判定された場合、処理は、ステップＳ３３に進む。ステップＳ３３では、タイミング制御部２４は、出力時刻差分が所定の閾値以上であることが連続して検出された検出回数（をカウントする変数）unsync_countを、１だけインクリメントして、処理は、ステップＳ３５に進む。 If it is determined in step S32 that the output time difference is equal to or greater than the predetermined threshold, the process proceeds to step S33. In step S33, the timing control unit 24 increments the number of detections (a variable for counting) unsync_count in which the output time difference is continuously detected to be equal to or greater than a predetermined threshold by 1, and the process proceeds to step S35. Proceed to
　また、ステップＳ３２において、出力時刻差分が所定の閾値以上でないと判定された場合、処理は、ステップＳ３４に進む。ステップＳ３４では、タイミング制御部２４は、検出回数unsync_countを、０にリセットし、処理は、ステップＳ３５に進む。 If it is determined in step S32 that the output time difference is not equal to or greater than the predetermined threshold, the process proceeds to step S34. In step S34, the timing control unit 24 resets the detection count unsync_count to 0, and the process proceeds to step S35.
　ステップＳ３５では、図２のステップＳ１２と同様に、タイミング制御部２４は、ユーザがビデオを見ていない旨の通知があるかどうか、すなわち、視線検出部２２から供給される検出結果情報が、ユーザがビデオを見ていない旨を表しているかどうかを判定する。 In step S35, as in step S12 of FIG. 2, the timing control unit 24 determines whether there is a notification that the user is not watching the video, that is, the detection result information supplied from the line-of-sight detection unit 22 Determines whether or not indicates that the video is not being watched.
　ステップＳ３５において、検出結果情報が、ユーザがビデオを見ていない旨を表していないと判定された場合、すなわち、ユーザがビデオを見ていない場合、処理は、ステップＳ３６に進む。ステップＳ３６では、タイミング制御部２４は、検出回数unsync_countの閾値resync_countをデフォルト値に設定して、処理は、ステップＳ３７に進む。 If it is determined in step S35 that the detection result information does not indicate that the user does not watch the video, that is, if the user does not watch the video, the process proceeds to step S36. In step S36, the timing control unit 24 sets the threshold value resync_count of the number of detections unsync_count to a default value, and the process proceeds to step S37.
　ステップＳ３７では、タイミング制御部２４は、検出回数unsync_countが、閾値resync_countより大であるかどうかを判定する。 In step S37, the timing control unit 24 determines whether or not the number of detections unsync_count is greater than the threshold value resync_count.
　ステップＳ３７において、検出回数unsync_countが、閾値resync_countより大であると判定された場合、すなわち、出力時刻差分が所定の閾値以上であることが、閾値resync_countよりも多い回数だけ連続した場合、処理は、ステップＳ３８に進む。 In step S37, if it is determined that the number of detections unsync_count is greater than the threshold value resync_count, that is, if the output time difference is equal to or greater than the predetermined threshold value, the process continues as follows: Proceed to step S38.
　ステップＳ３８では、タイミング制御部２４は、AV同期制御を行い、処理を終了する。 In step S38, the timing control unit 24 performs AV synchronization control and ends the process.
　ここで、AV同期制御では、出力時刻差分が所定の閾値より小さくなるように、例えば、注目フレームの出力のスキップやリピートの制御が行われる。 Here, in the AV synchronization control, for example, output skip and repeat control of the frame of interest are performed so that the output time difference becomes smaller than a predetermined threshold.
　すなわち、注目フレームの出力時刻が、注目オーディオフレームの出力時刻よりも進んでいる場合、注目フレームの出力時刻を遅らせるために、注目フレームのリピートの制御が行われる。 That is, when the output time of the target frame is ahead of the output time of the target audio frame, repeat control of the target frame is performed in order to delay the output time of the target frame.
　また、注目フレームの出力時刻が、注目オーディオフレームの出力時刻よりも遅れている場合、注目フレームの出力時刻を進ませるために、注目フレームのスキップの制御が行われる。 Further, when the output time of the target frame is delayed from the output time of the target audio frame, skip control of the target frame is performed in order to advance the output time of the target frame.
　一方、ステップＳ３７において、検出回数unsync_countが、閾値resync_countより大でないと判定された場合、すなわち、出力時刻差分が所定の閾値以上でないか、又は、出力時刻差分が所定の閾値以上であっても、出力時刻差分が所定の閾値以上であることが、閾値resync_countより多い回数だけ連続していない場合、処理は、ステップＳ３９に進む。 On the other hand, if it is determined in step S37 that the number of detections unsync_count is not greater than the threshold resync_count, that is, even if the output time difference is not equal to or greater than the predetermined threshold, If it is not consecutive that the output time difference is equal to or greater than the predetermined threshold value more than the threshold resync_count, the process proceeds to step S39.
　ステップＳ３９では、タイミング制御部２４は、バッファ１９から注目フレームを出力させ、処理は、終了する。 In step S39, the timing control unit 24 outputs the frame of interest from the buffer 19, and the process ends.
　一方、ステップＳ３５において、検出結果情報が、ユーザがビデオを見ていない旨を表していると判定された場合、処理は、ステップＳ４０に進み、タイミング制御部２４は、スキップやリピートが行われやすくなるように、閾値resync_countを設定する。 On the other hand, if it is determined in step S35 that the detection result information indicates that the user does not watch the video, the process proceeds to step S40, and the timing control unit 24 is likely to be skipped or repeated. The threshold resync_count is set so that
　すなわち、タイミング制御部２４は、閾値resync_countを、デフォルト値の、例えば、0.8倍に設定する。 That is, the timing control unit 24 sets the threshold resync_count to a default value, for example, 0.8 times.
　以上のように、ステップＳ４０において、スキップやリピートが行われやすくなるように、スキップ閾値及びリピート閾値のそれぞれが設定された後、処理は、ステップＳ３７に進み、以下、上述の場合と同様の処理が行われる。 As described above, after each of the skip threshold and the repeat threshold is set in step S40 so that skip and repeat are easily performed, the process proceeds to step S37, and the same process as described above is performed. Is done.
　但し、いまの場合、ユーザがビデオを見ておらず、そのため、スキップやリピートが行われやすくなるように、閾値resync_countが設定されている。 However, in this case, the threshold resync_count is set so that the user does not watch the video, and therefore, skipping and repeating are easy to be performed.
　そのため、ステップＳ３８での注目フレームの出力のリピートやスキップの制御が行われやすくなる。 Therefore, it becomes easy to perform output repeat and skip control of the frame of interest in step S38.
　その結果、閾値resync_countが、デフォルト値であれば、近い将来に、スキップやリピートが行われる可能性が高い場合において、ユーザがビデオを見ていないときに、先立って、スキップやリピートが行われる。 As a result, if the threshold value resync_count is a default value, skipping and repeating are performed in advance when the user is not watching the video when there is a high possibility of skipping and repeating in the near future.
　したがって、ユーザが、スキップやリピートによって生じる、ビデオのぎこちない動きを見ることで違和感を感じることを防止することができる。 Therefore, it is possible to prevent the user from feeling uncomfortable by seeing the awkward movement of the video caused by skipping or repeating.
　なお、図３のフローチャートに従った処理は繰り返し行われる。 Note that the processing according to the flowchart of FIG. 3 is repeated.
　ここで、図３では、ユーザがビデオを見ているかどうかによって、検出回数unsync_countの閾値resync_countを設定し、検出回数unsync_countが、閾値resync_countより大である場合、すなわち、出力時刻差分が所定の閾値以上であることが、閾値resync_countよりも多い回数だけ連続した場合に、スキップやリピートの制御を行うこととしたが、スキップやリピートの制御は、単に、出力時刻差分と所定の閾値との大小関係に応じて行うことができる。 Here, in FIG. 3, a threshold resync_count of the number of detections unsync_count is set depending on whether the user is watching video, and when the number of detections unsync_count is larger than the threshold resync_count, that is, the output time difference is equal to or greater than a predetermined threshold However, skip and repeat control is performed when the number of consecutive times is greater than the threshold resync_count, but skip and repeat control is simply based on the magnitude relationship between the output time difference and the predetermined threshold. Can be done accordingly.
　すなわち、ユーザがビデオを見ているかどうかによって、出力時刻差分の所定の閾値を設定し、出力時刻差分が所定の閾値以上である場合に、スキップやリピートの制御を行うことができる。この場合、ユーザがビデオを見ているときには、所定の閾値がデフォルト値に設定され、ユーザがビデオを見ていないときには、所定の閾値がデフォルト値よりも小さい値に設定される。 That is, a predetermined threshold value for the output time difference is set depending on whether the user is watching the video, and when the output time difference is equal to or greater than the predetermined threshold value, skip and repeat control can be performed. In this case, when the user is watching the video, the predetermined threshold is set to a default value, and when the user is not watching the video, the predetermined threshold is set to a value smaller than the default value.
　以上のように、ユーザがビデオを見ているかどうかによって、出力時刻差分の所定の閾値を設定する場合も、図３のように、ユーザがビデオを見ているかどうかによって、検出回数unsync_countの閾値resync_countを設定する場合と同様に、ユーザがビデオを見ていないときに、スキップやリピートの制御を行われやすくなる。 As described above, even when the predetermined threshold value of the output time difference is set depending on whether the user is watching the video, the threshold value resync_count of the number of detections unsync_count depends on whether the user is watching the video as shown in FIG. As in the case of setting, skip and repeat control can be easily performed when the user is not watching the video.
　したがって、ユーザが、スキップやリピートによって生じる、ビデオのぎこちない動きを見ることで違和感を感じることを防止することができる。 Therefore, it is possible to prevent the user from feeling uncomfortable by seeing the awkward movement of the video caused by skipping or repeating.
　なお、本実施の形態では、コンテンツのストリームとして、MPEGのストリームを採用したが、コンテンツのストリームは、MPEGのストリームに限定されるものではない。 In this embodiment, an MPEG stream is used as the content stream. However, the content stream is not limited to the MPEG stream.
　＜本技術を適用したコンピュータの説明＞ <Description of computer to which this technology is applied>
　次に、タイミング制御部２４の一連の処理は、ハードウェアにより行うこともできるし、ソフトウェアにより行うこともできる。一連の処理をソフトウェアによって行う場合には、そのソフトウェアを構成するプログラムが、マイクロコンピュータ等のコンピュータにインストールされる。 Next, a series of processing of the timing control unit 24 can be performed by hardware or can be performed by software. When a series of processing is performed by software, a program constituting the software is installed in a computer such as a microcomputer.
　そこで、図４は、上述した一連の処理を実行するプログラムがインストールされるコンピュータの一実施の形態の構成例を示している。 Therefore, FIG. 4 shows a configuration example of an embodiment of a computer in which a program for executing the series of processes described above is installed.
　プログラムは、コンピュータに内蔵されている記録媒体としてのハードディスク１０５やROM１０３に予め記録しておくことができる。 The program can be recorded in advance in a hard disk 105 or a ROM 103 as a recording medium built in the computer.
　あるいはまた、プログラムは、リムーバブル記録媒体１１１に格納（記録）しておくことができる。このようなリムーバブル記録媒体１１１は、いわゆるパッケージソフトウエアとして提供することができる。ここで、リムーバブル記録媒体１１１としては、例えば、フレキシブルディスク、CD-ROM(Compact Disc Read Only Memory)，MO(Magneto Optical)ディスク，DVD(Digital Versatile Disc)、磁気ディスク、半導体メモリ等がある。 Alternatively, the program can be stored (recorded) in the removable recording medium 111. Such a removable recording medium 111 can be provided as so-called package software. Here, examples of the removable recording medium 111 include a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disc, a DVD (Digital Versatile Disc), a magnetic disc, and a semiconductor memory.
　なお、プログラムは、上述したようなリムーバブル記録媒体１１１からコンピュータにインストールする他、通信網や放送網を介して、コンピュータにダウンロードし、内蔵するハードディスク１０５にインストールすることができる。すなわち、プログラムは、例えば、ダウンロードサイトから、ディジタル衛星放送用の人工衛星を介して、コンピュータに無線で転送したり、LAN(Local Area Network)、インターネットといったネットワークを介して、コンピュータに有線で転送することができる。 The program can be installed on the computer from the removable recording medium 111 as described above, or can be downloaded to the computer via the communication network or the broadcast network and installed on the built-in hard disk 105. That is, the program is transferred from a download site to a computer wirelessly via a digital satellite broadcasting artificial satellite, or wired to a computer via a network such as a LAN (Local Area Network) or the Internet. be able to.
　コンピュータは、CPU(Central Processing Unit)１０２を内蔵しており、CPU１０２には、バス１０１を介して、入出力インタフェース１１０が接続されている。 The computer includes a CPU (Central Processing Unit) 102, and an input / output interface 110 is connected to the CPU 102 via the bus 101.
　CPU１０２は、入出力インタフェース１１０を介して、ユーザによって、入力部１０７が操作等されることにより指令が入力されると、それに従って、ROM(Read Only Memory)１０３に格納されているプログラムを実行する。あるいは、CPU１０２は、ハードディスク１０５に格納されたプログラムを、RAM(Random Access Memory)１０４にロードして実行する。 When an instruction is input by the user operating the input unit 107 via the input / output interface 110, the CPU 102 executes a program stored in a ROM (Read Only Memory) 103 accordingly. . Alternatively, the CPU 102 loads a program stored in the hard disk 105 into a RAM (Random Access Memory) 104 and executes it.
　これにより、CPU１０２は、上述したフローチャートにしたがった処理、あるいは上述したブロック図の構成により行われる処理を行う。そして、CPU１０２は、その処理結果を、必要に応じて、例えば、入出力インタフェース１１０を介して、出力部１０６から出力、あるいは、通信部１０８から送信、さらには、ハードディスク１０５に記録等させる。 Thereby, the CPU 102 performs processing according to the flowchart described above or processing performed by the configuration of the block diagram described above. Then, the CPU 102 outputs the processing result as necessary, for example, via the input / output interface 110, from the output unit 106, transmitted from the communication unit 108, and further recorded in the hard disk 105.
　なお、入力部１０７は、キーボードや、マウス、マイク等で構成される。また、出力部１０６は、LCD(Liquid Crystal Display)やスピーカ等で構成される。 Note that the input unit 107 includes a keyboard, a mouse, a microphone, and the like. The output unit 106 includes an LCD (Liquid Crystal Display), a speaker, and the like.
　ここで、本明細書において、コンピュータがプログラムに従って行う処理は、必ずしもフローチャートとして記載された順序に沿って時系列に行われる必要はない。すなわち、コンピュータがプログラムに従って行う処理は、並列的あるいは個別に実行される処理（例えば、並列処理あるいはオブジェクトによる処理）も含む。 Here, in the present specification, the processing performed by the computer according to the program does not necessarily have to be performed in chronological order in the order described as the flowchart. That is, the processing performed by the computer according to the program includes processing executed in parallel or individually (for example, parallel processing or object processing).
　また、プログラムは、１のコンピュータ（プロセッサ）により処理されるものであっても良いし、複数のコンピュータによって分散処理されるものであっても良い。さらに、プログラムは、遠方のコンピュータに転送されて実行されるものであっても良い。 Further, the program may be processed by one computer (processor), or may be distributedly processed by a plurality of computers. Furthermore, the program may be transferred to a remote computer and executed.
　さらに、本明細書において、システムとは、複数の構成要素（装置、モジュール（部品）等）の集合を意味し、すべての構成要素が同一筐体中にあるか否かは問わない。したがって、別個の筐体に収納され、ネットワークを介して接続されている複数の装置、及び、１つの筐体の中に複数のモジュールが収納されている１つの装置は、いずれも、システムである。 Furthermore, in this specification, the system means a set of a plurality of components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing. Accordingly, a plurality of devices housed in separate housings and connected via a network and a single device housing a plurality of modules in one housing are all systems. .
　なお、本技術の実施の形態は、上述した実施の形態に限定されるものではなく、本技術の要旨を逸脱しない範囲において種々の変更が可能である。 Note that the embodiments of the present technology are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present technology.
　例えば、本技術は、１つの機能をネットワークを介して複数の装置で分担、共同して処理するクラウドコンピューティングの構成をとることができる。 For example, the present technology can take a cloud computing configuration in which one function is shared by a plurality of devices via a network and is jointly processed.
　また、上述のフローチャートで説明した各ステップは、１つの装置で実行する他、複数の装置で分担して実行することができる。 Further, each step described in the above flowchart can be executed by one device or can be shared by a plurality of devices.
　さらに、１つのステップに複数の処理が含まれる場合には、その１つのステップに含まれる複数の処理は、１つの装置で実行する他、複数の装置で分担して実行することができる。 Further, when a plurality of processes are included in one step, the plurality of processes included in the one step can be executed by being shared by a plurality of apparatuses in addition to being executed by one apparatus.
　また、本明細書に記載された効果はあくまで例示であって限定されるものではなく、他の効果があってもよい。 Further, the effects described in the present specification are merely examples and are not limited, and other effects may be obtained.
　なお、本技術は、以下のような構成をとることができる。 In addition, this technique can take the following structures.
　＜１＞
　ユーザの視線の検出結果に応じて、コンテンツを出力する出力タイミングを制御するタイミング制御部を備える
　コンテンツ処理装置。
　＜２＞
　前記コンテンツは、ビデオを含み、
　前記タイミング制御部は、前記出力タイミングの制御として、前記ビデオの出力のスキップ又はリピートの制御を行う
　＜１＞に記載のコンテンツ処理装置。
　＜３＞
　前記タイミング制御部は、前記ユーザの視線の検出結果に応じて、前記ユーザが、前記ビデオを見ていない場合に、前記スキップ又はリピートが行われやすくなるように、前記スキップ又はリピートを制御する
　＜２＞に記載のコンテンツ処理装置。
　＜４＞
　前記ビデオは、ビデオバッファに記憶されてから出力され、
　前記タイミング制御部は、
　　前記ビデオバッファに記憶された前記ビデオの蓄積量が、前記スキップを行うスキップ閾値以上である場合に前記スキップの制御を行い、前記ビデオバッファに記憶された前記ビデオの蓄積量が、前記リピートを行うリピート閾値以下である場合に前記リピートの制御を行い、
　　前記ユーザが、前記ビデオを見ていない場合に、前記スキップ閾値を小さくするとともに、前記リピート閾値を大きくする
　＜３＞に記載のコンテンツ処理装置。
　＜５＞
　前記コンテンツは、オーディオをさらに含み、
　前記タイミング制御部は、
　　これから出力しようとする前記ビデオ及び前記オーディオそれぞれを出力すべき出力時刻の差分が、所定の閾値以上である場合に、前記出力時刻の差分が小さくなるように、前記スキップ又はリピートの制御を行い、
　　前記ユーザが、前記ビデオを見ていない場合に、前記スキップ又はリピートが行われやすくなるように、前記スキップ又はリピートを制御する
　＜３＞に記載のコンテンツ処理装置。
　＜６＞
　前記ビデオのスキップでは、前記ビデオの出力が飛ばされるか、又は、複数枚の前記ビデオから１枚のビデオが生成される
　＜２＞ないし＜５＞のいずれかに記載のコンテンツ処理装置。
　＜７＞
　ユーザの視線を検出する視線検出部をさらに備える
　＜１＞ないし＜６＞のいずれかに記載のコンテンツ処理装置。
　＜８＞
　前記ビデオは、前記ビデオを表示する表示装置の表示レートと一致していないフレームレートのビデオ、又は、前記ビデオの伝送レートと一致していない出力レートで出力されるビデオである
　＜２＞ないし＜４＞のいずれかに記載のコンテンツ処理装置。
　＜９＞
　前記コンテンツは、同時に出力すべき前記ビデオと前記オーディオとがずれて出力されるコンテンツである
　＜５＞に記載のコンテンツ処理装置。
　＜１０＞
　ユーザの視線の検出結果に応じて、コンテンツを出力する出力タイミングを制御することを含む
　コンテンツ処理方法。
　＜１１＞
　ユーザの視線の検出結果に応じて、コンテンツを出力する出力タイミングを制御するタイミング制御部
　として、コンピュータを機能させるためのプログラム。 <1>
A content processing apparatus including a timing control unit that controls output timing of outputting content according to a detection result of a user's line of sight.
<2>
The content includes a video,
The content processing apparatus according to <1>, wherein the timing control unit performs skip or repeat control of the video output as control of the output timing.
<3>
The timing control unit controls the skip or repeat according to a detection result of the user's line of sight so that the skip or repeat is easily performed when the user is not watching the video. 2>.
<4>
The video is output after being stored in a video buffer,
The timing controller is
The skip control is performed when the accumulation amount of the video stored in the video buffer is equal to or greater than a skip threshold value for performing the skip, and the accumulation amount of the video stored in the video buffer performs the repeat. If the repeat threshold is not greater than the repeat control,
The content processing device according to <3>, wherein when the user is not watching the video, the skip threshold is decreased and the repeat threshold is increased.
<5>
The content further includes audio,
The timing controller is
When the difference between the output times at which the video and the audio to be output are to be output is equal to or greater than a predetermined threshold, the skip or repeat control is performed so that the difference between the output times becomes small.
The content processing apparatus according to <3>, wherein when the user does not watch the video, the skip or repeat is controlled so that the skip or repeat is easily performed.
<6>
In the video skip, the output of the video is skipped, or one video is generated from the plurality of videos. The content processing device according to any one of <2> to <5>.
<7>
The content processing apparatus according to any one of <1> to <6>, further including a line-of-sight detection unit that detects a user's line of sight.
<8>
The video is a video having a frame rate that does not match the display rate of a display device that displays the video, or a video that is output at an output rate that does not match the transmission rate of the video. <2> to <4>. The content processing apparatus according to any one of 4).
<9>
The content processing apparatus according to <5>, wherein the content is content that is output with the video and audio to be output simultaneously shifted.
<10>
A content processing method including controlling output timing of outputting content according to a detection result of a user's line of sight.
<11>
A program for causing a computer to function as a timing control unit that controls output timing of outputting content in accordance with a detection result of a user's line of sight.
　１１　TSバッファ，　１２　DEMUX，　１３　ビデオESバッファ，　１４　オーディオESバッファ，　１５　キャプションバッファ，　１６　ビデオデコーダ，　１７　オーディオデコーダ，　１８　キャプションデコーダ，　１９ないし２１　バッファ，　２２　視線検出部，　２３　バッファ蓄積量監視部，　２４　タイミング制御部，　３１　多重分離部，　３２　ビデオPESバッファ，　３３　オーディオPESバッファ，　３４　キャプションPESバッファ，　３５ないし３７　PESヘッダ検出部，　３８　時刻情報抽出部，　１０１　バス，　１０２　CPU，　１０３　ROM，　１０４　RAM，　１０５　ハードディスク，　１０６　出力部，　１０７　入力部，　１０８　通信部，　１０９　ドライブ，　１１０　入出力インタフェース，　１１１　リムーバブル記録媒体 11 TS buffer, 12 DEMUX, 13 video ES buffer, 14 audio ES buffer, 15 caption buffer, 16 video decoder, 17 audio decoder, 18 caption decoder, 19-21 buffer, 19 to 21 buffer, 22 line of sight detection unit, 23 buffer accumulation amount monitoring unit, 24 timing control unit, 31 demultiplexing unit, 32 video PES buffer, 33 audio PES buffer, 34 caption PES buffer, 35 to 37 PES header detection unit, 38 time information extraction unit, 101 bus, 102 CPU, 103 ROM, 104 RAM , 105 hard disk, 106 output unit, 107 input unit, 108 communication unit, 109 drive, 110 I / O interface, 11 The removable recording medium

Claims (11)

  1. 　ユーザの視線の検出結果に応じて、コンテンツを出力する出力タイミングを制御するタイミング制御部を備える
    　コンテンツ処理装置。     A content processing apparatus including a timing control unit that controls output timing of outputting content according to a detection result of a user's line of sight.
  2. 　前記コンテンツは、ビデオを含み、
    　前記タイミング制御部は、前記出力タイミングの制御として、前記ビデオの出力のスキップ又はリピートの制御を行う
    　請求項１に記載のコンテンツ処理装置。     The content includes a video,
    The content processing apparatus according to claim 1, wherein the timing control unit performs skip or repeat control of the video output as control of the output timing.
  3. 　前記タイミング制御部は、前記ユーザの視線の検出結果に応じて、前記ユーザが、前記ビデオを見ていない場合に、前記スキップ又はリピートが行われやすくなるように、前記スキップ又はリピートを制御する
    　請求項２に記載のコンテンツ処理装置。     The timing control unit controls the skip or repeat according to a detection result of the user's line of sight so that the skip or repeat is easily performed when the user is not watching the video. Item 3. The content processing apparatus according to Item 2.
  4. 　前記ビデオは、ビデオバッファに記憶されてから出力され、
    　前記タイミング制御部は、
    　　前記ビデオバッファに記憶された前記ビデオの蓄積量が、前記スキップを行うスキップ閾値以上である場合に前記スキップの制御を行い、前記ビデオバッファに記憶された前記ビデオの蓄積量が、前記リピートを行うリピート閾値以下である場合に前記リピートの制御を行い、
    　　前記ユーザが、前記ビデオを見ていない場合に、前記スキップ閾値を小さくするとともに、前記リピート閾値を大きくする
    　請求項３に記載のコンテンツ処理装置。     The video is output after being stored in a video buffer,
    The timing controller is
    The skip control is performed when the accumulation amount of the video stored in the video buffer is equal to or greater than a skip threshold value for performing the skip, and the accumulation amount of the video stored in the video buffer performs the repeat. If the repeat threshold is not greater than the repeat control,
    The content processing apparatus according to claim 3, wherein when the user does not watch the video, the skip threshold is decreased and the repeat threshold is increased.
  5. 　前記コンテンツは、オーディオをさらに含み、
    　前記タイミング制御部は、
    　　これから出力しようとする前記ビデオ及び前記オーディオそれぞれを出力すべき出力時刻の差分が、所定の閾値以上である場合に、前記出力時刻の差分が小さくなるように、前記スキップ又はリピートの制御を行い、
    　　前記ユーザが、前記ビデオを見ていない場合に、前記スキップ又はリピートが行われやすくなるように、前記スキップ又はリピートを制御する
    　請求項３に記載のコンテンツ処理装置。     The content further includes audio,
    The timing controller is
    When the difference between the output times at which the video and the audio to be output are to be output is equal to or greater than a predetermined threshold, the skip or repeat control is performed so that the difference between the output times becomes small.
    The content processing apparatus according to claim 3, wherein when the user does not watch the video, the skip or repeat is controlled so that the skip or repeat is easily performed.
  6. 　前記ビデオのスキップでは、前記ビデオの出力が飛ばされるか、又は、複数枚の前記ビデオから１枚のビデオが生成される
    　請求項２に記載のコンテンツ処理装置。     The content processing apparatus according to claim 2, wherein in the video skip, the output of the video is skipped or one video is generated from the plurality of videos.
  7. 　ユーザの視線を検出する視線検出部をさらに備える
    　請求項１に記載のコンテンツ処理装置。     The content processing apparatus according to claim 1, further comprising: a line-of-sight detection unit that detects a user's line of sight.
  8. 　前記ビデオは、前記ビデオを表示する表示装置の表示レートと一致していないフレームレートのビデオ、又は、前記ビデオの伝送レートと一致していない出力レートで出力されるビデオである
    　請求項２に記載のコンテンツ処理装置。     The video is a video having a frame rate that does not match a display rate of a display device that displays the video, or a video that is output at an output rate that does not match a transmission rate of the video. Content processing apparatus.
  9. 　前記コンテンツは、同時に出力すべき前記ビデオと前記オーディオとがずれて出力されるコンテンツである
    　請求項５に記載のコンテンツ処理装置。     The content processing apparatus according to claim 5, wherein the content is content that is output with the video and the audio to be output simultaneously shifted.
  10. 　ユーザの視線の検出結果に応じて、コンテンツを出力する出力タイミングを制御することを含む
    　コンテンツ処理方法。     A content processing method including controlling output timing of outputting content according to a detection result of a user's line of sight.
  11. 　ユーザの視線の検出結果に応じて、コンテンツを出力する出力タイミングを制御するタイミング制御部
    　として、コンピュータを機能させるためのプログラム。     A program for causing a computer to function as a timing control unit that controls output timing of outputting content in accordance with a detection result of a user's line of sight.
PCT/JP2016/074104 2015-09-01 2016-08-18 Content processing device, content processing method, and program WO2017038480A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2015171992A JP2017050677A (en) 2015-09-01 2015-09-01 Content processing device, content processing method and program
JP2015-171992 2015-09-01

Publications (1)

Publication Number Publication Date
WO2017038480A1 true WO2017038480A1 (en) 2017-03-09

Family

ID=58187427

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2016/074104 WO2017038480A1 (en) 2015-09-01 2016-08-18 Content processing device, content processing method, and program

Country Status (2)

Country Link
JP (1) JP2017050677A (en)
WO (1) WO2017038480A1 (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008225046A (en) * 2007-03-13 2008-09-25 Pioneer Electronic Corp Display device
JP2012217097A (en) * 2011-04-01 2012-11-08 Denso Corp On-vehicle moving image display device

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008225046A (en) * 2007-03-13 2008-09-25 Pioneer Electronic Corp Display device
JP2012217097A (en) * 2011-04-01 2012-11-08 Denso Corp On-vehicle moving image display device

Also Published As

Publication number Publication date
JP2017050677A (en) 2017-03-09

Similar Documents

Publication Publication Date Title
US10433002B1 (en) Simultaneously viewing multiple camera angles
US9942622B2 (en) Methods and systems for synchronizing media stream presentations
US8620134B2 (en) Video and audio reproducing apparatus and video and audio reproducing method for reproducing video images and sound based on video and audio streams
US9319566B2 (en) Display apparatus for synchronizing caption data and control method thereof
JP6313704B2 (en) Reception device and synchronization processing method thereof
US20110187927A1 (en) Device and method for synchronisation of digital video and audio streams to media presentation devices
US20180295391A1 (en) Multiplexing apparatus, receiving apparatus, multiplexing method, and delay adjustment method
KR20180083132A (en) Electronic apparatus, and operating method for the same
EP3621309A1 (en) Transmission system for multi-channel image, control method therefor, and multi-channel image playback method and apparatus
US20080198921A1 (en) Method and apparatus for reproducing digital broadcasting
KR100619041B1 (en) Video synchronization apparatus and video synchronization method
WO2014115295A1 (en) Video display device and video display method
CN102413335A (en) Manual adjustment device and method for program audio and video synchronization
US20110064391A1 (en) Video-audio playback apparatus
KR101700626B1 (en) Multi angle view processing apparatus
EP1889488B1 (en) Audio-video processing system
WO2017038480A1 (en) Content processing device, content processing method, and program
US10694240B2 (en) Method for decoding an audio/video stream and corresponding device
US20220408140A1 (en) Moving image reproduction apparatus, moving image reproduction system, and moving image reproduction method
KR20150058809A (en) Apparatus and method for reproducing multi image
KR102634845B1 (en) Device and method for outputting content
JP5938800B2 (en) Video receiving apparatus and video receiving method
JP5476179B2 (en) Tuner switching device, tuner switching system, and method for controlling tuner switching device
KR100499519B1 (en) Method for Controlling Audio Lip-Synchronization
EP3506641A1 (en) Methods and techniques for reducing latency in changing channels in a digital video environment

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16841512

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16841512

Country of ref document: EP

Kind code of ref document: A1