JP2009218874A - Recording/reproducing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a recording/reproducing device which can suppress an increase of throughput and can prevent a thumbnail image not to be seen from being displayed as an original image. <P>SOLUTION: When a thumbnail image generating means 4 generates and stores a thumbnail image corresponding to each position per preset reproduction start position, a voice detection means 6 analyzes voice information stored in a predetermined section corresponding to the reproduction start position corresponding to the thumbnail image, and determines whether a voice of a predetermined pattern previously designated exists in the information. A thumbnail image processing means 7 processes a thumbnail image so that a clarity degree of at least one part of an area is reduced with respect to the thumbnail image corresponding to the voice information in which the presence of the voice of the predetermined pattern is determined. A thumbnail image display means 5 makes a display means 8 display the thumbnail image. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a recording / reproducing apparatus capable of recording and reproducing video information and audio information, and in particular, generates and displays a thumbnail image related to a reproduction start position when reproducing recorded video information and audio information. The present invention relates to a recording / reproducing apparatus having a function to

  As is well known, recording / playback of a DVD (Digital Versatile Disc) recorder or HDD (Hard Disc Drive) recorder that records and plays back video information and audio information (hereinafter referred to as content) such as program content broadcast from a broadcasting station. The device exists.

  In the recording / reproducing apparatus, a number of reproduction start positions can be defined for the recorded content. This playback start position is usually called a bookmark, a chapter or simply a mark. In addition, the reproduction start position information (mark information) may be held and used in a recording / reproducing apparatus, or may be used by recording the information on a recording medium and reading it out.

  The above-described mark can be arbitrarily set by the title creator or the user who reproduces the content. In some cases, the recording / playback apparatus can automatically set a mark by detecting a change in audio information or a change in video information. Then, the moving image recorded at the position corresponding to the set mark is captured as a frame image, reduced as necessary, and displayed as a thumbnail image list (thumbnail image list) on the screen.

  In some cases, the moving image at the mark position is not captured, but another still image for the thumbnail image is prepared and the still image is displayed on the screen. Then, by viewing the thumbnail images displayed in the thumbnail image list, the user can roughly understand the contents of the content stored at the position corresponding to the mark, and can start reproduction from that position.

  As a conventional example of generating and displaying a thumbnail image, the user is allowed to specify specific information for specifying video information on which the thumbnail image is based, and the specific information is stored, while the recording of the content on the recording medium is finished, There is one that acquires video information indicated by specific information from content recorded on a recording medium, generates a thumbnail image from the obtained video information, and stores the thumbnail image (for example, see Patent Document 1 below).

In addition, in order to distinguish and display the face of a person appearing in the video, the face is detected from the video to identify the face, while the frame in which the face is reflected is detected from the video, and the face image is detected from the frame. There is a technique for identifying the characters in the video by extracting and grouping the faces of the same character from all the extracted face images, extracting the representative face image for each character (see Patent Document 2 below) ).
JP 2003-32581 A (summary) JP 2001-167110 A (summary)

  However, as described in Patent Document 1, the user specifies specific information for specifying video information that is the basis of the thumbnail image, and acquires video information indicated by the specific information from the content recorded on the recording medium. The thumbnail image generated from the obtained video information may include an image that is not problematic for the user but is not desired to be seen by others or children. One example is an image related to a person such as an individual's face.

  Therefore, when it is not desired to show a person image or the like, for example, if the technique described in Patent Document 2 is used, it is analyzed whether or not there is a person or the like in the thumbnail image. Can be displayed. However, the processing for detecting a person in an image has a problem that the processing amount is large even if the screen is about the size of a thumbnail image.

  The present invention has been made in view of the above-described problems, and its object is to prevent the display of thumbnail images that are not desired to be viewed as the original images while suppressing an increase in the processing amount. It is to provide a recording / reproducing apparatus.

In order to achieve the above object, the present invention relates to video and audio recording means for recording video information and audio information related to the video information in association with each other on a recording medium,
Video / audio reproduction means for reproducing the video information and the audio information recorded on the recording medium;
Display means for displaying the video information reproduced by the video / audio reproduction means;
Reproduction start position setting means for setting a reproduction start position of the video information and the audio information by the video / audio reproduction means;
Thumbnail image generation means for generating a thumbnail image corresponding to each position for each of the playback start positions set by the playback start position setting means, and storing the generated thumbnail images;
Acquire audio information recorded in a predetermined section corresponding to the playback start position corresponding to the thumbnail image generated by the thumbnail image generation means, analyze the acquired audio information, and specify in advance in the audio information Voice detection means for determining whether or not there is a predetermined pattern of voice;
The thumbnail image generation means stored in the thumbnail image generation means so as to reduce the clarity of at least a part of the thumbnail image corresponding to the audio information determined by the sound detection means to have the sound of the predetermined pattern. Thumbnail image processing means for processing thumbnail images and re-storing them in the thumbnail image generating means;
Thumbnail image display means for causing the display means to display the thumbnail image stored in the thumbnail image generation means;
The recording / reproducing apparatus provided.

  According to the present invention, it is possible to prevent a thumbnail image that is not desired to be viewed from being displayed as an original image while suppressing an increase in processing amount.

  Hereinafter, the present invention will be described in detail based on preferred embodiments shown in the drawings. FIG. 1 is a functional block diagram showing a configuration of an embodiment of a recording / reproducing apparatus according to the present invention. In the present embodiment, for example, video / audio recording means 1 records video information and audio information on a recording medium 10 such as a DVD or HD, while video / audio reproduction means 2 records video recorded on the recording medium 10. Play information and audio information. The reproduced video information is displayed on the display means 8. When the video information is reproduced by the video / audio reproduction means 2, the title of the content including the video information may be displayed. The reproduction start position setting means 3 is a means for setting a mark for video information. Specifically, the content recorded on the recording medium 10 is automatically set by setting a mark based on the mark information input by the user, or the apparatus itself detects a change in audio information or a change in video information. The mark is set automatically. The thumbnail image generation unit 4 generates a thumbnail image in association with the mark set by the reproduction start position setting unit 3.

  In the present embodiment, the sound information corresponding to the position of the thumbnail image generated by the thumbnail image generating means 4 is acquired, the acquired sound information is analyzed, and a predetermined pattern of sound specified in advance in the sound information is analyzed. The sound detection means 6 for determining whether or not there is sound, and the clarity of at least a part of the thumbnail image corresponding to the sound information determined to have sound of a predetermined pattern by the sound detection means 6. The thumbnail image processing means 7 for processing the thumbnail images stored in the thumbnail image generation means 4 and the thumbnail image display for causing the display means 8 to display the processed thumbnail images stored in the thumbnail image generation means 4 as a list. Means 5 are provided. In this way, by reducing the clarity of thumbnail images corresponding to audio information having a predetermined pattern of audio, an increase in the processing amount is suppressed, and thumbnail images that are not desired to be displayed are displayed as the original images. This can be prevented.

  FIG. 2 is a block diagram showing a specific configuration of the recording / reproducing apparatus shown in FIG. In FIG. 2, a recording medium 10 is a disk capable of recording and reproducing a video file and a still image data file, and a drive device 11 that rotates and drives the recording medium 10 to read / write information and read / write information. And a buffer 12 for storing data to be stored. As a recording side component, an encoder unit 13 is provided that encodes audio and video data captured via an input unit 16 such as a TV tuner and supplies the encoded audio and video data to the buffer 12. In addition, as a component on the reproduction side, an image decoder unit 14 that receives image data from the buffer 12 and decodes it, and stores still image data obtained by decoding and capturing by the image decoder unit 14, as well as thumbnails When image data is prepared in the recording medium 10, a still image data unit 15 that extracts and stores the thumbnail image data and an audio decoder unit 22 that receives audio data from the buffer 12 and decodes the data are provided. ing.

  In addition, the audio detection unit 17 that acquires audio information corresponding to the position of the thumbnail image from the audio decoder unit 22 and determines whether the audio information includes a predetermined pattern of audio specified in advance by the user; A display device 18 that displays image data obtained by decoding by the image decoder unit 14 or image data stored in the still image data unit 15, and a method for causing a user to select a mark position according to a video displayed on the display device 18. The operation of the selection means 19 composed of, for example, a mouse, a remote controller, etc., and the operation of the encoder unit 13, the image decoder unit 14, the still image data unit 15, the audio detection unit 17, the display device 18, and the audio decoder unit 22 are controlled. Control means 20 composed of a CPU or the like and control information such as information indicating whether or not to detect a person are set. And a memory 21.

  Regarding the operation of the recording / reproducing apparatus configured as described above, the thumbnail image display processing will be described with reference to the flowcharts of FIGS. FIG. 3 shows a thumbnail image display process performed mainly by the control means 20 constituted by a CPU or the like. Here, it is assumed that titles of a plurality of contents recorded on the recording medium 10 are displayed on the display device 18. Therefore, first, in step S101, the user looks at the title displayed on the display device 18, and uses the selection means 19 to designate a frame for generating a thumbnail image. Next, in step S102, the frame specified by the user is interpreted, and the location of the recording medium 10 from which the content corresponding to this frame is to be read is determined. Next, in step S103, the MPEG-2 moving image data recorded at the location determined in step S102 is acquired from the encoded moving image recorded in the recording medium 10 via the driving device 11. . Next, in step S104, the moving image data acquired in step S103 is decoded and captured by the image decoder unit 14, and the thumbnail image data for the frame specified in step S101 is stored in the still image data unit 15. . If another thumbnail image data corresponding to the frame specified in step S101 is prepared in the recording medium 10, the MPEG-2 moving image recorded in the specified frame is not read. The thumbnail image data is read and stored in the still image data section 15.

  Next, in step S105, it is determined whether the thumbnail image data stored in the still image data unit 15 in step S104 is prepared separately from the moving image data. If it is determined in step S105 that the thumbnail image data is prepared separately from the moving image data, the processes in steps S106 and S107 are skipped and the process proceeds to step S108. On the other hand, if it is determined in step S105 that the thumbnail image data is not prepared separately from the moving image data, that is, if it is determined that the thumbnail image data is obtained by decoding the moving image data. From the memory 21, it is determined whether it is a mode in which all thumbnail images can be displayed in step S106 or a mode in which at least a part of the area is displayed with reduced clarity (thumbnail non-display mode). Get information and make decisions. This information is set in the memory 21 by any one of the following methods (1) and (2).

(1) The user sets in advance whether the mode enables the display of all thumbnail images by the selection means 19 or the mode for reducing the clarity of a predetermined area of the thumbnail image (thumbnail non-display mode). Then, the information is stored in the memory 21.
(2) The viewing restriction information (information for restricting programs and contents that can be viewed) recorded in the recording medium to be reproduced or in the contents is stored in the memory 21.

As described above, in step S106, information stored in the memory 21 is acquired, and it is determined whether or not the thumbnail non-display mode is set. If it is determined that the thumbnail non-display mode is selected, the person detection and processing of the thumbnail image are performed using the audio detection unit 17 in step S107. The details of the processing related to step S107 will be described later. If it is determined in step S106 that the mode is not the thumbnail non-display mode, the process proceeds to step S108. In step S108, an image obtained by processing the thumbnail image or the thumbnail image as it is is displayed on the display device 18.
The above is a rough flow of the thumbnail image display processing.

Next, the person detection and processing of thumbnail images performed in step S107 will be described with reference to the flowchart of FIG. First, in step S201, audio data of a predetermined section near the place where the thumbnail image is acquired is acquired via the driving device 11 for several seconds. In step S202, the audio data acquired in step S201 is decoded via the audio decoder unit 22. Next, in step S203, it is determined whether or not there is a human voice as a predetermined pattern of voice specified in advance in the voice data acquired in step S202 via the voice detection unit 17. If it is determined in step S203 that there is a human voice, image processing is performed on the thumbnail image area (x, y)-(x1, y2) (rectangular area) in step S204. Thereafter, the process proceeds to step S205, where the image-processed image is combined with the original image, and a thumbnail image subjected to image processing is generated.
The above is the flow of person detection and processing of thumbnail images.

  Here, an example of a method for acquiring audio data corresponding to the place where the thumbnail image is acquired will be described below. When the content is an MPEG2 transport stream, as shown in FIG. 6, the image data, audio data, and other data of the content are stored in a plurality of transport packets composed of 188 bytes. Specifically, each transport packet includes header information and a payload, and one frame of image data or audio data is divided and stored together with control information in the payload portion.

  The data composed of a combination of several payload parts is a PES packet header including information such as PES (Packetized Elementary Stream) packet data obtained by dividing an image stream or audio stream into an appropriate size and packetized, and reproduction time It is made up of. One of the information in the PES packet header is a time (PTS: Presentation Time Stamp) at which image data and audio data included in the PES packet data are displayed.

  In order to search for audio data corresponding to a thumbnail image, as shown in FIG. 7, a PTS of image data V (Video Data) selected by a user or automatically selected by a device is acquired as a thumbnail image. Then, the audio data A (Audio Data) arranged after the image data V is searched, and the audio data A having the same PTS is selected. In general, image data and audio data having the same PTS are specified because the audio data A is arranged after the image data V because of the relationship between the size of the image data and the audio data per unit time. You only need to search behind the place you did. In the figure, SP is sub-picture data.

Here, a method for detecting a human voice in determining whether there is a human voice in the audio data performed in step S203 will be described. The human voice is detected using, for example, the following method (1) or (2).
(1) The voice sampling frequency is checked, and the level between human voice frequencies (about 100-3500 Hz) is above a certain value. Before performing frequency determination, there are cases where determination is made after adding audio band correction, dynamic range compression, noise canceller, or the like.
(2) Further, the decoded speech is converted into feature amount data, and speech recognition is performed while comparing the feature with the already recorded data to recognize whether the speech contains a human language. In this case, it is not necessary to recognize even a word, and it is sufficient to recognize whether it is a language.

Next, the image processing process performed in step S204 will be described below. The image processing performed in step S204 does not know the image in detail, but the image processing that can be understood what the image is blurry, that is, by using the image processing that reduces the clarity, from the image Use what you can guess what the image is. As an example,
・ Image processing is performed by image processing methods such as blurring and mosaic processing.

Here, the “blurring process” is a process of calculating the color of a pixel at a certain point and the color of the surrounding four pixels (up, down, left and right) to obtain the color of the pixel at a certain point.
For example, assuming that the ratio of the color of the pixel to be processed and the color of surrounding pixels is 2: 1, the color of the pixel (x, y) to be processed is as follows.
Ambient color average = ((x, y + 1) color + (x + 1, y) color + (x, y−1) color + (x−1, y) color) / 4
Color component of (x, y) = (color component of (x, y) × 2 + average of surrounding color components) / 3
The above calculation is performed for each of R, G, and B colors.

  In the above example, the surrounding 4 pixels are used and mixed at a ratio of 2: 1. However, not only 4 pixels are necessarily used, but one pixel may be used in the upper, lower, left, and right directions for each pixel.

  “Mosaic processing” is a method of replacing a certain area with the average color of the pigment in the area. For example, the average value of the pigment in the region is obtained every 4 pixels × 4 pixels, and the region of 4 pixels × 4 pixels is replaced with the average color. This is not necessarily 4 pixels × 4 pixels, but can be a different size.

Next, a process for determining an area (hereinafter referred to as a blur area) for performing “blurring process” or “mosaic process” will be described below. The following two types of areas (1) and (2) can be selected as the blur area.
(1) Perform on the entire thumbnail image.
(2) Perform only on a part of the thumbnail image.
In the case of only a part, there are various methods for determining the location of the part, but this time, from the viewpoint of suppressing the processing amount, as an example, a method of extracting a skin color part and making it an area to be blurred will be described with reference to the flowchart of FIG. .

First, in step S301, it is determined whether to blur the entire thumbnail image area or only a part of the area. If it is determined that the entire thumbnail image is to be blurred, the region of the entire thumbnail image is determined to be the blurring processing pair region in step S306. If it is determined that a part is to be blurred, in step S302, if the color notation data of the thumbnail image is RGB, each pixel is converted to YUV data. This conversion formula is performed by the following conversion formula.
Y = (77 * R / 256) + (150 * G / 256) + (29 * B / 256)
U (Cb) =-(44 * R / 256)-(87 * G / 256) + (131 * B / 256) +128
V (Cr) = (131 * R / 256)-(110 * G / 256)-(21 * B / 256) +128
(If the value is less than 0, 0, and if it is more than 256, 255.)

Next, in order to delete the flesh color that has entered as noise, noise detection and removal processing of the thumbnail image is performed in step S303. For noise detection and removal processing, a technique well known in image processing is used. This time, as an example, the edge of the thumbnail image is extracted, and for each pixel extracted as an edge, there is a color connection either up, down, left, or right with respect to each target pixel and its top, bottom, left, and right pixels This is a method of removing noise by averaging with the target pixel in a direction where there is more connection. Of course, other methods can be employed. Next, in step S304, it is checked whether or not the YUV pixel data of each pixel is within the threshold value determined as the skin color. The skin color threshold is the above YUV data.
64 ≦ Y ≦ 240, 94 ≦ U ≦ 125, 131 ≦ V ≦ 254
And This threshold value can be changed according to the image condition of the thumbnail image.

  Next, in step S305, the region processed in steps S302 to S304 and determined as the skin color is set as (x, y)-(x1, y2). There may be a plurality of such areas. Finally, the blurring process is performed in step S307 on the region (x, y)-(x1, y2) set in step S305 or step S306.

  As is apparent from the above description, according to the present embodiment, when displaying a thumbnail image, while suppressing an increase in processing amount, a thumbnail image that is not desired to be displayed is displayed as the original image. Can be prevented. As a result, it is possible to conceal information such as images related to personal information and adult images from other people and children and to display a thumbnail image list with peace of mind.

  In the above embodiment, human beings are targeted. However, not only human display, but there is a thumbnail image that the content owner desires to lower the clarity, there is a predetermined pattern of sound indicating the thumbnail image. It is also possible to determine whether or not the thumbnail image is processed and displayed based on the result. For example, when it is not desired to display a thumbnail image of a dog, it is determined whether there is a dog's voice, and the thumbnail image is not displayed as the original image.

It is a functional block diagram which shows the structure of one Embodiment of the recording / reproducing apparatus which concerns on this invention. It is a block diagram which shows the specific structure of the recording / reproducing apparatus shown in FIG. FIG. 3 is a flowchart showing a rough flow of thumbnail image display processing in order to explain the operation of the recording / reproducing apparatus shown in FIG. 2. FIG. FIG. 3 is a flowchart showing a flow of processing for detecting and processing a person in a thumbnail image in order to explain the operation of the recording / reproducing apparatus shown in FIG. FIG. 3 is a flowchart showing the flow of blur area detection processing in order to explain the operation of the recording / reproducing apparatus shown in FIG. 2. FIG. FIG. 3 is a diagram showing the structure of an MPEG2 transport stream in order to explain the operation of the recording / reproducing apparatus shown in FIG. FIG. 3 is a diagram showing the relationship between the positions of image data and audio data in order to explain the operation of the recording / reproducing apparatus shown in FIG. 2.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Recording medium 11 Drive apparatus 12 Buffer 13 Encoder part 14 Image decoder part 15 Still image data part 16 Input means 17 Audio | voice detection part 18 Display device 19 Selection means 20 Control means 21 Memory 22 Audio | voice decoder part

Claims (1)

Video and audio recording means for recording video information and audio information related to the video information in association with each other on a recording medium;
Video / audio reproduction means for reproducing the video information and the audio information recorded on the recording medium;
Display means for displaying the video information reproduced by the video / audio reproduction means;
Reproduction start position setting means for setting a reproduction start position of the video information and the audio information by the video / audio reproduction means;
Thumbnail image generation means for generating a thumbnail image corresponding to each position for each of the playback start positions set by the playback start position setting means, and storing the generated thumbnail images;
Acquire audio information recorded in a predetermined section corresponding to the reproduction start position corresponding to the thumbnail image generated by the thumbnail image generation means, analyze the acquired audio information, and specify in advance in the audio information Voice detection means for determining whether or not there is a predetermined pattern of voice;
The thumbnail image generation means stored in the thumbnail image generation means so as to reduce the clarity of at least a part of the thumbnail image corresponding to the audio information determined by the sound detection means to have the sound of the predetermined pattern. Thumbnail image processing means for processing thumbnail images and re-storing them in the thumbnail image generating means;
Thumbnail image display means for causing the display means to display the thumbnail image stored in the thumbnail image generation means;
Recording / reproducing apparatus provided.

Images