JP2009212605A - Information processing method, information processor, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To highly accurately classify respective pixel data in image data in a short time. <P>SOLUTION: A recording and reproducing device, as initial clustering processing, decides whether or not the feature value of pixel data to be input is within a prescribed threshold from the feature value of already input pixel data every time the pixel data are input, and classifies the pixel data into a first number of clusters by imparting the same ID as the one of the already input pixel data when they are within the prescribed threshold. After the initial clustering processing, the recording and reproducing device classifies the pixel data classified into the first number of clusters into a second number (target number) of clusters by K-means or the like, as main clustering processing. Since the number of the clusters of the pixel data to be the object of the main clustering processing is reduced by the initial clustering processing, the main clustering processing is accelerated. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an information processing method, an information processing apparatus, and a program for classifying pixel data constituting image data into a plurality of clusters.

  Conventionally, a process (clustering process) for classifying pixel data constituting image data based on predetermined feature data such as a color feature and a texture feature is known. This clustering process is a typical technique for executing an image segmentation process for extracting a plurality of regions from image data, for example. Image segmentation processing is used as preprocessing in various fields of image processing such as object coding and image search.

  Typical clustering methods include, for example, K-means (K-means), fuzzy c-means (Fussy c-means), etc. In these clustering methods, until the data converges and the clustering process is completed. There is a problem that it takes an enormous amount of time.

As one of the techniques for solving such a problem, for example, in Patent Document 1 below, the first thinned image Ia obtained by thinning the input image Ii to be divided at the first thinning rate Ra is divided into regions by the K average algorithm. Then, the approximate number and center of the regions of the input image Ii are estimated, and then the estimation result is used to thin out the input image Ii at a second thinning rate Rb that is smaller than the first thinning rate Ra. The thinned-out image Ib is divided into regions by the K-average algorithm to calculate the number and center of the input image Ii, and then included in the input image Ii in the most appropriate region among the calculated regions An image region dividing method for allocating all pixels to be recorded is described.
Japanese Patent Laying-Open No. 2005-011375 (paragraph [0012] etc.)

  However, since the technique described in Patent Document 1 uses a K-average algorithm when the first thinned image and the second thinned image are each divided into regions, even the thinned image is divided into two regions. Processing takes time. Further, if the thinning rate is increased, the area division processing time can be shortened, but the accuracy of the area division processing is lowered. Since the divided region generated by using the thinned image determines the divided region to which all the pixels included in the input image are allocated, the decrease in the accuracy of the region dividing process using the thinned image is reduced. This also affects the accuracy of subsequent image processing using the region.

  In view of the circumstances as described above, an object of the present invention is to provide an information processing method, an information processing apparatus, and a program capable of classifying pixel data in image data with high accuracy and in a short time.

  In order to solve the above-described problem, an information processing method according to a main aspect of the present invention sequentially inputs a plurality of pixel data constituting image data, and a predetermined feature value of each input pixel data is already input. It is determined for each input whether or not the pixel value is within a predetermined range value from the feature value of the pixel data, and the pixel data that has been input to the pixel data that is determined that the feature value is within the predetermined range value The first identification information that is the same as the data is given, and the second identification information that is different from the first identification information is given to the pixel data determined that the feature value is not within the predetermined range value, A plurality of pixel data is classified into a first number of clusters.

  Here, the predetermined feature value is, for example, a color feature or texture feature of each pixel expressed as multidimensional feature vector data.

  According to the configuration of the present invention, a conventional clustering technique such as K-means is performed by sequentially adding identification information to pixel data according to whether or not the feature value of the input pixel data is a predetermined range value. It is possible to classify pixel data with high accuracy and in a short time compared to the case of using.

  The information processing apparatus further regards the pixel data classified into the same cluster by the classification as pixel data having the same feature value, and determines the plurality of classified pixel data by the predetermined clustering method. You may classify | categorize into the 2nd number cluster smaller than the number of 1. FIG.

  Here, the predetermined clustering method is, for example, a K-means method, a fuzzy c-means method, an entropy method, a Ward's method, a self-organizing map (Self -organizing maps, SOM), etc., but is not limited to these. The pixel data having the same feature value is, for example, the average value of the feature values of the pixel data belonging to the same cluster.

  In addition, when there are a plurality of pieces of already input pixel data, the first identification information is a plurality of different pieces of identification information. When the feature value of the pixel data to which the identification information is to be added is not within a predetermined range value from any of the feature values of the pixel data having the plurality of different pieces of identification information, the second identification information is included in the pixel data. Is granted. In this case, the first number is three or more. That is, the above configuration does not mean that the number of clusters is only two, but means that the pixel data is classified into at least two clusters.

  According to the configuration of the present invention, the input pixel data is first classified into a first number of clusters according to whether the feature value is within a predetermined range value, and the pixel data classified into the same cluster is It can be regarded as pixel data having the same feature value, and the number (class) of pixel data is reduced, and then each pixel data can be classified into a second number of clusters. In this way, by reducing the number of classes of pixel data and then performing clustering, the number of processes and processing time required for convergence of the clustering process are greatly increased compared to conventional clustering methods that directly cluster a large number of data. Can be reduced. That is, the processing load of the clustering process can be reduced as much as possible.

  Further, since all the pixel data constituting the image data is used to classify into the first number of clusters, for example, the processing time is shortened as compared with a case where an image obtained by thinning image data at a predetermined thinning rate is used. However, the classification process can be executed with higher accuracy.

  The information processing method may further include a plurality of continuous first pixel data existing in a predetermined direction among the plurality of pixel data classified into the first number of clusters, and the predetermined direction. A plurality of continuous second pixel data different from the first pixel data, and at least one third pixel data existing between the first pixel data and the second pixel data in the predetermined direction. Pixel data is extracted, the identification information of the first pixel data and the second pixel data is the same, and the identification information of the third pixel data is the first and second pixel data May be replaced with the identification information of the first pixel data and the identification information of the second pixel data.

  Thus, when the identification information of the third pixel data existing between the first pixel data and the second pixel data is different from the identification information of the first and second pixel data, the third pixel The noise can be removed by regarding the data as noise and replacing the identification information. Thereby, the said classification process can be performed more efficiently. Here, the predetermined direction is, for example, the X direction or the Y direction.

  In the information processing method, a high-frequency component in each input pixel data may be removed by a low-pass filter.

  Thereby, noise that could not be replaced by the replacement process can be removed, and the classification process can be executed more efficiently.

  The information processing method may further divide the image data into the second number of regions having an arbitrary shape based on a plurality of pixel data classified into the second number of clusters.

  Thereby, so-called image segmentation processing can be executed using the pixel data classified into the second number of clusters.

  The information processing method further detects a motion vector between the plurality of image data for each of the divided second number of regions, and uses the plurality of image data based on the detected motion vector. A predetermined video feature generated by camera operation may be detected in the video data to be configured.

  Accordingly, the video feature can be efficiently detected by detecting the motion vector for each of the second number of regions divided based on the second number of clusters. Here, the predetermined video features generated by the camera operation are motion features such as pan, tilt, and zoom.

  In the information processing method, the step of detecting the video feature calculates the number of pixels in the second number of regions of the plurality of image data, respectively, and the number of pixels in the plurality of image data is the largest. The predetermined video feature in the video data may be detected based on a motion vector detected between regions.

  Here, the motion vector is detected between the regions having the largest number of pixels (area) when the image data is divided into a moving object region and a background image region in the video data. This is because the background video is considered to have a larger area than the moving object, and the movement of the background video is considered to indicate the camera operation in the video data. As a result, even if there is a moving object in the video data, it is possible to detect the video feature by ignoring the movement of the moving object and detecting only the camera movement and detecting the motion vector.

  The information processing method may further encode the pixel data for each of the divided second number of regions.

  Thereby, an object encoding process can be performed more efficiently.

  The information processing method further generates a feature vector for each of the divided second number of regions, and generates the generated feature vector for each of the second number of clusters between the plurality of image data. In comparison, the similarity between the plurality of image data may be determined.

  Thereby, other image data similar to certain image data can be searched efficiently.

  In the information processing method, the step of determining similarity calculates the number of pixels in the second number of regions of the plurality of image data, respectively, and the number of pixels is the largest among the plurality of pixel data. The feature vectors may be compared between regions to determine the similarity between the plurality of image data.

  Here, the reason why the feature vectors are compared between the regions having the largest number of pixels (area) is that the region having the largest area is considered to have the most influence on the similarity between the image data. As a result, the similarity determination process between the image data can be executed more efficiently and at high speed.

  In the information processing method, the step of classifying into the first number of clusters may repeat the classification by changing the predetermined range value until the first number reaches a predetermined number.

  As a result, the number of clusters can be reduced to an optimum number by repeating the classification process by changing the predetermined range value according to the classification result.

  In the information processing method, the step of classifying into the first number of clusters controls not to execute the classification into the second number of clusters when the first number reaches a predetermined number. It doesn't matter.

  Thereby, when the first number can be classified into a sufficiently small number, the processing time can be further shortened by not performing further classification.

  An information processing apparatus according to another aspect of the present invention includes an input unit that sequentially inputs a plurality of pixel data constituting image data, and a feature value of each input pixel data is a feature value of already input pixel data. Is determined for each input, and the pixel data for which the feature value is determined to be within the predetermined range value is the same as the first input pixel data. Identification information is provided, second identification information different from the first identification information is assigned to pixel data for which the feature value is determined not to be within the predetermined range value, and the plurality of pixel data are set to the first A first classifying unit that classifies the plurality of classified data into a plurality of clusters, and the pixel data classified into the same cluster is regarded as pixel data having the same feature value, and the plurality of classified pixel data are subjected to a predetermined clustering method. The first ; And a second classifying means for classifying the smaller second number of clusters.

  Here, the information processing apparatus is, for example, a recording / playback apparatus such as a PC (Personal Computer), an HDD (Hard Disk Drive) / DVD / BD (Blu-ray Disc) recorder, a server apparatus, a television apparatus, a game machine, or a digital camera. And various electronic devices such as digital video cameras and mobile phones.

  According to the configuration of the present invention, the first classifying unit first classifies the input pixel data into the first number of clusters according to whether the feature value is within the predetermined range value, and the same cluster. The pixel data classified into (2) is regarded as pixel data having the same feature value, the number of classes (types) of the pixel data is reduced, and then each pixel data is converted into the second data by the second classification means. It can be classified into a number of clusters. In this way, by reducing the number of classes of pixel data and then performing clustering, the number of processes and processing time required for convergence of the clustering process are greatly increased compared to conventional clustering methods that directly cluster a large number of data. Can be reduced. That is, the processing load of the clustering process can be reduced as much as possible. Further, since all the pixel data constituting the image data is used to classify into the first number of clusters, for example, the processing time is shortened as compared with a case where an image obtained by thinning image data at a predetermined thinning rate is used. However, the classification process can be executed with higher accuracy.

  A program according to still another aspect of the present invention includes a step of sequentially inputting a plurality of pixel data constituting image data to an information processing apparatus, and a predetermined feature value of each input pixel data is already input. It is determined for each input whether or not the feature value is within a predetermined range value from the feature value of the pixel data, and the already-input pixel data is added to the pixel data that is determined that the feature value is within the predetermined range value. And the second identification information different from the first identification information to the pixel data determined that the feature value is not within the predetermined range value, Classifying the pixel data into a first number of clusters, the pixel data classified into the same cluster as pixel data having the same feature value, Cluster It is for and a step of classifying the first second number of clusters fewer than the number by grayed technique.

  As described above, according to the present invention, each pixel data in the image data can be classified with high accuracy and in a short time.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
First, a first embodiment of the present invention will be described. In this embodiment, a recording / reproducing apparatus is applied as the information processing apparatus.

FIG. 1 is a diagram showing a configuration of a recording / reproducing apparatus 100 according to the present embodiment.
As shown in the figure, a recording / reproducing apparatus 100 includes a CPU (Central Processing Unit) 1, a RAM (Random Access Memory) 2, an operation input unit 3, a segmentation processing unit 20, a video feature detection unit 4, a digital tuner 5, an IEEE1394. Interface 6, Ethernet (registered trademark) / wireless LAN (Local Area Network) interface 7, USB (Universal Serial Bus) interface 8, memory card interface 9, HDD 10, optical disk drive 11, buffer controller 13, selector 14, demultiplexer 15, An AV (Audio / Video) decoder 16, an OSD (On Screen Display) 17, a video D / A (Digital / Analog) converter 18, and an audio D / A converter 19 are provided.

  The CPU 1 appropriately accesses the RAM 2 or the like as necessary, and controls the entire blocks of the recording / reproducing apparatus 100. The RAM 2 is a memory that is used as a work area of the CPU 1 and temporarily stores an OS (Operating System), a program, processing data, and the like.

  The operation input unit 3 includes a button, a switch, a key, a touch panel, a light receiving unit for an infrared signal transmitted from a remote controller (not shown), and the like. Output to.

  Under the control of the CPU 1, the digital tuner 5 receives a broadcast signal of a digital broadcast program through an antenna (not shown), and selects and demodulates a broadcast signal of a specific channel. This broadcast signal is output to the demultiplexer 15 via the selector 14 for reproduction, or recorded on the HDD 10 via the buffer controller 13 or recorded on the optical disk 12 inserted in the optical disk drive 11.

  The IEEE1394 interface 6 can be connected to an external device such as a digital video camera. For example, video content shot and recorded by a digital video camera is played back or recorded on the HDD 10 or the optical disc 12 in the same manner as the video content of a broadcast program received by the digital tuner 5.

  The Ethernet (registered trademark) / wireless LAN interface 7 inputs, for example, video content recorded on a PC or other recording / playback apparatus via Ethernet (registered trademark) or wireless LAN. This video content can also be played back and recorded on the HDD 10 or the optical disc 12.

  The USB interface 8 inputs video content from a device such as a digital camera or an external storage device such as a so-called USB memory via the USB. This video content can also be played back and recorded on the HDD 10 or the optical disc 12.

  The memory card interface 9 is connected to, for example, a memory card with a built-in flash memory, and inputs video content recorded on the memory card. This video content can also be played back and recorded on the HDD 10 or the optical disc 12.

  The HDD 10 records various video contents received as a broadcast signal or input from an external device on a built-in hard disk, and reads them from the hard disk and outputs them to the buffer controller 13 during reproduction. The HDD 10 also stores an OS, a program for executing image segmentation processing and video feature detection processing, which will be described later, and various other programs and data. Note that the recording / reproducing apparatus 100 may store the OS, various programs, and data in another recording medium such as a flash memory (not shown) instead of the HDD 10.

  The optical disk drive 11 records the video content and the like on the optical disk 12, reads them during reproduction, and outputs them to the buffer controller 13. The optical disk 12 is, for example, a DVD, a BD, a CD, or the like.

  For example, the buffer controller 13 controls the timing and amount of data written to the HDD 10 or the optical disk 12 of video content continuously supplied from the digital tuner 5 and other various interfaces, and writes the video content intermittently. . Further, the buffer controller 13 controls the read timing and data amount of the video content recorded on the HDD 10 and the optical disk 12 and continuously supplies the video content read intermittently to the demultiplexer 15.

  The selector 14 selects video content input from any of the digital tuner 5, various interfaces, the HDD 10, and the optical disk drive 11 based on a control signal from the CPU 1.

  The demultiplexer 15 separates the multiplexed video content input from the buffer controller 13 into a video signal and an audio signal, and outputs them to the AV decoder 16.

  The AV decoder 16 decodes a video signal and an audio signal encoded in a format such as MPEG (Moving Picture Expert Group) -2 or MPEG-4, for example, and converts the video signal to the OSD 17 and the audio signal to the D / A. Output to the converter 19.

  The OSD 17 generates graphics or the like to be displayed on a display (not shown), performs synthesis processing and switching processing with the video signal, and outputs the processed video signal to the video D / A converter 18. The video D / A converter 18 converts the video signal subjected to graphic processing by the OSD 17 into an NTSC (National Television Standards Committee) signal by D / A conversion, and outputs and displays it on a display (not shown).

  The audio D / A converter 19 D / A converts the audio signal input from the AV decoder 16 and outputs it to a speaker (not shown) for reproduction.

  The segmentation processing unit 20 extracts a plurality of areas (objects) from the video data before decoding by the AV decoder 16 or the image data constituting the video data after decoding. Details of this segmentation processing will be described later.

  The video feature detection unit 4 detects video features generated by camera operations such as panning, tilting, and zooming from the segmented video data. Details of this video feature detection processing will also be described later.

FIG. 2 is a diagram showing an outline of the segmentation process.
As shown in FIG. 6A, segmentation processing is performed from an original image including objects such as a building 21, a person 22, and a road 23, as shown in FIG. This is a process of extracting the areas 21a, 22a and 23a corresponding to the road 23 (a process of dividing the areas 21a, 22a and 23a). Conventional segmentation processing is executed by classifying each pixel data constituting an original image into a plurality of clusters using a clustering technique such as K-means. However, in this conventional segmentation process, it takes time until the data is converged by the clustering process, which is a burden on the entire system. Therefore, in this embodiment, as will be described later, the clustering process is performed in two stages to reduce the processing time.

FIG. 3 is a diagram showing the configuration of the segmentation processing unit 20.
As shown in the figure, the segmentation processing unit 20 includes an LPF (low-pass filter) processing unit 31, an ID processing unit 32, a threshold setting unit 33, an interpolation processing unit 34, and a K-means processing unit 35.

  The LPF processing unit 31 sequentially reads pixel data constituting each image data in the video data, for example, from the HDD 10 or the like, removes high frequency components (noise) in each pixel data, and outputs them to the ID processing unit 32. For example, the LPF processing unit 31 corrects the value of the input pixel data so as to be the average value of the immediately preceding and immediately following pixel data.

  The ID processing unit 32 classifies each pixel data into a predetermined number of clusters by giving identification data (ID) to the pixel data sequentially input from the LPF processing unit 31 based on a predetermined threshold value. The data is output to the interpolation processing unit 34. Details of the processing in the ID processing unit 32 will be described later.

  The threshold value setting unit 33 sets a threshold value (range) that serves as a reference for ID assignment in the ID processing unit 32.

  The interpolation processing unit 34 performs ID replacement processing on each pixel data to which the ID is assigned by the ID processing unit 32 based on a predetermined condition, and outputs the result to the K-means processing unit 35. Details of this interpolation processing will be described later.

  The K-means processing unit 35 classifies the pixel data input from the interpolation processing unit 34 into a plurality of clusters by clustering processing based on the K-means method, and outputs the classification result as a region extraction result.

  Each image data processed in this embodiment is, for example, VGA (Video Graphic Array) size (640 × 480 dots). Therefore, 640 × 480 = 307,200 pieces of pixel data are sequentially input to the LPF processing unit 31 for each image data. That is, at this input time, the number of clusters of pixel data constituting one image data is 307,200.

  In the present embodiment, the pixel data is composed of color feature data and texture feature data. The color feature data is, for example, extracted based on a histogram of image data, and is, for example, three-dimensional data in which each component of RGB is represented by 8 bits (0 to 255). Of course, the color feature data may be expressed based on the color difference signal (Y / Cb / Cr). The texture feature data is extracted as a frequency component of each pixel data, for example, by performing wavelet transform processing on each pixel in the image data. Texture feature data is also normalized to 8-bit three-dimensional data corresponding to the color feature data.

  The threshold values set by the threshold value setting unit 33 are set in the color feature data and texture feature data, respectively. The threshold value of the color feature data is set to about ± 2, for example, and the threshold value of the texture feature data is set to about ± 3, for example.

  In this embodiment, the LPF processing unit 31 to the interpolation processing unit 34 execute initial clustering processing of pixel data, and the K-means processing unit 35 executes this clustering processing. As described above, the number of clusters of pixel data 307,200 at the time of input is reduced to about several tens by the initial clustering process, and is reduced to about 5 to 20 by the present clustering process.

FIG. 4 is a diagram showing the configuration of the ID processing unit 32.
As shown in the figure, the ID processing unit 32 includes a data memory unit 41, a comparison / determination unit 42, a switch 43, an ID memory unit 44, a maximum ID memory unit 45, and an address counter unit 46.

  The data memory unit 41 sequentially inputs and temporarily stores pixel data. When new pixel data is input, the pixel data as a comparison determination target with the pixel data is output to the comparison determination unit 42. To do.

  The comparison determination unit 42 compares the input pixel data with the pixel data stored in the data memory unit 41 based on the threshold set by the threshold setting unit 33, and the input pixel data is It is determined whether the pixel data stored in the data memory unit 41 is within the threshold range, and the comparison determination result is output to the switch 43.

  The switch 43 assigns the same ID as the ID stored in the ID memory unit 44 or a new ID to the pixel data to be compared and determined according to the comparison determination result by the comparison determination unit 42, and the ID memory unit 44 and Output to CPU1.

  The ID memory unit 44 stores the ID of each pixel data given by the comparison determination unit 42 and supplies the maximum ID value of each ID to the maximum ID memory unit 44.

  The maximum ID memory unit 45 stores the maximum ID value, and when the comparison determination unit 42 determines that the input pixel data is outside the threshold range, the maximum ID value is incremented by 1, Output to the switch 43. In this case, the switch 43 gives the new maximum ID value to the input pixel data and outputs it to the ID memory unit 44.

  The address counter unit 46 manages the correspondence between the address of the pixel data storage area stored in the data memory unit 41 and the address of the ID storage area of each pixel data stored in the ID memory unit 44. To do.

  The data memory unit 41 and the ID memory unit 44 may be physically separate memory elements, or one memory element may be divided for each memory.

  Next, the operation of the recording / reproducing apparatus 100 configured as described above will be described.

FIG. 5 is a flowchart showing a schematic flow of the operation of the recording / reproducing apparatus 100 in the present embodiment.
As shown in the figure, the segmentation processing unit 20 of the recording / reproducing apparatus 100 sequentially inputs pixel data constituting image data (step 51), and executes initial clustering processing for each input of the pixel data (step 51). 52). Next, the segmentation processing unit 20 executes the clustering process using K-means (step 53), and extracts a plurality of regions from each image data based on the clustering process result (step 54).

  Then, the video feature detection unit 4 of the recording / playback apparatus 100 detects a motion vector for each of the extracted regions between a plurality of image data constituting one video content, thereby providing a camera feature in the video content. Is detected (step 55).

FIG. 6 is a flowchart showing the flow of the initial clustering process in step 72 described above.
As shown in the figure, when the ID processing unit 32 of the segmentation processing unit 20 receives pixel data from the LPF processing unit 31, it first executes an initialization process (step 61). That is, the ID processing unit 32 sets n = 0 for counting the number of pixel data, id (n) indicating the ID value, and a processed flag flg (n) for determining that the initial clustering process is completed, respectively. , Id (n) = 0 and flg (n) = 0.

  Subsequently, the ID processing unit 32 reads the pixel data d (n) (step 62), and writes and stores the read pixel data in the data memory unit 41 (step 63).

  Subsequently, the ID processing unit 32 compares the pixel data read in step 62 with the pixel data already stored in the data memory unit 41, and the read pixel data is already stored in the pixel data. To determine whether it is within the threshold range (step 64).

Here, details of the comparison determination process and the ID assignment process will be described. FIG. 7 is a flowchart showing a detailed flow of each process.
As shown in the figure, first, the ID processing unit 32 initializes a counter value k of pixel data, which is a comparison target of the read pixel data, already stored in the data memory unit 41 to k = 0. (Step 71). Subsequently, the ID processing unit 32 determines whether or not the processed flag flg (k) of the already stored pixel data d (k) is flg (k) = 1, that is, the pixel data d (k). It is confirmed whether or not the ID assignment process has been completed (step 72).

  If flg (n) = 0 (Yes), that is, if the ID allocation process for the pixel data d (k) has been completed, the ID processing unit 32 stores the pixel data d (k) in the data memory unit. Read from 41 (step 73).

  Subsequently, the ID processing unit 32 determines that the value of the pixel data d (n) read by the comparison / determination unit 42 is within the range of the threshold value dth from the value of the already stored pixel data d (k). Whether or not (step 74). As described above, since the threshold value dth is set for each of the color feature data and the texture feature data of the pixel data d (k), the ID processing unit 32 sets the threshold value dth for each of the color feature data and the texture feature data. judge.

  As a result of the threshold determination, when it is determined that the read pixel data d (n) is within the threshold from the stored pixel data d (k), that is, | d (n) −d (k) | ≦ If it is determined that it is dth (Yes), the ID processing unit 32 assigns the same ID as the pixel data d (k) to the pixel data d (n) (step 75).

  As a result of the threshold determination, when it is determined that the read pixel data d (n) is not within the threshold from the stored pixel data d (k), that is, | d (n) −d (k) |> If it is determined that it is dth (Yes), the ID processing unit 32 increments the counter value k by 1 (step 78). Then, the ID processing unit 32 determines whether or not the incremented k is larger than the pixel data number n, that is, whether or not there are no more IDs to be allocated (step 78).

  When it is determined that k> n, the ID processing unit 32 detects the maximum ID value idmax from the maximum ID memory unit 45 and sets the value obtained by incrementing the maximum ID value idmax by 1 as the pixel data d (n (Step 80).

  If it is determined that k ≦ n, the ID processing unit 32 performs the processing after step 72, that is, the comparison determination processing between the pixel data d (k) incremented by 1 and the pixel data d (n). To do.

  When the ID assignment process is completed, the ID processing unit 32 sets the processed flag flg (n) for the pixel data d (n) to flg (n) = 1 (step 76).

  If flg (k) = 0 in Step 72 (No), the ID processing unit 32 proceeds to Step 77, increments k by 1, and executes the processing after Step 78.

  Returning to FIG. 6, when the ID allocation process is completed, the ID processing unit 32 increments the counter value n of the pixel data d (n) by 1 (step 65), and the incremented n sets the threshold value nth of the number of data. It is determined whether or not it has been exceeded (step 66). If n ≦ nth (No), the ID processing unit 32 returns to step 62 and repeats the subsequent processing. When n> nth, that is, when the assignment of IDs to all the pixel data d (n) in the predetermined image data is completed, the ID processing unit 32 performs the ID interpolation processing by the interpolation processing unit 34. Is executed (step 67), and the process proceeds to the clustering process.

8 and 9 are diagrams conceptually showing the ID assignment process.
As shown in FIG. 8, the ID processing unit 32 assigns each pixel data d an ID based on whether or not the pixel data d is within the range of the threshold value dth from the already stored pixel data. Data d is classified into a plurality of clusters (clusters A to B) (initial clustering). Pixel data belonging to each cluster is regarded as pixel data having the same value by the initial clustering process, that is, for example, pixel data having an average value of a plurality of pixel data belonging to each cluster even though the values are different. It is.

  As shown in FIG. 9, sequentially read pixel data is sequentially compared with the first read pixel data among the pixel data belonging to each cluster previously given an ID.

  That is, in the figure, the pixel data d (2) read second is compared with the pixel data d (1) previously assigned ID = 1, and the pixel data d (2) is compared with the pixel data d. Since it is within the threshold value dth from (1), ID = 1 is assigned to the pixel data d (2).

  The pixel data d (3) read third is compared with the pixel data d (1). Since the pixel data d (3) is not within the threshold value dth from the pixel data d (1), the pixel data d ( 3) is given ID = 2 which is incremented by 1 from ID = 1.

  The pixel data d (4) read fourth is sequentially compared with pixel data d (1) having ID = 1 and d (2) having ID = 2, and the pixel data d (4) Since neither the data d (1) nor the pixel data d (2) is within the threshold dth, the pixel data d (3) is given ID = 3 which is incremented by 1 from ID = 2.

  The pixel data d (5) read fifth is within the threshold value dth from both the pixel data d (1) having ID = 1 and the pixel data d (3) having ID = 3 (with the threshold value dth). However, in this case, ID = 1 previously assigned (with a small ID count value) is assigned to the pixel data d (5). Of course, when the ranges of the threshold value dth overlap, an ID (in this case, ID = 3) assigned later (ID count value is large) may be assigned.

  Next, details of the ID interpolation process in step 67 of FIG. 6 will be described.

FIG. 10 is a flowchart showing a schematic flow of the ID interpolation process. FIG. 11 is a diagram conceptually showing the ID interpolation processing.
As shown in FIG. 10, the ID interpolation process includes an ID replacement process (step 101) in the X direction in the image data and an ID replacement process (step 102) in the Y direction.

  That is, as shown in FIGS. 11A and 11B, the interpolation processing unit 34 has the same ID (ID = n) for the pixel data in the X direction and the Y direction in the image data. The ID of pixel data having different IDs (ID = m) existing between a plurality of pixel data is replaced with ID = n of pixel data existing on both sides thereof. As a result, the noise component in the image data is removed. In other words, the interpolation processing unit 34 hardly changes the data value for each pixel, and the pixel data existing between similar pixel data is also assumed to be similar pixel data. When the data IDs are different, the pixel data is regarded as noise.

  FIG. 12 is a flowchart showing the flow of the ID replacement process in the X direction, and FIG. 13 is a flowchart showing the flow of the ID replacement process in the Y direction.

  In both figures, the interpolation processing unit 34 considers, for example, the upper left end of one piece of image data as the origin, and processes pixel data sequentially from the origin in the X direction (right direction) and the Y direction (down direction). Yes. Further, the maximum coordinates of the pixel data in the X direction and the Y direction are assumed to be xmax and ymax, respectively. When the image data is VGA size, since there are 640 pixel data in the X direction and 480 in the Y direction, xmax = 640 and ymax = 480.

  In the present embodiment, attention is paid to five consecutive pixel data in the X direction and the Y direction, for example. These five pixel data are d (m), d (m + 1), d (m + 2), d (m + 3), and d (m + 4) (m ≧ 0), and the ID of each pixel data is ID (m), ID ( If m + 1), ID (m + 2), ID (m + 3), and ID (m + 4), should the interpolation processing unit 34 replace ID (m + 2) of the pixel data d (m + 2) sandwiched between the two data before and after? Judge whether or not. Details of the processing will be described below.

  As shown in FIG. 12, first, the interpolation processing unit 34 initializes the coordinate counter value of pixel data in the Y direction in one image data to y = 0 (step 121). Subsequently, the interpolation processing unit 34 initializes the coordinate counter value of the pixel data in the X direction in the image data to x = 4.

  Subsequently, the interpolation processing unit 34 determines whether or not the five pixel data satisfy the replacement condition in the X direction (step 123). The substitution condition in the X direction is expressed by the following formula.

ID ((x−4) + y · xmax) = ID ((x−3) + y · xmax) =
ID ((x-1) + y · xmax) = ID (x + y · xmax)
And ID ((x−2) + y · xmax) ≠ ID (x + y · xmax)

  That is, among the five pieces of pixel data, the first, second, fourth, and fifth pixel data have the same ID and are different from the third pixel data ID. It is determined whether or not it has been done.

  If it is determined that the replacement condition is satisfied, the interpolation processing unit 34 replaces the ID of the third pixel data with the IDs of the other four pixel data (step 124). When it is determined that the replacement condition is not satisfied, the interpolation processing unit 34 does not execute the ID replacement process.

  Thereafter, the interpolation processing unit 34 increments the X coordinate counter value by 1 (step 125), determines whether or not the incremented X coordinate is xmax (step 126), and determines that the X coordinate is not xmax. If it is determined (No), the X coordinate is moved to the right until the X coordinate reaches xmax, and the processing in steps 123 and 124 is repeated.

  If it is determined that the X coordinate is xmax (Yes), the counter value of the Y coordinate is incremented by 1 (step 127), and it is determined whether or not the incremented Y coordinate is ymax (step 128). ), When it is determined that the Y coordinate is not ymax (No), the Y coordinate is moved downward until the Y coordinate becomes ymax, and the processing of step 122 to step 128 is repeated.

  As described above, the ID replacement process for all the pixel data constituting the image data in the X direction is completed.

  As shown in FIG. 13, the replacement process for the Y direction can be executed in the same manner as the ID replacement process shown in FIG. The substitution condition in the Y direction is expressed by the following formula.

ID (x + (y−4) · xmax) = ID (x + (y−3) · xmax) =
ID (x + (y-1) .xmax) = ID (x + y.xmax)
And ID (x + (y−2) · xmax) ≠ ID (x + y · xmax)

  The noise components in the image data are removed by the above replacement processing in the X direction and the Y direction. As a result, the number of clusters generated by the initial clustering is reduced, and the subsequent clustering process can be speeded up.

  Note that, depending on the image data, pixel data that is noise may be included continuously, or may be included intermittently, for example, every other pixel data. In such a case, as described above, noise may not be removed if the ID replacement process is executed only for one piece of pixel data sandwiched between two pieces of pixel data on both sides as described above. However, in the present embodiment, since the LPF processing unit 31 removes noise before the ID processing by the ID processing unit 32, it is possible to remove such noise. Of course, the interpolation processing unit 34 may execute the ID replacement process using the replacement conditions as in step 123 of FIG. 12 and step 133 of FIG. 13 in order to remove the continuous or intermittent noise. Absent.

FIG. 14 is a graph showing the relationship between the number of clusters generated by the initial clustering process and the threshold value.
As shown in the figure, the initial cluster number and the threshold value are in an inversely proportional relationship. If the threshold is too large, the classification accuracy of the pixel data will be low, and if the threshold is too small, the convergence time of the subsequent clustering process will be long. Therefore, the permissible range (n1 to n2) of the initial number of clusters depends on the range of the target cluster number (n0 to n1) of the main clustering process by the K-means (for example, 5 to 20 clusters). A range (for example, 20 to 50 clusters) that does not cause a burden and can obtain the effect of classification is set, and a threshold range (Th1 to Th2) is also set accordingly.

  The threshold value setting unit 33 repeats the initial clustering process by changing the threshold value until the number of initial clusters falls within the allowable range (n1 to n2) according to the result of the initial clustering process without fixing the threshold value in advance. Thus, the threshold setting allowable range may be learned.

  As described above, the pixel data classified into the first number of clusters (reduced number of clusters) by the initial clustering process (ID assignment process) is subjected to the ID replacement process and the K-means processing unit 35. In this clustering process, the second number (target cluster number) of clusters is smaller than the first number.

  Then, the segmentation processing unit 20 extracts the second number of areas from the image data for each cluster classified by the main clustering process. That is, the segmentation processing unit 20 divides the image data into a second number of regions having an arbitrary shape. The segmentation processing unit 20 executes this segmentation processing for all the image data constituting each video content, and stores the result in the HDD 10 or the flash memory.

FIG. 15 is a conceptual diagram showing a comparison between the two-stage clustering process in the present embodiment described above and the conventional clustering process. FIGS. 9A to 9C conceptually show the distribution of feature vector data in a multi-dimensional (six-dimensional) feature vector space using the pixel data (feature data) as feature vector data.
In the conventional clustering processing, clustering processing is performed on the I (for example, 307,200) initial pixel data in FIG. 6A by the method such as K-means as shown in FIG. The number is reduced to K2 (for example, 5 to 20) clusters. However, if the initial data is directly processed by a technique such as K-means in this way, a long time is required until the number of clusters converges to the above K2.

  Therefore, in the present embodiment, as shown in FIG. 5B, the initial clustering process using ID assignment first classifies the cluster into K1 (for example, 20 to 50) clusters, and then performs the main clustering using K-means. By processing, it is classified into K1 clusters shown in FIG.

  As described above, by reducing the number of clusters by the initial clustering process, the time until the pixel data is classified into the target number of clusters can be greatly shortened, and the load on the recording / reproducing apparatus 100 can be reduced.

  Next, the camera feature detection process in step 55 of FIG. 5 using the image data segmented as described above will be described.

FIG. 16 is a diagram schematically showing the relationship between camera operations between image data and motion vectors.
As shown in FIGS. 6A and 6B, a region A of a house that is a stationary object is obtained from the image F1 at time t and the image F2 at time t2 in one video data by the image segmentation process. Then, a case is assumed in which three regions of a vehicle region B that is a moving object and a region C that is a background are extracted.

  For example, when a panning operation is performed in the right direction from the image F1 to the image F2, the panning operation is indicated by the motion vector of the region B that is a moving object, as shown in FIG. The motion vectors of the regions A and C, which are still objects, are not. In general, the background area has a larger number of pixels (area) than the area of the moving object.

  Therefore, the video feature detection unit 4 detects the motion vector of the extraction region having the maximum number of pixels among the extraction regions as a representative motion vector between the image data, and determines the camera feature based on the motion vector. To do.

FIG. 17 is a flowchart showing the flow of camera feature detection processing by the video feature detection unit 4.
As shown in the figure, first, the video feature detection unit 4 inputs a plurality of image data in one video content for each extraction region extracted by the segmentation process (step 171). Subsequently, the video feature detection unit 4 calculates the number of pixels of the plurality of extraction regions for each image data, and selects the extraction region having the maximum number of pixels (step 172). In FIG. 16, since the number of pixels in the areas A to C is C> A≈B, the area C is selected.

  Subsequently, the video feature detection unit 4 performs block matching processing between each selected extraction region of the plurality of image data, and detects a motion vector (step 173). By this processing, only the motion vector V3 is detected in the case of FIG.

  Subsequently, the video feature detection unit 4 performs multiple regression analysis processing based on the detected motion vector data (step 174), and calculates camera feature coefficients (affine coefficients) such as pan, tilt, and zoom (step 174). 175).

  Then, the video feature detection unit 4 determines the camera feature between the image data based on the camera feature coefficient of the selected extraction region, and outputs the result (step 176). In the case of FIG. 16, it is determined that panning has been performed based on the calculated pan coefficient.

FIG. 18 is a flowchart showing details of the motion vector detection process in step 173.
As shown in the figure, the video feature detection unit 4 includes an extraction region selected in one image data (hereinafter referred to as a reference frame), an interval of 1 frame, an interval of 10 frames, an interval of 20 frames, and the like. Block matching processing is executed for each extracted region selected in each frame at intervals of 30 frames, and motion vector data is detected (steps 181 to 184). For example, frames at intervals of each frame are held in a frame memory (not shown) for each frame interval, and are read from the frame memory each time a block matching process is performed with the reference frame.

  Subsequently, the video feature detection unit 4 estimates motion vector data in a frame (hereinafter referred to as a search frame) at a predetermined frame interval (for example, 40 frame interval) based on the motion vector data detected for each frame interval. (Step 185), the estimated motion vector data is output as final motion vector data (step 186). This estimation process can be realized, for example, by calculating the gradient of motion vector data at each frame interval and multiplying the average value of each gradient by the frame interval to be estimated (for example, 40).

  Then, the video feature detection unit 4 determines whether or not motion vector data has been output for each selected extraction region of all the frames constituting one video content, and until there is no frame for which a motion vector should be detected. The above steps are repeated (step 187).

  Next, an affine transformation model for calculating affine coefficients by the multiple regression analysis process in step 174 of FIG. 17 will be described.

  FIG. 19 is a diagram showing an affine transformation model. The affine transformation model is a model for describing translation, enlargement / reduction, and rotation of a three-dimensional object as coordinate transformation processing using a matrix. Since the camera features such as pan, tilt, and zoom are considered to be parallel movement and enlargement / reduction of an object in the reference frame, it is possible to describe the camera features by using an affine transformation model.

Here, in the video content, when the frame interval is not large, the following approximation process can be performed on the assumption that the rotation angle θ is small for the rotation feature.
sinθ ≒ θ
cos θ ≒ 1

  Therefore, the affine transformation model can be modified as shown in FIG. And a camera characteristic can be detected by calculating | requiring each coefficient from this detected motion vector using this affine transformation model. That is, for each camera feature of pan, tilt, and zoom, predetermined threshold values Pth, Tth, and Zth are set, and each camera feature is detected by comparing with each affine coefficient processed from the detected motion vector. can do.

  FIG. 20 is a diagram showing processing for obtaining an affine coefficient by multiple regression analysis. As shown in the figure, the video feature detection unit 4 sets the explanatory variables as x and y coordinates (xn, yn) of the detection target point (Pn) in the extraction region of the reference frame, and sets the non-explanatory variables (object variables). Then, multiple regression analysis processing is performed as the x and y coordinates (xm, ym) of the motion vector detection position (Pm) in the search frame extraction region to obtain pan, tilt, and zoom coefficients Px, Py, and Zx. .

  The video feature detection unit 4 compares the coefficients Px, Py, and Zx with the threshold values Pth, Tth, and Zth, and detects each camera feature when each coefficient is greater than each threshold value. The detection is output.

  Note that the video feature detection unit 4 may detect the pan, tilt, and zoom cameras separately for left pan / right pan, left tilt / right tilt, and zoom in / zoom out, respectively. This distinction can be easily made by referring to the sign of the affine coefficient.

  As described above, by detecting a motion vector between image data based on an extraction region having the maximum number of pixels (area) among a plurality of regions extracted by image segmentation, it is influenced by the movement of the moving object. Therefore, the motion vector can be detected with high accuracy, and more accurate camera features can be detected.

  The recording / reproducing apparatus 100 performs processing such as digest reproduction (highlight scene reproduction) based on the detected camera characteristics. In other words, the recording / reproducing apparatus 100 regards a frame in which each camera feature is detected in the video content as a scene in which the photographer is paying attention, extracts such a frame from the video content, and sends the digest video to the user. Can be provided.

[Second Embodiment]
Next, a second embodiment of the present invention will be described.

  FIG. 21 is a diagram showing a configuration of the recording / reproducing apparatus 200 according to the present embodiment. In the present embodiment, the recording / reproducing apparatus 200 can execute the image search process by determining the similarity between a plurality of image data for each region extracted by the image segmentation process.

  As shown in the figure, the recording / reproducing apparatus 200 includes an image search unit 30 instead of the video feature detection unit 4 of the recording / reproducing apparatus 100 shown in FIG. 1 of the first embodiment. Since other parts are the same as those of the recording / reproducing apparatus 100 in the first embodiment, the description thereof is omitted.

FIG. 22 is a flowchart showing a flow of operations of the recording / reproducing apparatus 200 according to the present embodiment.
As shown in the figure, in the recording / reproducing apparatus 200, the segmentation processing unit 20 performs initial clustering processing, main clustering processing, and region extraction processing on each pixel data of a plurality of input image data (step 221). ~ 224). Since these processes are the same as those in the first embodiment, description thereof is omitted. Subsequently, the recording / reproducing apparatus 200 performs an image search process using the image search unit 30 (step 225).

FIG. 23 is a flowchart showing a detailed flow of the image search process. FIG. 24 is a diagram conceptually showing the image search process.
As shown in both figures, first, the image search unit 30 calculates the number of pixels in the extraction region extracted by the segmentation process, and selects the extraction region having the maximum number of pixels (step 231).

  Subsequently, the image search unit 30 generates each feature vector of the selected extraction region (step 232). That is, as shown in FIG. 24, the image search unit 30 3D color feature data (cn = (dnr, dng, dnb)) of pixel data constituting each image data and 3D texture feature data ( Based on tn = (dnl, dnlh, dnhl)), 6-dimensional feature vector data Vn = (dnll, dnlh, dnhl, dnll, dnlh, dnhl) are created.

  Subsequently, the image search unit 30 uses the feature vector data of one image data (reference image) of the generated feature vector data as a search key, and the feature vector data of the reference image and other image data (search The similarity between the reference image and the search image is determined by executing the inter-vector distance calculation with the feature vector data of the image) (step 233).

  Then, the image search unit 30 outputs the determination result as an image search result (step 234). For example, the image search unit 30 sets a predetermined threshold for the above-described vector distance, and if the inter-vector distance between the feature vector of the reference image and the feature vector of the search image is within the threshold, the search image is similar Output as an image.

  As described above, in the present embodiment, the speed of the feature vector generation process is increased by executing the image search process using the feature vector data of the region extracted by the accelerated image segmentation process. As a result, the image search process can be speeded up.

[Third Embodiment]
Next, a third embodiment of the present invention will be described.

  In the present embodiment, the recording / reproducing apparatus applies the region extraction result by the image segmentation process to the object encoding process in the video data encoding process.

FIG. 25 is a diagram showing a configuration of the recording / reproducing apparatus 300 according to the present embodiment.
As shown in the figure, the recording / reproducing apparatus 200 is different from the recording / reproducing apparatuses 100 and 200 shown in the first and second embodiments in that the video feature detection unit 4 and the image search unit 40 are eliminated. Yes. Further, an AV codec 251 capable of encoding and decoding video data in the MPEG format is provided in place of the AV decoder 16 of the recording / playback apparatuses 100 and 200. The AV codec 251 is responsible for the object encoding process. Since other parts are the same as those of the recording / reproducing apparatus 100 in the first embodiment, the description thereof is omitted.

FIG. 26 is a flowchart showing the flow of object encoding processing in the present embodiment.
As shown in the figure, in the recording / reproducing apparatus 100, the segmentation processing unit 20 performs initial clustering processing, main clustering processing, and region extraction processing on each pixel data of a plurality of input image data (step 261). ~ 264). Since these processes are the same as those in the first and second embodiments, description thereof will be omitted. Subsequently, the recording / reproducing apparatus 100 executes an encoding process of the video data by the AV codec 251 (step 265).

FIG. 27 is a flowchart showing the flow of the encoding process in step 265.
As shown in the figure, the AV codec 251 inputs video data for each area extracted by the image segmentation process (step 271).

  Subsequently, the AV codec 251 performs motion prediction for each extraction region (step 272). Specifically, the AV codec 251 detects a motion vector for each extraction region from each image data (frame) of the input video data and prediction reference image data stored in the frame memory. . Next, the AV codec 251 performs motion compensation on the prediction reference image data using the motion vector, and generates a predicted image. Further, the AV codec 251 calculates the difference between the input image and the predicted image for each extraction region.

  Subsequently, the AV codec 251 performs DCT transformation (discrete cosine transformation) processing on the difference data for each extraction region (step 273), and further quantizes the difference data after the DCT transformation (step 274).

  Subsequently, the AV codec 251 assigns a two-dimensional Huffman code by variable length coding (VLC) to the difference data for each extraction region after quantization, and also adds 2 to the motion vector data detected for each extraction region. A dimensional Huffman code is assigned (step 275).

  Then, the AV codec 251 multiplexes the difference data and motion vector data to which the Huffman code is assigned, and outputs it as encoded data (step 276).

  As described above, in the present embodiment, the speed of the video encoding process can be increased by using the region extracted by the accelerated image segmentation process for the object encoding process.

  The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the scope of the present invention.

  In the first embodiment, the video feature detection unit 4 detects only the motion vector of the extraction region having the maximum number of pixels (area). However, the video feature detection unit 4 may detect the motion vectors of the plurality of extraction regions, and may detect the motion vector detected most frequently as a motion vector between the entire image data.

  In addition, the video feature detection unit 4 detects motion vectors of a plurality of extraction regions, assigns a weight according to the number of pixels (area) of the extraction region to each motion vector, and determines a motion vector as the entire image data. It may be detected. In other words, the video feature detection unit 4 weights the motion vector data of each extraction region using an evaluation function that can be weighted higher as the number of pixels in the extraction region is larger, thereby evaluating the motion vector of the entire image data. The value may be calculated. As a result, more accurate motion vector detection reflecting the number of pixels in all extraction regions is possible.

  Further, the video feature detection unit 4 may use a motion vector between extraction regions having the smallest number of pixels (area) among a plurality of extraction regions as a motion vector of the entire image data. As a result, motion vector detection focusing on the moving object can be performed.

  In the first embodiment, the video feature detection unit 4 detects camera features by the image segmentation process and the motion vector detection process. However, the video feature detection unit 4 can also apply this image segmentation process and motion vector detection process to, for example, a process of detecting a moving object.

  That is, the video feature detection unit 4 includes the moving object by determining whether the object in the image data is moving or stationary by the image segmentation process and the motion vector detection process for each extraction region. Image data (frame) can be extracted.

  For example, the video feature detection unit 4 can extract a frame in which a region occupies a large proportion and includes a moving object, and can perform digest playback of the frame section. This is because a large area can be regarded as an area that the photographer is paying attention to by zooming or the like, and a moving area can be regarded as an active area.

  In the second embodiment, the image search unit 30 generates feature vector data from pixel data (feature data) of only an extraction region having the maximum number of pixels (area). However, the image search unit 30 generates feature vector data from the pixel data of a plurality of extraction regions, and assigns a weight according to the number of pixels (area) of the extraction region to the inter-vector distance calculation result of each feature vector. Then, the similarity between both images may be determined. That is, the image search unit 30 performs an inter-vector distance calculation for each extraction region between the reference image and the search image, and calculates an evaluation function that can be weighted higher as the number of pixels in the extraction region is larger. The evaluation value of similarity may be calculated by weighting using them.

  In the first to third embodiments, the segmentation processing unit 20 extracts color features and texture features from the baseband signal decoded by the AV decoder 16 (or AV codec 251). However, the segmentation processing unit 20 may extract color features and texture features from the image data before decoding. In this case, the segmentation processing unit 20 can extract each feature based on the DC coefficient of the DCT coefficient.

  In the first to third embodiments, the segmentation processing unit 20 executes the two-stage clustering process of the initial clustering process by ID assignment and the main clustering process by K-means. However, the segmentation processing unit 20 may abandon the K-means processing unit 35 and terminate the image segmentation process only with the initial clustering process without performing the main clustering process. Further, even when the segmentation processing unit 20 has the K-means processing unit 35, if the pixel data can be classified into clusters of the target number of clusters by the initial clustering process, the present clustering process is not performed, but at that time. The image segmentation process may be terminated.

  In the first and second embodiments, the recording / reproducing apparatus uses K-means as the clustering method of the clustering process. However, as the clustering method, other methods such as fuzzy c-means, entropy method, Ward method, SOM, etc. may be used.

  In the first to third embodiments, an example in which the present invention is applied to a recording / reproducing apparatus has been described. However, the present invention can be similarly applied to other various electronic devices such as a PC, a server device, a television device, a game device, a digital camera, a digital video camera, and a mobile phone.

It is the figure which showed the structure of the recording / reproducing apparatus which concerns on the 1st Embodiment of this invention. It is the figure which showed the outline | summary of the segmentation process. It is the figure which showed the structure of the segmentation process part in the 1st Embodiment of this invention. It is the figure which showed the structure of the ID process part in the 1st Embodiment of this invention. 3 is a flowchart showing a schematic flow of the operation of the recording / reproducing apparatus in the first embodiment of the present invention. It is the flowchart which showed the flow of the initial clustering process in the 1st Embodiment of this invention. It is the flowchart which showed the detailed flow of the comparison determination process between pixel data and ID allocation process among the initial clustering processes in the 1st Embodiment of this invention. It is the figure which showed notionally ID allocation processing in the 1st Embodiment of this invention. It is the figure which showed notionally ID allocation processing in the 1st Embodiment of this invention. It is the flowchart which showed the schematic flow of the ID interpolation process in the 1st Embodiment of this invention. It is the figure which showed notionally ID interpolation processing in the 1st Embodiment of this invention. It is the flowchart which showed the flow of the ID replacement process about the X direction in the 1st Embodiment of this invention. It is the flowchart which showed the flow of ID replacement processing about the Y direction in the 1st Embodiment of this invention. It is the graph which showed the relationship between the number of clusters produced | generated by the initial clustering process in the 1st Embodiment of this invention, and a threshold value. It is the conceptual diagram which showed the two-stage clustering process in the 1st Embodiment of this invention, and compared with the conventional clustering process. It is the figure which showed typically the relationship between the camera operation | movement between image data and a motion vector. It is the flowchart which showed the flow of the camera feature detection process by the image | video feature detection part in the 1st Embodiment of this invention. It is the flowchart which showed the detail of the motion vector detection process in the 1st Embodiment of this invention. It is the figure which showed the affine transformation model. It is the figure which showed the process which calculates | requires an affine coefficient by multiple regression analysis. It is the figure which showed the structure of the recording / reproducing apparatus which concerns on the 2nd Embodiment of this invention. 6 is a flowchart showing a flow of operations of a recording / reproducing apparatus according to a second embodiment of the present invention. It is the flowchart which showed the detailed flow of the image search process in the 2nd Embodiment of this invention. It is the figure which showed notionally the image search process in the 2nd Embodiment of this invention. It is the figure which showed the structure of the recording / reproducing apparatus which concerns on the 3rd Embodiment of this invention. It is the flowchart which showed the flow of the object encoding process in the 3rd Embodiment of this invention. It is the flowchart which showed the flow of the encoding process in the 3rd Embodiment of this invention.

Explanation of symbols

1 ... CPU
4 ... Video feature detection unit 10 ... HDD
DESCRIPTION OF SYMBOLS 20 ... Segmentation process part 30 ... Image search part 31 ... LPF process part 32 ... ID process part 33 ... Threshold setting part 34 ... Interpolation process part 35 ... K-means process part 40 ... Image search part 41 ... Data memory part 42 ... Comparison Determination unit 44 ... ID memory unit 45 ... Maximum ID value memory unit 46 ... Address counter unit 100, 200, 300 ... Recording / reproducing device 251 ... AV codec

Claims (14)

A plurality of pixel data constituting image data are sequentially input,
It is determined for each input whether a predetermined feature value of each input pixel data is within a predetermined range value from the feature value of already input pixel data, and the feature value is the predetermined range value. The first identification information that is the same as the already input pixel data is assigned to the pixel data that is determined to be within, and the pixel data that is determined not to be within the predetermined range value is added to the first pixel data. An information processing method for classifying the plurality of pieces of pixel data into a first number of clusters by assigning second identification information different from the identification information. The information processing method according to claim 1, further comprising:
The pixel data classified into the same cluster by the classification is regarded as pixel data having the same feature value, and the second plurality of classified pixel data is less than the first number by a predetermined clustering method. An information processing method that classifies a number of clusters. The information processing method according to claim 2, further comprising:
Among a plurality of pieces of pixel data classified into the first number of clusters, a plurality of continuous first pixel data existing in a predetermined direction and the first pixel data existing in the predetermined direction are A plurality of different continuous second pixel data and at least one third pixel data existing between the first pixel data and the second pixel data in the predetermined direction;
When the identification information of the first pixel data and the second pixel data is the same, and the identification information of the third pixel data is different from the first and second pixel data, the first pixel data and the second pixel data are different from each other. 3. An information processing method for replacing the identification information of the third pixel data with the identification information of the first and second pixel data. The information processing method according to claim 3, further comprising:
An information processing method for removing high-frequency components in the input pixel data by a low-pass filter. The information processing method according to claim 3, further comprising:
An information processing method for dividing the image data into the second number of regions having an arbitrary shape based on a plurality of pixel data classified into the second number of clusters. The information processing method according to claim 5, further comprising:
Detecting a motion vector between a plurality of the image data for each of the divided second number of regions;
An information processing method for detecting a predetermined video feature generated by a camera operation in video data composed of a plurality of the image data based on the detected motion vector. An information processing method according to claim 6,
Detecting the video feature comprises:
Calculating the number of pixels of the second number of regions of the plurality of image data,
An information processing method for detecting the predetermined video feature in the video data based on a motion vector detected between regions having the largest number of pixels in the plurality of image data. The information processing method according to claim 5, further comprising:
An information processing method for encoding the pixel data for each of the divided second number of regions. The information processing method according to claim 3, further comprising:
Generating a feature vector for each of the divided second number of regions;
An information processing method that compares the generated feature vectors for each of the second number of clusters among a plurality of the image data to determine similarity between the plurality of image data. An information processing method according to claim 9,
The step of determining the similarity includes:
Calculating the number of pixels of the second number of regions of the plurality of image data,
An information processing method for determining similarity between the plurality of image data by comparing the feature vectors between regions having the largest number of pixels among the plurality of pixel data. An information processing method according to claim 3,
The step of classifying into the first number of clusters repeats the classification by changing the predetermined range value until the first number reaches a predetermined number. An information processing method according to claim 3,
The step of classifying into the first number of clusters performs control so as not to execute the classification into the second number of clusters when the first number reaches a predetermined number. Input means for sequentially inputting a plurality of pixel data constituting image data;
It is determined for each input whether the feature value of each input pixel data is within a predetermined range value from the feature value of the already input pixel data, and the feature value is within the predetermined range value. The first identification information that is the same as the already input pixel data is assigned to the pixel data determined to be, and the first identification information is determined to be the pixel data determined that the feature value is not within the predetermined range value. First classification means for classifying the plurality of pieces of pixel data into a first number of clusters by giving second identification information different from
The pixel data classified into the same cluster is regarded as pixel data having the same feature value, and the classified pixel data is classified into a second number of clusters smaller than the first number by a predetermined clustering method. An information processing apparatus comprising: a second classification unit. In the information processing device,
Sequentially inputting a plurality of pixel data constituting the image data;
It is determined for each input whether a predetermined feature value of each input pixel data is within a predetermined range value from the feature value of already input pixel data, and the feature value is the predetermined range value. The first identification information that is the same as the already input pixel data is assigned to the pixel data that is determined to be within, and the pixel data that is determined not to be within the predetermined range value is added to the first pixel data. Classifying the plurality of pixel data into a first number of clusters, giving second identification information different from the identification information of
The pixel data classified into the same cluster is regarded as pixel data having the same feature value, and the classified pixel data is classified into a second number of clusters smaller than the first number by a predetermined clustering method. A program for executing the steps to be executed.

Images