JPH05165896A - Picture data retrieval device and picture data retrieval method - Google Patents

Abstract

PURPOSE:To improve the retrieval efficiency by storing hierarchical encoded picture data according to hierarchy, selecting and displaying picture data which is similar to a prescribed criterion by reading the lowest hierarchical data at the time of a retrieval. CONSTITUTION:A picture signal to be obtained from an input terminal 1 is encoded by a hierarchical encoder 2 and is stored at the prescribed position on a disk 8 according to hierarchy through a disk input/output control unit 3. When picture data is retrieved, the frame number of a picture to be a retrieval object is successively outputted from a retrieval unit 4 to the disk input/ output control unit 3. The disk input/output control unit 3 reads the picture data having the pertinent number of layers of a designated picture data from the disk 8 and outputs the data to the degree of similarity decision unit 5. At this place, the lowest hierarchy picture data having a small amount of information is read from the disk 8, each picture data is compared with the prescribed criterion of degree of similarity of the degree of similarity decision unit 5, and picture data which is similar to this criterion is selected and displayed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image data retrieving apparatus and an image data retrieving method for selecting desired image data from a large amount of accumulated image data.

[0002]

2. Description of the Related Art Conventionally, as a method of this kind, for example, a research report of Information Processing Society of Japan, multimedia communication and distributed processing 42
-5 are listed. Here, hierarchically,
Image data base that stores encoded image data
The system is disclosed.

[0003]

In the above prior art, the encoded image data after retrieval is transmitted to reduce the transmission time. Further, the efficiency of browsing is improved by decoding only the lower order code of the hierarchically encoded data. However, when a large amount of processing is required, the operator must see the images reproduced from all the lower codes, and the search efficiency itself is not efficient.

An object of the present invention is to use a lower code of image data which is hierarchically coded, give an arbitrary condition to the lower code, and automatically search for a desired image. To provide a search method.

[0005]

In order to achieve the above object, in the present invention, layer-encoded image data is stored for each layer. At the time of retrieval, of the above-mentioned coded data, first, only the data of the lower hierarchy is read, this is compared with a predetermined criterion, and only the image data that satisfies this criterion is read again.

[0006]

In general, the hierarchically encoded data of the lower layer expresses the original data extremely roughly, and by adding the upper layer to this, it becomes gradually detailed. For example, the data hierarchically coded by the frequency component is represented by only the low frequency component in the lower layer, and more sensitive edge components and the like appear as the upper layer is added.

The amount of data in the lower layers is a fraction of the amount of data in all the layers, and the time required for the search process also decreases in proportion to this. Utilizing this fact, a rough search is first performed at high speed using only the data of the lower hierarchy. As a result, the data of the lower hierarchy of some of the retrieved candidate groups is decoded, displayed on the user, and the candidates are narrowed down interactively. At this time,
The time required for the transfer and decoding of the lower layer data is proportional to the amount of the data, and is a fraction of that in the case of using all the layers.

Next, a search process is performed on the remaining candidate group also by using a higher layer code to narrow down the candidates, present decoded images again, and narrow down the candidates. By repeating the above procedure, the target image can be retrieved at high speed and interactively.

[0009]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 shows a first embodiment for retrieving image data according to the present invention. The image signal obtained from the input terminal 1 is first encoded by the hierarchical encoder 2 and stored in a predetermined position on the disk 8 through the disk input / output control unit 3. For hierarchical coding, for example, Communication of t
It is described in detail in he ACM, Vol. 34, No. 4 (1991.4). The image data is stored in the disk 8 in layers. For example, when layering with frequency components, if the block size is 8 × 8 and the number of layers is 16, conversion coefficients 1 to 4 are assigned to the lowest layer, 5 to 8 are assigned to layer 2, and thereafter each layer is assigned. Each of the four coefficients is stored. By performing the above process for a plurality of types of image sequences, a hierarchically encoded image database is constructed on the disc.

Next, a method of retrieving image data required by the user from the above-mentioned image database will be described.
The frame number of the image to be searched is sequentially output from the search unit 4 to the disk input / output control unit 3.
The disk input / output control unit 3 reads the image data of the corresponding number of layers from the specified image data from the disk and outputs it to the similarity determination unit 5. The image data is compared with a predetermined criterion to determine how close it is to the target image.

FIG. 2 shows the configuration of the similarity determination unit 5. Here, it is assumed that image data of a predetermined frame is stored in advance in the image data memory 23 by the disk input / output control unit 3. Data with the outputs of the counters 20, 21 and 22 as addresses are sequentially read from this memory. The horizontal counter 21 counts the clock signal Φ ₁ generated for each pixel, and the output thereof indicates the horizontal address within the block of the data read from the image data memory. On the other hand, the vertical counter 22 counts the carry of the horizontal counter 21, and its output is the image data.
Indicates the vertical address within a block of data read from the data memory. The base numbers of the counters 21 and 22 match the block size. Therefore, if the block size is 8 × 8, each of the counters 21 and 22 is octal. The block counter 20 counts the carry of the vertical counter 22, and the output thereof indicates the block number of the data read from the image data memory. The radix of this counter corresponds to the number of blocks constituting one frame.

The image data designated by the counter and read from the image data memory is input to the accumulator 26. The accumulator 26 is cleared at the beginning of the frame in synchronization with the clock signal Φ ₂ . The input data is added to the contents of the accumulator 26 in synchronization with the addition sync signal 212. The addition sync signal is generated when the horizontal address and the vertical address are equal to or more than a certain value. Comparator 2
The output of 5 is 1 when the horizontal counter value is greater than or equal to the threshold value THh,
When it is less than this, it becomes 0. Similarly, the output of the comparator 24 is 1 when the vertical counter value is equal to or larger than the threshold value THv, and is 0 when the vertical counter value is smaller than the threshold value THv. The outputs of the comparators 24 and 25 are input to the logical product circuit 27, which becomes 1 only when both are 1, and the clock signal 211 is passed from the logical product circuit 28. As a result, the sum of the high frequency components of the orthogonal transform coefficients is calculated for each block in the frame. FIG. 3 shows this example.
The calculated value is compared with the threshold value THf by the comparator 29.
The result is output to the search unit 4.

FIG. 4 shows the structure of the search unit. Frame
The frame counter 40 stores a frame number 43 for starting a search and a frame number 44 for the final search. The counter outputs the frame number 42 to be searched to the disk input / output control unit 3 in synchronization with the clock signal Φ ₂ 210. As a result of the similarity determination, the frame number 9 having the similarity 7 of 1 is output through the gate 41. The above processing is performed for all the frames to be searched, and the search end flag 11 is used to notify the disk input / output control unit 3 of the end of the search.

The above process is interactively performed using the upper hierarchical data to narrow down the candidate images. This procedure is shown in FIG. First, the user specifies the frame that is the search range,
Select search criteria. For example, as described above, the high frequency component amount is selected as the search criterion. Start only with the lowest number of layers. The subsequent processing is repeated until the user is satisfied.

The search process is repeated for all the frames within the search range. Thereafter, the lowest layer encoded data belonging to the retrieved frame is transmitted to the user terminal, decoded and displayed. The user sees this and responds whether or not to continue the search. If the search is to be continued, the search range is designated again, the upper one layer is designated, and the search is repeated again.

In the above example, the case of interactively retrieving a frame having many high frequency components has been described. This allows
An image with many fine patterns, for example, an image containing many checker patterns can be detected interactively. Since only low-layer data is used in the initial stage where there are many detection targets, the search process can be performed in a short period of time, and only low-layer data is transmitted and displayed to the user terminal. It can be done quickly and economically. After the search range is narrowed down to some extent, a detailed search is performed using the upper hierarchy that requires a long processing time, so that the processing time can be suppressed as a whole, which is efficient. In the above embodiment, the description has been limited to the case of searching for a frame having many high frequency components, but the present invention can be applied to searching for a frame having a frequency component of a certain area. For example, when comparing the outputs from the vertical / horizontal counters 21 and 22 with a predetermined threshold value, the lower limit (TH _v ,
It is possible to search for image data by adding the condition of the upper limit in addition to the condition of TH _h ) and taking the AND of them. Specific values of these conditions can be determined in advance by measuring the frequency components in the vertical / horizontal directions from the image serving as a reference for analogy determination, and considering the allowable amount of the values. By doing so, it is possible to quickly search only image data similar to a specific image from a huge amount of image data.

Further, in the above embodiment, it is also possible to decode the hierarchically encoded data and perform the search in order from the lower hierarchy. FIG. 6 shows this configuration. The difference from the first embodiment in this figure is that the image data is decoded before being input to the similarity determination unit. The rest of the configuration is the same as that of the first embodiment, so it is omitted here.

FIG. 7 shows the configuration of a similarity determination unit for searching an image having a large amount of red color components in the decoded data. Here, the disk input / output control unit 3 previously decodes image data of a predetermined frame into the R component memory 71, the G component memory 72, and the B component memory 73, and stores the image data for each color component. Be present. The data output from the counter 74 is used as an address from this memory.
Data is read in order. The image data read from each memory is input to the comparators 75, 76 and 77, and the threshold value TH
r, THg, THb. 1 is output for components exceeding each threshold value. This output is gate 7
It is input to 8, and outputs 1 when only the R component exceeds the threshold value, and outputs 0 otherwise. That is, this output determines that the corresponding pixel exhibits "redness" above a certain level. The output is connected to the count control input of the counter 79, and only when this is 1, the clock signal Φ
Count ₁ 211. By repeating the above process for one frame, the amount of red pixels occupied in the target frame is calculated. If this exceeds the threshold THf, the output of the comparator 29 is 1, and otherwise 0.
This output is output to the search unit. The subsequent processing is the same as in the first embodiment. As a result, a frame having a large proportion of red pixels is searched. The above process is repeated in the same manner as in the procedure of FIG. 5 until the number of layers is increased and the user is satisfied.

In the above example, the case of interactively searching for a frame having a large amount of red component has been described. For example, it is conceivable to search for a clothing catalog or the like and interactively detect a red-based product. Similar to the first embodiment, since only low-layer data is used in the initial stage where there are many detection targets, search processing can be performed in a short period of time, and only low-layer data is transmitted to the user terminal. , Because it is displayed, this is also fast,
And can be done economically. After the search range is narrowed down to some extent, a detailed search is performed using the upper hierarchy that requires a long processing time, so that the processing time can be suppressed as a whole, which is efficient. In addition, since the search is performed using the decrypted data, the search can be performed based on a more direct criterion.

Next, a third embodiment using a plurality of judgment criteria for searching will be described. FIG. 8 shows this configuration. Second
It is characterized in that it has a large number of similarity determination units 5 as compared with the embodiment of FIG. The configurations of the hierarchical encoder 2, the disk input / output unit 3 and the like are the same as those in the above-mentioned embodiment, and the description thereof will be omitted here. However, the hierarchical encoder 2 performs a plurality of types of hierarchization and stores them in the disc separately. FIG. 9 shows this example. In this figure, only the R component is shown, and it is assumed that similar data exists for the G and B components. An example of four layers is shown, and it is assumed that only the lowest layer has three types of hierarchical structure. Of the three layers, the first layer type is a layer based on spectral components,
An example of this is shown in FIG. It is assumed that the input image is converted into a spectral component of 8 bits per pixel and is processed in 8 × 8 block units. The spectral components are read in the order shown in FIG. 11, and in the lowest layer, only the first 16 components per block are stored for 1620 blocks per frame and 1 frame for 8 frames. The second, third, and fourth hierarchies are also stored for the spectral components, and 16th to 17th to 32nd, 33 to 48th, and 49th to 64th coefficients are stored respectively.

The second layer type is a layer based on time, that is, frame dropping is performed to form a layer. FIG. 12 shows this example. For one frame, 8-bit data per pixel is stored for each block, and the data is stored for every four frames, that is, for the first and fourth frames in this example. The amount of data at this time is the same as that of the first layer of the spectral component.

The third layer type is a layer based on amplitude components. An example of this is shown in FIG. Upper 2 bits per coefficient,
Stored are 64 coefficients per block, 1620 blocks per frame, and 8 frames for one sequence.
The amount of data at this time is the same as that of the two types of layers.

The data of the three types of the three lowest layers is read out first, the two types of layer data are decoded by the local decoder 60, and the remaining one type is not decoded and corresponds respectively. Are distributed to the similarity determination units 5. In the hierarchical similarity determination unit 5, the decoded data is the same as that shown in FIG. 7, and the undecoded data is that shown in FIG. However, both are configured so that the comparator 29 at the final stage is removed.
The operation is similar to the above. Each similarity determination unit is multiplied by the weight set by the user in advance, the sum is taken, and then output to the search unit. The subsequent processing is the same as in the first and second embodiments. If necessary, the above process is repeated in the same manner as the procedure of FIG. 5 by increasing the number of layers and repeating until the user is satisfied. Further, in addition to increasing the number of layers in the procedure of FIG. 5 and interactively changing the weighting coefficient, the image desired by the user can be searched. Also, by setting the weighting coefficient to 0, it is possible to change the type of hierarchy used for the search.

In the above-described embodiment, as in the first and second embodiments, only the data of the lower hierarchy is used in the initial stage where there are many objects to be detected, so that the retrieval process can be performed in a short period of time, and the data of the lower hierarchy is Since only the data is transmitted and displayed to the user terminal, this can be done at high speed and economically. Further, since a plurality of layered data are used as search criteria after adding user-defined weights, it is possible to flexibly use various types of criteria to perform efficient search work. You can

[0025]

According to the present invention, since only low-layer data is used in the initial stage where there are many detection targets, search processing can be performed in a short period of time, and only low-layer data can be used by the user. This is also fast and economical because it is transmitted and displayed on the terminal. After the search range is narrowed down to some extent, a detailed search is performed using the upper hierarchy that requires a long processing time, so that the processing time can be suppressed as a whole, which is efficient. It is sufficient to interactively perform the above search process to the extent required by the user, and a very flexible search operation becomes possible.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a first exemplary embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of a similarity determination unit in the first exemplary embodiment of the present invention.

FIG. 3 is a diagram showing a range of orthogonal transform coefficients for which a sum is calculated in the first embodiment of the present invention.

FIG. 4 is a diagram showing a configuration of a search unit in the first embodiment of the present invention.

FIG. 5 is a diagram showing a conversational search procedure in the first embodiment of the present invention.

FIG. 6 is a diagram showing a configuration of a second exemplary embodiment of the present invention.

FIG. 7 is a diagram showing a configuration of a similarity determination unit according to a second exemplary embodiment of the present invention.

FIG. 8 is a diagram showing a configuration of a third exemplary embodiment of the present invention.

FIG. 9 is a diagram showing how a hierarchical code is stored in the third embodiment of the present invention.

FIG. 10 is a diagram showing how hierarchical codes are stored by spectral components in the third embodiment of the present invention.

FIG. 11 is a diagram showing a scan order of orthogonal transform coefficients in the third embodiment of the present invention.

FIG. 12 is a diagram showing how hierarchical codes are stored according to time in a third embodiment of the present invention.

FIG. 13 is a diagram showing how hierarchical codes are stored by amplitude components in the third embodiment of the present invention.

[Explanation of symbols]

2 ... Hierarchical encoder, 5 ... Similarity determination unit, 4 ... Search unit, 60 ... Local decoder.

Claims (5)

[Claims] 1. An image data retrieval device for selecting arbitrary image data from a plurality of image data stored in advance in a plurality of hierarchies and hierarchized in a plurality of hierarchies. A recording unit for storing the image data in association with the frame number, and the image data of the lowest hierarchy having a small amount of information is sequentially read from the storage unit, and each image data is compared with a predetermined similarity determination standard to perform the similarity determination. An image data retrieving apparatus comprising: a similarity determining means for selecting image data similar to a reference; and a means for displaying the selected image data. 2. The similarity determining means determines the target image data by the weighted sum of the coded image data in each layer and the reference value obtained by comparing the determination criteria corresponding thereto. The image data according to claim 1, wherein the image data is selected.
Search device. 3. The similarity determining means searches for an image using the coded data of the lowest layer, and if the amount of data is equal to or more than a predetermined amount, the coded data of a higher layer is searched. Using the image again to calculate the selected data amount, the above procedure is repeated until the selected image data amount becomes less than or equal to a predetermined amount. The image data retrieval apparatus according to claim 1, wherein data is selected. 4. The similarity determining means searches for an image using the coding data of the lowest hierarchy and presents a representative frame to the user to select a candidate range. 2. A desired image data is selected by performing a search using the data of the upper hierarchy within the range and repeating the above procedure to narrow down the search range.
An image data retrieval device according to the item. 5. The target image data is selected according to the type and number of hierarchical codes used for retrieval set according to an instruction from the user, as the similarity criterion. The image data retrieval device according to claim 1.