CN1692646A - Image description system and method thereof - Google Patents
Image description system and method thereof Download PDFInfo
- Publication number
- CN1692646A CN1692646A CN 200380100383 CN200380100383A CN1692646A CN 1692646 A CN1692646 A CN 1692646A CN 200380100383 CN200380100383 CN 200380100383 CN 200380100383 A CN200380100383 A CN 200380100383A CN 1692646 A CN1692646 A CN 1692646A
- Authority
- CN
- China
- Prior art keywords
- record
- image
- look
- characteristic quantity
- classification chart
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
In a picture description system, a memory unit stores description schemes defined for every category of picture. When a picture is specified, a control unit specifies features extractable from the specified picture with reference to a description scheme in the memory unit associated with category of the specified picture. A description file generating unit extracts data associated with the specified features from the specified picture and generates a description file of the specified picture.
Description
Technical field
The present invention relates to be used for record and narrate the system and the method thereof of the various features of multimedia messages.
Background technology
Following with the internet is network broadband of representative, and usually, not only online widely text (literal) information that provides also provides the multimedia messages that comprises image and sound.This can contact diversified information easily for the user is an advantage, but also negative effect can occur, because the information that provides is too assorted too much, so instead real necessary Useful Information more and more is difficult to obtain.
Under this situation,, the technology of metadata as searching object just come into one's own as the method for retrieving, filter or organize multimedia messages effectively.Metadata is the data that show the feature of extracting out with certain form compactly from content of multimedia, by with its as directly the retrieval object, can improve recall precision.Especially, vision or information acoustically is most to be difficult to showing with concrete language, is proper with showing with the form of metadata behind the information quantization of more approaching perception again.
Under such background, provide the unified method of the metadata of recording and narrating content of multimedia by MPEG-7.Common so-called MPEG-7 Visual (video) is a part wherein, and it provides the standardized format ISO/IEC15938-3 of the signal characteristic of recording and narrating video content (below, be called the video features amount).In MPEG-7 Visual, the video that the video features amount and being used for of having stipulated video content is recorded and narrated the video features amount is recorded and narrated the generation method of symbol.Have again, picture material comprises rectangular image resemble the digital photos, clips and pastes pattern arbitrarily shaped images such as (clip-art), as the rectangle live image (video sequence) of the set of rectangular frame, in the live image the arbitrary shape district or as object video of object sequence etc.
Below, record and narrate the example that accords with as video, record and narrate symbol with the edge: edge histogram (EdgeHistogram) is that example illustrates existing image description system.
Edge histogram is that local edge information is made the figure that forms behind the histogram, is to be used for image is divided into 4 * 4 sections, records and narrates the record symbol that there is the edge of what 5 types stipulating in each section with 3 bits respectively.The characteristic quantity of edge histogram generates as following.
D＝[E
ij(i＝1、2、…16，j＝1、2、…5)]
Here, E
IjJ border element among the expression piece i (order of raster scan).The structure of recording and narrating symbol resembles and carries out following.At first, image is divided in length and breadth each 4 section totally 16 sections.Secondly, utilize the edge of mask (mask) computing to each zone detection all directions.When calculating output, the Nogata of histogrammic correspondence is thrown a ticket, thus the construction feature amount above threshold value.
The characteristic quantity that generates is according to resembling the grammer of stipulating the table 1 in MPEG-7 Visual part, for example resemble to record and narrate the table 2.
Table 1
<complexType?name＝″EdgeHistogramType″final＝″#all>
<complexContent>
<extension?base＝″mpeg7:VisualDType″>
<sequence>
<element?name＝″BinCounts″>
<sinpleType>
<restriction>
<simpleType>
<listitemType＝″mpeg7:unsigned3″/>
</simpleType>
<lengthvalue＝″80″/>
</restriction>
</simpleType>
</element>
</sequence>
</extension>
</complexContent>
</complexType>
Table 2
<Descriptor?xsi:Type＝″EdgeHistogramType″>
<BinCounts>121123.....11511</BinCounts>
</Descriptor>
Utilization is recorded and narrated system's conduct that token is stated picture signal characteristics by the video of MPEG-7 Visual decision " MPEG-7 XM software " provide.In this system, the user specifies becoming the image that generates the object of recording and narrating symbol, selects the video features amount of extracting out.Constitute the video of selecting and record and narrate of the image extraction of the video features amount of symbol from appointment.Thus, can generate the record file, this record file utilizes video to record and narrate token and has stated the video features amount of having extracted out.
About having used the image description of recording and narrating symbol, various schemes have been proposed.For example, open in the 2002-170116 communique, disclose a kind of method, record and narrate in the symbol and imbedded enough spatial informations, record and narrate image, carry out the identification of image easily according to this content the spy.
As mentioned above, metadata is the data that show the feature of extracting out with certain form compactly from content of multimedia, by it is directly improved recall precision as searching object.Therefore, how generating the metadata that can show content of multimedia rightly, is the key factor that directly influences recall precision and precision.
But, in above-mentioned existing systems, no matter be the record symbol that can utilize for the kind of image, still unavailable record symbol, all irrelevant with the kind of image, use all image descriptions to accord with and record and narrate image.Therefore, use sometimes certain type image and inappropriate record accorded with and carry out image description.For example, utilize mobile activity to record and narrate sometimes and accord with the rectangular image of recording and narrating as rest image, and so on.
In addition, directly use in other systems, must support all videos to record and narrate the tool using of symbol in order to make the record file that makes towards the system of certain particular type.Therefore, there is the very large problem of system scale.
Relevant with above-mentioned explanation, open in the 2001-57057 communique the spy and to record disc reproducing apparatus.In this conventional example, audio/video data is read and audio/video order information, target information, title are provided with positional information and management information from CD by the portion of reading.Portion is read in control part control.When dish can identify when being DVD-Audio, storage portion stores AMG, and then retrieval VGM when having VGM, store VGM simultaneously.Input part receives user's indication, selects one among AMG and the VGM.
In addition, open in the 2001-167095 communique the spy and record image indexing system.In this conventional example, characterization symbol generating unit extracts image feature amount from input image data out and generating feature is recorded and narrated symbol, and is stored in accordingly in the image information storage part with input image data.The attribute list generating unit makes attribute list according to the attribute information of following the input image data input.Image retrieval portion is when importing the search condition relevant with attribute information, export the attribute information that is fit to search condition behind the searching attribute table again, when input accords with relevant search condition with characterization, export the view data that is fit to search condition behind the retrieving image information storage part again.
In addition, open the interactive system that discloses in the 2001-292425 communique with media content the spy.In this conventional example, the controller for controlling media output equipment makes its output medium content.Dispenser is to the classification on metadata and the interaction element allocated semantics.Selection portion is selected 1 from a plurality of semantic types.Efferent output under the form that relies on selecteed semantic type belongs to the metadata or the interaction element of the semantic type of having selected.
In addition, open the using method that discloses audio video system in the 2001-346140 communique the spy.In this conventional example, handle sound, image at least and comprise a kind of in the middle of the animation of a plurality of frames, provide a kind of consumer taste to record and narrate, record and narrate and use sound, image at least and comprise a plurality of hobbies of the central a kind of relevant user of the animation of a plurality of frames.Provide the protection attribute that expression is open or maintain secrecy at least one hobby.
In addition, open the spy and disclose the use resume that are used for managing audio and video information among the 2002-184157 and record and narrate scheme.In this conventional example, use the resume program can the record of the content of multimedia of customer consumption be conducted interviews, have the ability of the action that monitoring user carries out on various machines such as AV device, terminal.Use the resume module by configuration layer, in the action of user's appointment, only collect the confirmed action message of record.When detecting the user action of having promised to undertake, use the resume program to specified action, take place constantly, with move the identifier of relevant program/content, the content record information of appending and be recorded in the user action resume composition.Use record information to use user's selection resume composition, carry out record with the subclass of predesignating that the form of table is recorded and narrated content, and show as classification chart.
Summary of the invention
The object of the present invention is to provide a kind of image description system and method, can extract the suitable feature amount out video content.
Another object of the present invention is to provide a kind of image description system, and the kind optimization of the instrument by making support is come simplied system structure.
Another purpose of the present invention is to provide a kind of image description system and method, and whether can verify the narration way of the record file of image suitable.
Image description system of the present invention has storage part, and storage is to the record classification chart of each image type definition; And control part, in case after image is specified just with reference to the record classification chart corresponding storage portion stores with the type of image this appointment, specify the characteristic quantity that can from specify image, extract out.
Here, also can and then have and record and narrate the file generating unit, extract out with by the relevant data of specify image characteristic specified amount, the record file of generation specify image.
In addition, control part preferably can make specific characteristic quantity show on display part selectively.Here, also can and then have the file of record generating unit, extract out, generate the record file of specify image from by the data of selecting the specify image characteristic specified amount relevant with characteristic quantity.
In addition, preferably and then have the file verification of record portion, uses the record file of the record classification chart checking corresponding by the generation of record file generating unit with the type of specify image.
In addition, storage part is stored preferably that the rectangular image of recording and narrating rectangular image is recorded and narrated classification chart, the arbitrarily shaped image of recording and narrating the image of arbitrary shape is recorded and narrated classification chart, is recorded and narrated as the rectangle live image of the live image of the set of rectangular frame and record and narrate classification chart and record and narrate as the image object of the target of the arbitrary shape in the live image of the set of rectangular frame and record and narrate at least one classification chart in the classification chart.At this moment, rectangular image is recorded and narrated classification chart and is preferably had at least one characteristic quantity in a plurality of characteristic quantities that comprise look distribution, look configuration, color temperature, lighting condition correction look (Illumination InvariationColor), edge distribution and texture at least.In addition, the characteristic quantity of at least one is made of at least 1 selectable record symbol respectively, look distribution characteristics amount is made of a plurality of record symbols that comprise advantage look (DominantColor), scalable look (ScalableColor) and look structure (ColorStructure) at least, wherein have at least 1 to be selectable, texture characteristic amount is browsed a plurality of record symbol formations of (TextureBrowsing) by comprising similar texture (HomogeneousTexture) and texture at least, and it is selectable preferably wherein having 1 at least.
In addition, arbitrarily shaped image is recorded and narrated classification chart and also can be had at least one characteristic quantity in a plurality of characteristic quantities that comprise look distribution, look configuration, color temperature, lighting condition correction look, edge distribution, texture and shape at least.At this moment, the characteristic quantity of at least one is made of at least 1 selectable record symbol respectively, the shape facility amount is made of a plurality of record symbols that comprise contour shape (ContourShape) and region shape (RegionShape) at least, can select wherein have 1 at least.
In addition, the rectangle live image is recorded and narrated at least one the characteristic quantity in a plurality of characteristic quantities that classification chart also can comprise the time series data, representative feature and the mobile activity that comprise rectangular frame at least.At this moment, the characteristic quantity of at least one is made of at least 1 selectable record symbol respectively, the time series data have and comprise look at least and distribute, the look configuration, color temperature, lighting condition correction look, the characteristic quantity of at least one in a plurality of characteristic quantities of edge distribution and texture, each characteristic quantity is made of at least 1 selectable record symbol respectively, look distribution characteristics amount is by comprising the advantage look at least, a plurality of record symbols of scalable look and look structure constitute, wherein have at least 1 to be selectable, texture constitutes by comprising a plurality of records symbols that similar texture and texture browse at least, preferably wherein has at least 1 to be selectable.
In addition, representative feature has at least one the characteristic quantity in a plurality of characteristic quantities that comprise look distribution, look configuration, color temperature, lighting condition correction look, edge distribution and texture at least, each characteristic quantity is made of at least 1 selectable record symbol respectively, look distribution characteristics amount is made of a plurality of record symbols that comprise advantage look, scalable look and look structure at least, wherein have at least 1 to be selectable, texture characteristic amount constitutes by comprising a plurality of records symbols that similar texture and texture browse at least, preferably wherein has at least 1 to be selectable.
Image object is recorded and narrated at least one the characteristic quantity in a plurality of characteristic quantities that classification chart preferably has the time series data, representative feature, mobile activity, movement of objects and the change of shape that comprise rectangular frame at least.At this moment, the characteristic quantity of at least one is made of at least 1 selectable record symbol respectively, movement of objects is made of a plurality of record symbols that comprise movement locus (MotionTrajectory) and parameter motion (ParameterMotion) at least, and it is selectable preferably wherein having 1 at least.
In addition, the time series data have at least one the characteristic quantity in a plurality of characteristic quantities that comprise look distribution, look configuration, color temperature, lighting condition correction look, edge distribution and texture at least, each characteristic quantity is made of at least 1 selectable record symbol respectively, look distribution characteristics amount is made of a plurality of record symbols that comprise advantage look, scalable look and look structure at least, wherein have at least 1 to be selectable, texture constitutes by comprising a plurality of records symbols that similar texture and texture browse at least, preferably wherein has at least 1 to be selectable.
In addition, representative feature has at least one the characteristic quantity in a plurality of characteristic quantities that comprise look distribution, look configuration, color temperature, lighting condition correction look, edge distribution and texture at least, each characteristic quantity is made of at least 1 selectable record symbol respectively, look distribution characteristics amount is made of a plurality of record symbols that comprise advantage look, scalable look and look structure at least, wherein have at least 1 to be selectable, texture constitutes by comprising a plurality of records symbols that similar texture and texture browse at least, preferably wherein has at least 1 to be selectable.
In addition, the storage part rest image that also can store the characteristic quantity of recording and narrating rest image is recorded and narrated classification chart, is recorded and narrated as the rectangle live image of the live image of the set of rectangular frame and record and narrate classification chart and record and narrate as the image object of the object of the arbitrary shape in the live image of the set of rectangular frame and record and narrate at least one classification chart in the classification chart.At this moment, rest image is recorded and narrated at least one the characteristic quantity in a plurality of characteristic quantities that classification chart also can have the look distribution characteristics amount that comprises at least, look configuration feature amount, color temperature characteristic quantity, lighting condition correction color characteristic amount, edge distribution characteristic quantity and texture characteristic amount.In addition, the characteristic quantity of at least one is made of at least 1 selectable record symbol respectively, look distribution characteristics amount is by comprising the advantage look at least, a plurality of record symbols of scalable look and look structure constitute, wherein have at least 1 to be selectable, look configuration feature amount is made of a plurality of record symbols that comprise look configuration (ColorLayout) at least, wherein have at least 1 to be selectable, the color temperature characteristic quantity is made of a plurality of record symbols that comprise color temperature at least, wherein have at least 1 to be selectable, lighting condition correction color characteristic amount is made of a plurality of record symbols that comprise lighting condition correction look at least, wherein have at least 1 to be selectable, the edge distribution characteristic quantity is made of a plurality of record symbols that comprise edge histogram at least, wherein have at least 1 to be selectable, texture characteristic amount constitutes by comprising a plurality of records symbols that similar texture and texture browse at least, preferably wherein has at least 1 to be selectable.Rest image is recorded and narrated classification chart and then is comprised the shape facility amount, and the shape facility amount is made of a plurality of record symbols that comprise contour shape and region shape at least, preferably wherein has at least 1 to be selectable.
In addition, the storage part rest image that also can store the characteristic quantity of the recording and narrating rest image live image recording and narrating classification chart and record and narrate live image is recorded and narrated at least one classification chart in the classification chart.At this moment, live image is recorded and narrated at least one the characteristic quantity in a plurality of characteristic quantities of mobile activity that classification chart has the representative feature of the time series data that comprise the activity diagram picture frame at least, live image and live image, and each characteristic quantity also can be selected to comprise at least 1 and record and narrate symbol.At this moment, live image record classification chart also can and then comprise the mobile record of live image and the change of shape record of live image.
In addition, record and narrate the image description system that classification chart is used for specifying with reference to the record classification chart corresponding with the type of the image of appointment the characteristic quantity that can extract out from the image of appointment, have at least one the characteristic quantity in a plurality of characteristic quantities of mobile activity of the representative feature of the time series data that comprise the activity diagram picture frame at least, live image and live image, each characteristic quantity also can comprise at least 1 selectable record symbol.Here, live image record classification chart also can and then comprise the mobile record of live image and the change of shape record of live image.
Another aspect of the present invention relates to a kind of image description method, step, the image of the record classification chart of each image type definition is specified the step of specifying the characteristic quantity that can extract out from specify image behind the back retrieval record classification chart corresponding with the type of this specify image again and show selectively and can realize from the step of the characteristic quantity of specify image extraction by storage.
Here, the image description method also can and then have the step of selecting desired characteristic quantity from the characteristic quantity that shows; With from specify image, extract regeneration behind the characteristic quantity according to desired characteristic quantity out and record and narrate the step of file.In addition, also can and then have the step of using the record classification chart corresponding to verify the record file that is generated with the type of specify image.
Another aspect of the present invention relates to the executable software product of a kind of computer, has image and specifies the back by the function of storage to the storage part retrieval of the record classification chart of each the image type definition record classification chart corresponding with the type of the image of this appointment; And specify the function of the characteristic quantity that can extract out from specify image and the function of the characteristic quantity that can from specify image, extract out of demonstration selectively according to the record classification chart of retrieval.
Here, the image description method also can and then have when having selected desired characteristic quantity from the characteristic quantity that software product shows, extracts the function of regeneration record file behind the characteristic quantity from specify image according to desired characteristic quantity.
In addition, software product also can and then have the function of using the record classification chart corresponding with the type of specify image to verify the record file that is generated.
Another aspect of the present invention relates to a kind of reference record classification chart corresponding with the type of the image of appointment and specify the record classification chart that uses in the image description system of the characteristic quantity that can extract out from specify image, have and comprise look distribution characteristics amount at least, look configuration feature amount, the color temperature characteristic quantity, lighting condition correction color characteristic amount, the characteristic quantity of at least one in a plurality of characteristic quantities of edge distribution characteristic quantity and texture characteristic amount, look distribution characteristics amount is by comprising (advantage look) at least, a plurality of record symbols of scalable look and look structure constitute, wherein have at least 1 to be selectable, look configuration feature amount is made of the record symbol that comprises the look configuration at least, wherein have at least 1 to be selectable, the color temperature characteristic quantity is made of the record symbol that comprises color temperature at least, wherein have at least 1 to be selectable, lighting condition correction color characteristic amount is made of the record symbol that comprises lighting condition correction look at least, wherein have at least 1 to be selectable, the edge distribution characteristic quantity is made of a plurality of record symbols that comprise edge histogram at least, wherein have at least 1 to be selectable, texture characteristic amount constitutes by comprising a plurality of records symbols that similar texture and texture browse at least, wherein has at least 1 to be selectable.
Record and narrate classification chart and then comprise the shape facility amount, the shape facility amount is made of a plurality of record symbols that comprise contour shape and region shape at least, wherein has at least 1 to be selectable.
As mentioned above, in the present invention, select significant video features amount easily, and, certain video features amount that has showed specify image can be extracted out.
In addition, by to each image type definition record and narrate classification chart, the characteristic quantity that should support and the kind of the instrument of record can be reduced to necessary bottom line, and can simplied system structure.
And then, wish to use the record classification chart corresponding to remove to verify the record file that has generated with the type of specify image.By the record file that will generate like this and original image description classification chart contrast, whether suitable, can further improve image retrieval efficient and precision if can verify the narration way of the record file of image.
Description of drawings
Fig. 1 is the block diagram of the image description system structure of expression the 1st embodiment of the present invention.
Fig. 2 is included in the ideograph that rectangular image is recorded and narrated the record instrument in the classification chart among expression the 1st embodiment.
Fig. 3 is that (eXtensible Markup Language: the rectangular image of extending mark language) being write as is recorded and narrated the schematic diagram of the example of classification chart with XML.
Fig. 4 is included in the ideograph that clip image is recorded and narrated the record instrument in the classification chart among expression the 1st embodiment.
Fig. 5 is the figure that clip image that expression is write as with XML is recorded and narrated the example of classification chart.
Fig. 6 is included in the ideograph that image sequence is recorded and narrated the record instrument in the classification chart among expression the 1st embodiment.
Fig. 7 is the figure that image sequence that expression is write as with XML is recorded and narrated the example of classification chart.
Fig. 8 is included in the ideograph that image object is recorded and narrated the record instrument in the classification chart among expression the 1st embodiment.
Fig. 9 is a schematic diagram of recording and narrating the example of classification chart with the image object that XML is write as.
Figure 10 is the figure that the video features amount of expression specify image when being rectangular image selected an example of picture.
Figure 11 is the figure that the video features amount of expression specify image when being arbitrarily shaped image selected an example of picture.
Figure 12 is the figure that the video features amount of expression specify image when being the rectangle live image selected an example of picture.
Figure 13 is the figure that the video features amount of expression specify image when being arbitrarily shaped image selected an example of picture.
Figure 14 is the flow chart of the image description action of expression the 1st embodiment.
Figure 15 is the block diagram of the image description system structure of expression the 2nd embodiment of the present invention.
Figure 16 is the block diagram of the image description system structure of expression the 3rd embodiment of the present invention.
Figure 17 is the figure that stagnant zone that expression is write as with XML is recorded and narrated the example of classification chart.
Figure 18 is the figure that live image that expression is write as with XML is recorded and narrated the example of classification chart.
Embodiment
Below, the image description system that present invention will be described in detail with reference to the accompanying.
(the 1st embodiment)
Fig. 1 is the block diagram of the image description system structure of expression the 1st embodiment of the present invention.In Fig. 1, input part 101 is input equipments such as keyboard or positioning equipment, specifies as the image that should extract the object of video features amount out, specifies the video features amount of extracting out, or is used for importing various command.Display part 102 is monitors, shows video features amount described later display frame, and collaborative input part 101 provides user interface.The program control processor 103 of native system is controlled with the video features amount by executive control program 104 and is extracted the relevant processing or the action of whole system out.
The image description system of present embodiment is provided with image description classification chart search part 105, image storage classification chart storage part 106, video features amount extraction unit 107 and records and narrates file generating unit 108.Image description classification chart search part 105, video features amount extraction unit 107 and record file generating unit 108 are carried out the retrieval of image description classification chart described later, the extraction of video features amount and the generation of record file respectively under the control of program control processor 103.
The a plurality of image description classification charts of storage in the image storage classification chart storage part 106.Here, storage rectangle image description classification chart 200, clip image (arbitrarily shaped image) are recorded and narrated classification chart 300, image sequence (rectangle live image) and are recorded and narrated at least 1 image description classification chart that classification chart 400, image object are recorded and narrated classification chart 500 or selected in the middle of their.The back describes these image description classification charts in detail.
When receiving image description classification chart search instruction from program control processor 103, image description classification chart search part 105 is from the type corresponding record classification chart of image description classification chart storage part 106 retrievals with the image of appointment.According to the image description classification chart of reading, in accordance with regulations form shows the kind (details aftermentioned) of the video features amount that can extract out from the image of appointment on display part 102.
When receiver, video characteristic quantity extract instruction, video features amount extraction unit 107 is imported the image of appointments from image data storage portion 110, and extracts the video features amount of appointment out from this image.Record and narrate file generating unit 108 and generate the record file of stating with video record token according to video features amount of extracting out and parameter.The record file storage of Sheng Chenging is in recording and narrating file storage part 109 thus, and is used for image retrieval etc.
The image description classification chart
(A) rectangular image is recorded and narrated classification chart
Signal characteristic with the rectangular image of record resemble the digital photos is recorded and narrated classification chart for purpose designs rectangular image.Its main purpose retrieves the image with similar signal mode from digital picture documents such as digital photos document.
The signal characteristic that obtains from rectangular image can be divided into 6 groups, that is, 1) the look distribution, 2) the look configuration, 3) color temperature, 4) lighting condition correction look, 5) edge, 6) texture.The video features amount that belongs to each group is determined as follows respectively.
1) advantage look/scalable look/look structure
2) look configuration
3) color temperature
4) lighting condition correction look
5) edge histogram
6) similar texture/texture is browsed
When in each group similar video features amount being arranged, all use improperly in the lump, wish to select as required one or more uses.Table 3 illustrates the differentiation of a plurality of video features amounts of distribution of expression look and texture for example and uses.
Table 3 rectangular image is recorded and narrated classification chart
| The signal characteristic group | The video features amount | Purposes |
| Look distributes | The advantage look | The correct record that limits the look zone is used |
| Scalable look | General application is used | |
| The look structure | Use when requiring high accuracy | |
| The look configuration | The look configuration | - |
| Color temperature | Color temperature | - |
| Lighting condition correction look | Lighting condition correction look | - |
| The edge | Edge histogram | - |
| Texture | Similar texture | Use during precision prescribed |
| Texture is browsed | Figure browse usefulness roughly |
The purposes of 3 characteristic quantities that the expression look distributes is as shown in table 3.Promptly, (1) the advantage look is applicable to the correct record that limits the look zone, (2) scalable look is applicable to and requires and the universal products such as application that present widely used look histogram has interchangeability, and (3) look structure is applicable to precision prescribed height such as medical imaging and situation that cost is not too considered.Therefore, the corresponding rectangular image that designs with these purposes is recorded and narrated classification chart, and it is a kind of that it can be selected in advantage look, scalable look and look structure at least.
During the differentiation of 2 characteristic quantities of representing grain was used, texture was browsed and is applicable to the situation that only need browse figure roughly, and similar texture is applicable to the purposes that required precision is higher.Therefore, the design rectangular image is recorded and narrated classification chart, makes it can select a kind of characteristic quantity as representing grain at least in similar texture and texture are browsed.And then the design rectangular image is recorded and narrated classification chart, makes it select the signal characteristic that needs in look distribution, look configuration, color temperature, lighting condition correction look, edge and texture.
Fig. 2 is the ideograph that rectangular image is recorded and narrated the record instrument in the classification chart that is included in of expression present embodiment.As shown in Figure 2, rectangular image is recorded and narrated the particular frame of classification chart 200 definite live images or the signal characteristic quantity of rectangle rest image.Rectangular image record classification chart 200 comprises look distribution record 201, look configuration record 202, edge record 203, color temperature record 204, the record 205 of lighting condition correction look and texture and records and narrates 206.
Fig. 3 is the figure of the example of the rectangular image record classification chart write as with XML (extending mark language) of expression.Recording and narrating classification chart can carry out with language arbitrarily, the record (or more record) that comprises arbitrarily to be comprised.Have, in Fig. 3, the title with the name attribute representation in the element element is arbitrarily again, but wishes it is the title that shows the feature of the record symbol of representing with type.
(B) clip image is recorded and narrated classification chart
The picture signal characteristics with arbitrary shape that is referred to as clip image with record is that purpose designed image folder is recorded and narrated classification chart.Its main purpose is to retrieve the folder with similar signal mode the material folder that uses from content production etc.The signal characteristic that obtains from rectangular image can adapt to the image of all arbitrary shapes.The signal characteristic that obtains from arbitrarily shaped image can also obtain shape facility except the signal characteristic that obtains from rectangular image.The video features amount of expression shape facility has contour shape and region shape, but that both use in the lump is improper, need select a kind of at least according to purpose.Table 4 also illustrates the differentiation of 2 video features amounts of expression shape facility and uses except that rectangular image is recorded and narrated classification chart.
Table 4 clip image is recorded and narrated classification chart
| The signal characteristic group | The video features amount | Purposes |
| Look distributes | The advantage look | The correct record that limits the look zone is used |
| Scalable look | General application is used | |
| The look structure | Use when requiring high accuracy | |
| The look configuration | The look configuration | - |
| Color temperature | Color temperature | - |
| Lighting condition correction look | Lighting condition correction look | - |
| The edge | Edge histogram | - |
| Texture | Similar texture | Use during precision prescribed |
| Texture is browsed | Figure browse usefulness roughly | |
| Shape | Contour shape | Use in the time of can recording and narrating closed curve and require firm rotary body character |
| Region shape | General service |
As shown in table 4, contour shape is applicable to the situation that can record and narrate closed curve and require firm rotary body character, and region shape is applicable to general purposes in addition.Therefore, designed image folder is recorded and narrated classification chart and is made it can select at least one characteristic quantity as the performance shape in contour shape and the region shape.
Fig. 4 is the ideograph that clip image is recorded and narrated the record instrument in the classification chart that is included in of expression present embodiment.Clip image is recorded and narrated the signal characteristic that classification chart determines to have the image of arbitrary shape.As shown in Figure 4, clip image record and narrate classification chart 300 comprise shape record and narrate 301, be included in rectangular image record and narrate look in the classification chart 200 distribute record and narrate 201, the look configuration records and narrates 202, the edge records and narrates 203, color temperature records and narrates 204, lighting condition correction look record and narrate 205 and texture record and narrate 206.Designed image folder image description classification chart makes it can select the signal characteristic that needs from their centres.
Fig. 5 is the figure that clip image that expression is write as with XML is recorded and narrated the example of classification chart.Recording and narrating classification chart can carry out with language arbitrarily, the record (or more record) that comprises arbitrarily to be comprised.Have, in Fig. 5, the title with the name attribute representation in the element element is arbitrarily again, but wishes it is the title that shows the feature of the record symbol of representing with type.
(C) image sequence record classification chart is that purpose designed image sequence is recorded and narrated classification chart with the signal characteristic of recording and narrating live image.Its main purpose is to retrieve the image with similar signal mode from image document.
The signal characteristic that obtains from live image is divided into 3 groups, that is, (1) to the time series data of the characteristic quantity of rectangular image, (2) representative is included in the characteristic quantity of all frames in the live image, and (3) are categorized as 3 mobile groups.The video features amount that belongs to each group can be determined as follows respectively.
1) video time series (VisualTimeSeries)
2) GofGop look (GofGopColor)
3) mobile activity (MotionActivity)
When recording and narrating being included in frame in the live image as the unit that gives characteristic quantity, can utilize time series to arrange gatherer (VisualTimeSeries), when live image integral body is recorded and narrated, can utilize representative feature gatherer (GofGopColor).In addition, also can utilize above-mentioned two kinds of gatherers.Characteristic quantity can be recorded and narrated symbol and be assigned to the position of liking.
The effect of gatherer is similar to adhesive, the characteristic quantity of recording and narrating the part of certain content is recorded and narrated Fu Qungui handle together.Video time series is that the characteristic quantity record symbol that will arrange on time shaft is returned the characteristic quantity of statement together, comprise: record and narrate the regular video time series (Regular Visual TimeSeries) of symbol and press 2 kinds of irregular video time series (Irregular Visual TimeSeries) that symbol is recorded and narrated in the variable interval configuration by the fixed intervals configuration, be assigned on the position of each frame but characteristic quantity can be recorded and narrated to accord with.In addition, the GofGop look can be recorded and narrated symbol with 1 characteristic quantity and distribute to whole live image.
The designed image sequence is recorded and narrated classification chart, makes it record and narrate time series data, the representative feature the classification chart and to move the signal characteristic that middle selection needs from being included in image sequence.Table 5 presentation video sequence is recorded and narrated classification chart.
Table 5 image sequence is recorded and narrated classification chart
| The signal characteristic group | The video features amount | Purposes |
| Time series | Video time series (rectangular image record classification chart) | The frame that is included in the live image is recorded and narrated |
| Representative feature | GofGop look (rectangular image record classification chart) | Whole live image is recorded and narrated |
| Move | Mobile activity | - |
Fig. 6 is the ideograph that image sequence is recorded and narrated the record instrument in the classification chart that is included in of expression present embodiment.Image sequence is recorded and narrated the signal characteristic that classification chart is determined image sequence (set of a plurality of frames).Image sequence is recorded and narrated classification chart 400 and is comprised the time series arrangement gatherer 401 to the characteristic quantity of rectangular image, characteristic quantity gatherer 402 and the mobile activity record 403 that representative is included in all frames in the live image.
Fig. 7 is the figure that image sequence that expression is write as with XML is recorded and narrated the example of classification chart.Recording and narrating classification chart can carry out with language arbitrarily, the record (or more record) that comprises arbitrarily to be comprised.Have, in Fig. 7, the title with the name attribute representation in the element element is arbitrarily again, but wishes it is the title that shows the feature of the record symbol of representing with type.
(D) image object is recorded and narrated classification chart
Image object in MPEG-4 (Video Object) is that purpose designed image target is recorded and narrated classification chart to record and narrate the arbitrary shaped region in the live image and the signal characteristic of object.Its main purpose is to retrieve the image object with similar signal mode the image object document that uses from content production etc.
The signal characteristic that obtains from image sequence can adapt to all image objects.The signal characteristic that obtains from arbitrarily shaped image can also obtain the mobile message or the shape change in time of target except the signal characteristic that obtains from rectangular image.The signal characteristic that obtains from image object is divided into 1) movement of objects information and 2) such 2 groups of change of shape.The video features amount that belongs to each group can be determined as follows.
1) movement locus/parameter motion
2) change of shape
The video features amount of expression movement of objects information comprises movement locus and parameter motion, but that both use in the lump is improper, need select a kind of at least according to purpose.
Table 6 image object is recorded and narrated classification chart
| The signal characteristic group | The video features amount | Purposes |
| Time series | Video time series (rectangular image record classification chart) | At the frame that is included in the live image |
| Representative feature | GofGop look (rectangular image record classification chart) | Whole live image is recorded and narrated |
| Move | Mobile activity | - |
| Movement of objects information | Movement locus | The time series that utilizes interpolation to record and narrate the representative point in zone changes |
| The parameter motion | Can be approximately the record of moving of the object of rigid body | |
| Change of shape | Change of shape | - |
The parameter motion utilizes 5 kinds of Move Modes such as affine transformation and perspective transform to be similar to moving of whole zone.Purpose is to record and narrate moving of the object can be approximately rigid body.
Movement locus is represented the change in location of the time series of regional representative point (for example center of gravity), and records and narrates the position of sampling point on the time shaft and the interpolating method between sampling point.Can consider to be used for the walking track etc. by the performance personage, thereby from surveillance camera image data base for example, select to have carried out the people etc. of specific action.Therefore, the designed image target is recorded and narrated classification chart, makes it select some characteristic quantities as the expression shape from movement locus and parameter motion.And then, design activity image description classification chart, make its time series data, the representative feature that can record and narrate the classification chart to be comprised from image sequence and move in the signal characteristic that more needs of selection.
Fig. 8 is the ideograph that image object is recorded and narrated the record instrument in the classification chart that is included in of expression present embodiment.The signal characteristic of arbitrary shaped region or object in image object record classification chart 500 definite live images.Image object is recorded and narrated classification chart 500 and is comprised that movement of objects record 501, change of shape record 502 and representative to image object are included in the characteristic quantity that rectangle live image (image sequence) is recorded and narrated all frames in the classification chart 400.
Fig. 9 is the figure that image object sequence that expression is write as with XML is recorded and narrated the example of classification chart.Recording and narrating classification chart can carry out with language arbitrarily, the record (or more record) that comprises arbitrarily to be comprised.Have, in Fig. 9, the title with the name attribute representation in the element element is arbitrarily again, but wishes it is the title that shows the feature of the record symbol of representing with type.
＜video features amount is selected the demonstration example of picture 〉
(1) rectangular image
Figure 10 is the figure that the video features amount of expression specify image when being rectangular image selected an example of picture.Such as described, rectangular image record classification chart 200 comprises look distribution record 201, look configuration record 202, edge record 203, color temperature record 204, the record 205 of lighting condition correction look and texture and records and narrates 206 (with reference to Fig. 2).In the present embodiment, on picture, show, make the user can from these record instruments, select the signal characteristic that needs by the XML record of execution graph 3 is routine.
As shown in figure 10, can use pointing device such as mouse selectively to show look distribute (Color Distribution) 601, colorspace distribution (Spatial Distributionof Color) 602, lighting condition correction look (Illumination Independent Color) 603, color temperature (Color Temperature) 604, rim space distribute (SpatialDistribution of Edges) 605 and pattern (Homogeneous Pattern) 606.
Such as described, during demonstration, distribute 601 for look, at least can the selective advantage look, in scalable look and the look structure one.In addition, for pattern 606, can select at least one in browsing of similar texture and texture at least.In addition, by utilizing button clicks 607 such as mouse, can begin to extract out the video features amount of having selected.
Like this,, image description system can be provided, the suitable feature amount can be only selected, extract out rectangular image by to the suitable image description classification chart of rectangular image definition.
(2) arbitrarily shaped image
Figure 11 is the figure that the video features amount of expression specify image when being arbitrarily shaped image selected an example of picture.Such as described, clip image record classification chart 300 comprises shape record 301, look distribution record 201, look configuration record 202, edge record 203, color temperature record 204, the record 205 of lighting condition correction look and texture and records and narrates 206 (with reference to Fig. 4).In the present embodiment, on picture, show, make the user can from these record instruments, select the signal characteristic that needs by the XML record of execution graph 5 is routine.
As shown in figure 11, can use that pointing device such as mouse shows selectively that look distributes 701, colorspace distribution 702, lighting condition correction look 703, color temperature 704, rim space distribute 705, pattern 706 and shape 707.
Such as described, during demonstration,, can select in contour shape and the region shape at least for shape 707.Distribute 701 for look, at least can the selective advantage look, in scalable look and the look structure one.In addition, for pattern 706, can select in browsing one of similar texture and texture at least.
When having selected desired record,, can begin to extract out the video features amount of having selected by utilizing button click OK such as mouse.Like this, record and narrate classification chart by arbitrarily shaped image is defined suitable clip image, thereby image description system can be provided, can only select and extract out the suitable feature amount arbitrarily shaped image.
(3) image sequence
Figure 12 is the figure that the video features amount of expression specify image when being the rectangle live image selected an example of picture.Such as described, image sequence is recorded and narrated classification chart 400 and is comprised time series arrangement gatherer 401, representative feature gatherer 402 and mobile activity record 403 (with reference to Fig. 6).In the present embodiment,, thereby on picture, show, make the user can from these record instruments, select the signal characteristic that needs by the XML record example of execution graph 7.
As shown in figure 12, can use pointing device such as mouse to show selectively to distribute to that time series arranges (VisualTimeSeries) 801 is included in that rectangular image is recorded and narrated video features amount in the classification chart, the rectangular image that is included in of distributing to representative feature (GofGopColor) 802 is recorded and narrated video features amount and mobile activity (MotionActivity) 803 in the classification chart.
When having selected desired record,, can begin to extract out the video features amount of having selected by utilizing button click OK such as mouse.Like this, record and narrate classification chart by the rectangle live image is defined suitable image sequence, thereby image description system can be provided, the suitable feature amount can only be selected, extract out in this system to the rectangle live image.
(4) image object
Figure 13 is the figure that the video features amount of expression specify image when being the arbitrary shape live image selected an example of picture.Such as described, movement of objects record 501, change of shape record 502 and representative that image object record classification chart 500 comprises image object are included in the characteristic quantity (with reference to Fig. 8) that rectangle live image (image sequence) is recorded and narrated all frames in the classification chart 400.In the present embodiment,, thereby on picture, show, make the user can from these record instruments, select the signal characteristic that needs by the XML record example of execution graph 9.
As shown in figure 13, can use pointing device such as mouse to have to select to show distribute to that time series arranges (VisualTimeSeries) 901 be included in that rectangular image is recorded and narrated video features amount in the classification chart, the rectangular image that is included in of distributing to representative feature (GofGopColor) 902 is recorded and narrated video features amount, mobile activity (MotionActivity) 903, movement of objects (Motion) 904 and change of shape (Shape Variation) 905 in the classification chart.
Such as described, for movement of objects 904, can select in the motion of movement locus and parameter at least.When having selected desired record,, just can begin to extract out the video features amount of having selected by utilizing button click OK such as mouse.Like this, record and narrate classification chart by the arbitrary shape live image is defined suitable image object, thereby image description system can be provided, can only select, extract out the suitable feature amount the arbitrary shape live image.
＜image description action 〉
Secondly, describe the whole action of present embodiment in detail.
Figure 14 is the flow chart of the image description action of expression present embodiment.At first, with the image description classification chart can be stored in by the form that kind is retrieved in the image description classification chart storage part 106.Promptly, as shown in Figure 1, image description classification chart storage part 106 storage rectangle image description classification charts 200, arbitrarily shaped image are recorded and narrated classification chart 300, image sequence records and narrates classification chart 400 and image object is recorded and narrated classification chart 500, in addition, extract the necessary parameter setting of video features amount (steps A 1) out.The user utilizes input part 101 to specify as the image (steps A 2) that generates the object of recording and narrating file.As the direct input picture filename of the appointment of the image of recording and narrating object, also can from the image that carries out the list demonstration in advance, select by the user.
When having specified specify image, program control processor 103 indicating images are recorded and narrated the record classification chart of classification chart search part 105 retrieval institute important plan pictures.Image description classification chart search part 105 is retrieved (steps A 3) as clue (key) to image description classification chart storage part 106 with the type of specify image.When finding the image description classification chart corresponding with the type of specify image, image description classification chart search part 105 is read this image description classification chart and is sent program control processor 103 back to.Program control processor 103 utilizes the image description classification chart of reading, and shows that on display part 102 which (steps A 4) characteristic quantity that can extract out from specify image is.
Specifically, when having specified rectangular image, the rectangular image that reference has been read is is recorded and narrated classification chart, resembles as shown in Figure 10 to show (steps A 3.1).When having specified arbitrarily shaped image, the arbitrarily shaped image that reference has been read is is recorded and narrated classification chart, resembles as shown in Figure 11 to show (steps A 3.2).When having specified image sequence, the image sequence that reference has been read is is recorded and narrated classification chart, resembles as shown in Figure 12 to show (steps A 3.3).When having specified image object, the image object that reference has been read is is recorded and narrated classification chart, resembles as shown in Figure 13 to show (steps A 3.4).Have, these demonstrations also can be carried out according to the indication from input part 101 again.
The user utilizes input part 101 to specify the characteristic quantity (steps A 5) that should extract out from extracted out the characteristic quantity list that is presented at display part 102.When having specified the characteristic specified amount, program control processor 103 instruction video characteristic quantity extraction units 107 are extracted desired characteristic quantity out.The image that video features amount extraction unit 107 is read in appointment from image data storage portion 110 is extracted characteristic specified amount (steps A 6) out from this image.
Record and narrate file generating unit 108 and use video record token to state characteristic quantity and the parameter (steps A 7) that generates by video features amount extraction unit 107, the data of recording and narrating are generated (steps A 8) as recording and narrating file.Recording and narrating file can be stored in the record file storage part 109.
As mentioned above, in the 1st embodiment, when utilizing input part 101 to specify image, the image description classification chart search part 105 retrievals image description classification chart corresponding with the type of image, and can be from the video features amount of specify image extraction with the illustrative form demonstration of Figure 10～Figure 13.Therefore, the user specifies the video features amount of extraction easily.In addition, the kind of support facility can be reduced to necessary bottom line, can provide system configuration simple image description system.
The record file that generates is included in the characteristic quantity in the record file of a certain specific image and is included in the similar degree of the characteristic quantity in the record file of other image by evaluation, thereby can also be used for the similar image retrieval etc. of retrieval of similar image.Because of having only suitable record file just can be used to as similar image retrieval etc., so can improve the reliability and the precision of retrieval.
(the 2nd embodiment)
Figure 15 is the block diagram of the image description system structure of expression the 2nd embodiment of the present invention.The 2nd embodiment of the present invention is on the basis of the 1st embodiment shown in Figure 1 and then comprise the file verification portion 111 of recording and narrating.
Record and narrate file verification portion 111 and read in the image description classification chart that obtains from image description classification chart search part 105, whether the record file of file generating unit 108 generations is recorded and narrated in checking correct.Specifically, whether whether kind that confirm to record and narrate the characteristic quantity that file records and narrates is defined in the image description classification chart, and confirm to record and narrate file and record and narrate according to the description method of image description classification chart regulation.When recording and narrating file is when recording and narrating according to the description method of image description classification chart regulation, and file is recorded and narrated in output.
As mentioned above, be provided with the file verification portion 111 of recording and narrating in the 2nd embodiment, record and narrate file and the contrast of image description classification chart by making, whether the narration way of can authentication image recording and narrating file is suitable.
The record file that generates is included in the characteristic quantity in the record file of a certain specific image and is included in the similar degree of the characteristic quantity in the record file of other image by evaluation, thereby can also be used for the similar image retrieval etc. of the image of retrieval of similar.Because of having only suitable record file just can be used to as similar image retrieval etc., so can improve the reliability and the precision of retrieval.
(the 3rd embodiment)
Figure 16 is the block diagram of the image description system structure of expression the 3rd embodiment of the present invention.Comprise the file verification portion 111 of recording and narrating.
The image description system of present embodiment utilizes program control processor 120 to go to realize image description classification chart search part 105 shown in Figure 1, video features amount extraction unit 107, record and narrate file generating unit 108 and record and narrate file verification portion 111 by software.That is, program control processor 120 can be realized the image description function with the function equivalent that has illustrated by the image description program 121 of execute store storage in the 1st and the 2nd embodiment.Input part 101, display part 102, image description classification chart storage part 106, record file storage part 109 and image data storage portion 110 are the same with the 1st and the 2nd embodiment, be subjected to carries out image to record and narrate program control processor 120 controls of program 121, realize image description system of the present invention.
(the 4th embodiment)
The 4th embodiment of the present invention and the 1st embodiment difference shown in Figure 1 are: in the 4th embodiment, the image object that the stagnant zone of recording and narrating rest image is recorded and narrated classification chart, the rectangle live image of recording and narrating the set of rectangular frame is recorded and narrated classification chart and recorded and narrated image object is recorded and narrated classification chart be stored in the image description classification chart storage part 106.Have, the rectangle live image is recorded and narrated classification chart and image object, and to record and narrate the classification chart that uses among classification chart and the 1st embodiment the same again.
＜rest image (Still Picture) is recorded and narrated classification chart 〉
For designing rest image, purpose records and narrates classification chart with the signal characteristic of recording and narrating all rest images.Its main purpose is to retrieve the image with similar signal mode from digital picture documents such as digital photos document.
The signal characteristic that obtains from rest image can be divided into 7 groups, that is, 1) the look distribution, 2) the look configuration, 3) color temperature, 4) lighting condition correction look, 5) edge, 6) texture, 7) shape.The video features amount that belongs to each group is determined as follows respectively.
1) advantage look/scalable look/look structure
2) look configuration
3) color temperature
4) lighting condition correction look
5) edge histogram
6) similar texture/texture is browsed
7) contour shape/region shape
For similar video features amount in look distribution, texture and the shape group, all use improperly in the lump, wish to select as required one or more uses.For the content and the using method of video features amount, because of with the 1st embodiment in narrate the same, so omitted (for example, with reference to table 3 and table 4) here.
Figure 17 is the figure that stagnant zone that expression is write as with XML is recorded and narrated the example of classification chart.Recording and narrating classification chart can carry out with language arbitrarily, the record (or more record) that comprises arbitrarily to be comprised.Have, in Figure 17, the title with the name attribute representation in the element element is arbitrarily again, but wishes it is the title that shows the feature of the record symbol of representing with type.
Compare with the 1st embodiment, recorded and narrated the decreased number of classification chart, therefore, can provide system configuration simple image description system.
(the 5th embodiment)
The 5th embodiment of the present invention and the 1st embodiment difference shown in Figure 1 are: in the 5th embodiment, the live image of the stagnant zone of recording and narrating rest image being recorded and narrated classification chart and recording and narrating live image is recorded and narrated classification chart and is stored in the image description classification chart storage part 106.Have, it is the same with the classification chart of above-mentioned the 4th embodiment record that stagnant zone is recorded and narrated classification chart again.
＜activity image description classification chart 〉
Signal characteristic with the record live image is a purpose design activity image description classification chart.The signal characteristic that obtains from live image can be divided into 5 groups, that is, (1) is to the time series data of the characteristic quantity of rectangular image, (2) representative is included in the characteristic quantity of all frames in the live image, (3) mobile activity, (4) movement of objects information, (5) change of shape.The video features amount that belongs to each group can be determined as follows.
1) video time series
2) GofGop look
3) mobile activity
4) movement locus/parameter motion
5) change of shape
Have again, for the content and the using method of video features amount, because of with the 1st embodiment in narrate the same, so omitted (for example, with reference to table 6) here.
Figure 18 is the figure that live image that expression is write as with XML is recorded and narrated the example of classification chart.Recording and narrating classification chart can carry out with language arbitrarily, the record (or more record) that comprises arbitrarily to be comprised.Have, in Figure 18, the title with the name attribute representation in the element element is arbitrarily again, but wishes it is the title that shows the feature of the record symbol of representing with type.
Compare with the 1st embodiment, recorded and narrated the decreased number of classification chart, therefore, can provide system configuration simple image description system.
As described above in detail,, when utilizing input part to specify image, take out the image description classification chart corresponding, and show the suitable video features amount that to extract out with image type if according to the present invention.Therefore, select significant video features amount easily, and, the video features amount that can show specify image really can be extracted out.Therefore, can improve the efficient and the precision of image retrieval.
In addition, by classification chart is recorded and narrated in each image type definition, the characteristic quantity that should support can be extracted out; And the kind of recording and narrating instrument is reduced to necessary bottom line, and can provide system configuration simple image description system.
And then by resembling record file and the contrast of image description classification chart that generates above, whether suitable, and can further improve image retrieval efficient and precision if can verify the narration way of the record file of image.
Claims (34)
1. an image description system is characterized in that having: storage part, the record classification chart that storage defines each image type; Control part can be from the characteristic quantity of described specify image extraction in case just specify with reference to the record classification chart corresponding with type this specify image described storage portion stores after image is specified.
2. the image description system of claim 1 record is characterized in that: and then have the file of record generating unit, from described specify image, extract out and the relevant data of described characteristic specified amount, generate the record file of described specify image.
3. the image description system of claim 1 record, it is characterized in that: described control part can make described characteristic specified amount show on described display part selectively.
4. the image description system of claim 3 record, it is characterized in that: and then have the file of record generating unit, from described specify image, extract the data relevant in the described characteristic specified amount out, and generate the record file of described specify image with selecteed characteristic quantity.
5. the image description systems of claim 2 or 4 records is characterized in that: and then have and record and narrate file verification portion, uses the record classification chart corresponding with the type of described specify image to verify record file by described record file generating unit generation.
6. the image description system of claim 1 record is characterized in that: described storage portion stores records and narrates that the rectangular image of rectangular image is recorded and narrated classification chart, the arbitrarily shaped image of recording and narrating the image of arbitrary shape is recorded and narrated classification chart, record and narrate as the rectangle live image of the live image of the set of rectangular frame and record and narrate classification chart and record and narrate as the image object of the object of the arbitrary shape in the live image of the set of rectangular frame and record and narrate at least one classification chart in the classification chart.
7. the image description system of claim 6 record is characterized in that: described rectangular image is recorded and narrated classification chart and is had at least one characteristic quantity in a plurality of characteristic quantities that comprise look distribution, look configuration, color temperature, lighting condition correction look, edge distribution and texture at least.
8. the image description system of claim 7 record is characterized in that: described at least one characteristic quantity is made of at least 1 selectable record symbol respectively,
Described look distribution characteristics amount is made of a plurality of record symbols that comprise advantage look, scalable look and look structure at least, wherein has at least 1 to be selectable,
Described texture characteristic amount constitutes by comprising a plurality of records symbols that similar texture and texture browse at least, wherein has at least 1 to be selectable.
9. the image description system of claim 6 record is characterized in that: described arbitrarily shaped image is recorded and narrated classification chart and is had at least one characteristic quantity in a plurality of characteristic quantities that comprise look distribution, look configuration, color temperature, lighting condition correction look, edge distribution, texture and shape at least.
10. the image description system of claim 9 record is characterized in that: described at least one characteristic quantity is made of at least 1 selectable record symbol respectively,
Described shape facility amount is made of a plurality of record symbols that comprise contour shape and region shape at least, wherein has at least 1 to be selectable.
11. the image description system of claim 6 record is characterized in that: described rectangle live image is recorded and narrated at least one the characteristic quantity in a plurality of characteristic quantities that classification chart comprises the time series data, representative feature and the mobile activity that comprise described rectangular frame at least.
12. the image description system of claim 11 record is characterized in that: described at least one characteristic quantity is made of at least 1 selectable record symbol respectively,
Described time series data have at least one the characteristic quantity in a plurality of characteristic quantities that comprise look distribution, look configuration, color temperature, lighting condition correction look, edge distribution and texture at least, and each characteristic quantity is made of at least 1 selectable record symbol,
Described look distribution characteristics amount is made of a plurality of record symbols that comprise advantage look, scalable look and look structure at least, wherein has at least 1 to be selectable,
Described texture is by comprising that at least a plurality of record symbols that similar texture and texture are browsed constitute, and wherein have at least 1 to be selectable.
13. the image description system of claim 11 record, it is characterized in that: described representative feature has at least one the characteristic quantity in a plurality of characteristic quantities that comprise look distribution, look configuration, color temperature, lighting condition correction look, edge distribution and texture at least, each characteristic quantity is made of at least 1 selectable record symbol
Described look distribution characteristics amount is made of a plurality of record symbols that comprise advantage look, scalable look and look structure at least, wherein has at least 1 to be selectable,
Described texture constitutes by comprising a plurality of records symbols that similar texture and texture browse at least, wherein has at least 1 to be selectable.
14. the image description system of claim 6 record is characterized in that: described image object is recorded and narrated at least one the characteristic quantity in a plurality of characteristic quantities that classification chart has the time series data, representative feature, mobile activity, movement of objects and the change of shape that comprise described rectangular frame at least.
15. the image description system of claim 14 record, it is characterized in that: described at least one characteristic quantity is made of at least 1 selectable record symbol respectively, described movement of objects is made of a plurality of record symbols that comprise the motion of movement locus and parameter at least, wherein has at least 1 to be selectable.
16. the image description system of claim 14 record, it is characterized in that: described time series data have at least one the characteristic quantity in a plurality of characteristic quantities that comprise look distribution, look configuration, color temperature, lighting condition correction look, edge distribution and texture at least, each characteristic quantity is made of at least 1 selectable record symbol
Described look distribution characteristics amount is made of a plurality of record symbols that comprise advantage look, scalable look and look structure at least, wherein has at least 1 to be selectable,
Described texture constitutes by comprising a plurality of records symbols that similar texture and texture browse at least, wherein has at least 1 to be selectable.
17. the image description system of claim 14 record, it is characterized in that: described representative feature has at least one the characteristic quantity in a plurality of characteristic quantities that comprise look distribution, look configuration, color temperature, lighting condition correction look, edge distribution and texture at least, each characteristic quantity is made of at least 1 selectable record symbol
Described look distribution characteristics amount is made of a plurality of record symbols that comprise advantage look, scalable look and look structure at least, wherein has at least 1 to be selectable,
Described texture constitutes by comprising a plurality of records symbols that similar texture and texture browse at least, wherein has at least 1 to be selectable.
18. an image description method is characterized in that having:
Storage is to the step of the record classification chart of each image type definition;
After in case image is specified just the retrieval record classification chart corresponding with the type of this specify image specify can be from the step of the characteristic quantity of specify image extraction again;
The step that shows the characteristic quantity that to extract out from described specify image selectively.
19. the image description method of claim 18 record is characterized in that: and then have the step of from the characteristic quantity of described demonstration, selecting desired characteristic quantity; And from described specify image, extract the step that regeneration behind the characteristic quantity is recorded and narrated file out according to desired characteristic quantity.
20. the image description method of claim 19 record is characterized in that: and then have the step of using the record classification chart corresponding to verify the record file of described generation with the type of described specify image.
21. executable software product of computer, it is characterized in that, has following function: in case after image is specified, just the storage part of the record classification chart of each image type definition is retrieved the record classification chart corresponding with the type of this specify image by storage;
Specifying according to the record classification chart of retrieval can be from the characteristic quantity of described specify image extraction; And
Showing selectively can be from the characteristic quantity of specify image extraction.
22. the image description program of claim 21 record, it is characterized in that: and then have when from the characteristic quantity of described demonstration, having selected desired characteristic quantity, just from described specify image, extract the function that regeneration behind the characteristic quantity is recorded and narrated file out according to desired characteristic quantity.
23. the software product of claim 22 record is characterized in that: and then have following function: use the record classification chart corresponding to verify the record file of described generation with the type of described specify image.
24. the image description system of claim 1 record is characterized in that:
Described storage portion stores is recorded and narrated the rest image of the characteristic quantity of rest image and is recorded and narrated classification chart, records and narrates as the rectangle live image of the live image of the set of rectangular frame and record and narrate classification chart and record and narrate as the image object of the object of the arbitrary shape in the live image of the set of rectangular frame and record and narrate at least one classification chart in the classification chart.
25. the image description system of claim 24 record is characterized in that: described rest image is recorded and narrated at least one the characteristic quantity in a plurality of characteristic quantities that classification chart has the look distribution characteristics amount that comprises at least, look configuration feature amount, color temperature characteristic quantity, lighting condition correction color characteristic amount, edge distribution characteristic quantity and texture characteristic amount.
26. the image description system of claim 25 record is characterized in that: described at least one characteristic quantity is made of at least 1 selectable record symbol respectively,
Described look distribution characteristics amount is made of a plurality of record symbols that comprise advantage look, scalable look and look structure at least, wherein has at least 1 to be selectable,
Described look configuration feature amount is made of the record symbol that comprises the look configuration at least, wherein has at least 1 to be selectable,
Described color temperature characteristic quantity is made of the record symbol that comprises color temperature at least, wherein has at least 1 to be selectable,
Described lighting condition correction color characteristic amount is made of the record symbol that comprises lighting condition correction look at least, wherein has at least 1 to be selectable,
Described edge distribution characteristic quantity is made of the record symbol that comprises edge histogram at least, wherein has at least 1 to be selectable,
Described texture characteristic amount constitutes by comprising a plurality of records symbols that similar texture and texture browse at least, wherein has at least 1 to be selectable.
27. the image description system of claim 25 record is characterized in that: described rest image is recorded and narrated classification chart and then is comprised the shape facility amount,
Described shape facility amount is made of a plurality of record symbols that comprise contour shape and region shape at least, wherein has at least 1 to be selectable.
28. the image description system of claim 1 record is characterized in that: the rest image record classification chart of the characteristic quantity of described storage portion stores record rest image and the live image of record live image are recorded and narrated at least one classification chart in the classification chart.
29. the image description system of claim 28 record, it is characterized in that: described live image is recorded and narrated at least one the characteristic quantity in a plurality of characteristic quantities of mobile activity that classification chart has the representative feature of the time series data that comprise described activity diagram picture frame at least, described live image, described live image, and each characteristic quantity comprises at least 1 selectable record symbol.
30. the image description system of claim 29 record is characterized in that: described live image is recorded and narrated classification chart and then is comprised the mobile record of described live image and the change of shape record of described live image.
31. a record classification chart that uses at image description system, this image description system reference record classification chart corresponding with the type of the image of appointment specified and can be it is characterized in that from the characteristic quantity of specify image extraction:
Have at least one the characteristic quantity in a plurality of characteristic quantities of the look distribution characteristics amount that comprises at least, look configuration feature amount, color temperature characteristic quantity, lighting condition correction color characteristic amount, edge distribution characteristic quantity and texture characteristic amount,
Described look distribution characteristics amount is made of a plurality of record symbols that comprise advantage look, scalable look and look structure at least, wherein has at least 1 to be selectable,
Described look configuration feature amount is made of the record symbol that comprises the look configuration at least, wherein has at least 1 to be selectable,
Described color temperature characteristic quantity is made of the record symbol that comprises color temperature at least, wherein has at least 1 to be selectable,
Described lighting condition correction color characteristic amount is made of the record symbol that comprises lighting condition correction look at least, wherein has at least 1 to be selectable,
Described edge distribution characteristic quantity is made of the record symbol that comprises edge histogram at least, wherein has at least 1 to be selectable,
Described texture characteristic amount constitutes by comprising a plurality of records symbols that similar texture and texture browse at least, wherein has at least 1 to be selectable.
32. the record classification chart of claim 31 record is characterized in that: described record classification chart and then comprise the shape facility amount,
Described shape facility amount is made of a plurality of record symbols that comprise contour shape and region shape at least, wherein has at least 1 to be selectable.
33. the image description system of claim 28 record is characterized in that:
Record and narrate the image description system that classification chart is used for specifying with reference to the record classification chart corresponding with the type of specify image the characteristic quantity that can extract out from specify image,
Have at least one the characteristic quantity in a plurality of characteristic quantities of mobile activity of the representative feature of the time series data that comprise the activity diagram picture frame at least, described live image and described live image, each characteristic quantity comprises at least 1 selectable record symbol.
34. the image description system of claim 29 record is characterized in that: described live image is recorded and narrated classification chart and then is comprised the mobile record of described live image and the change of shape record of described live image.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP355268/2002 | 2002-12-06 | ||
| JP2002355268 | 2002-12-06 | ||
| JP66399/2003 | 2003-03-12 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1692646A true CN1692646A (en) | 2005-11-02 |
| CN100454997C CN100454997C (en) | 2009-01-21 |
Family
ID=35347038
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB200380100383XA Expired - Fee Related CN100454997C (en) | 2002-12-06 | 2003-12-05 | Image description system and method thereof |
Country Status (2)
| Country | Link |
|---|---|
| JP (1) | JP4692784B2 (en) |
| CN (1) | CN100454997C (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112241258A (en) * | 2020-10-15 | 2021-01-19 | 科大国创软件股份有限公司 | Shape graph generation method and device for volatile data structure program verification |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110446088A (en) * | 2018-05-03 | 2019-11-12 | 尼尔森网联媒介数据服务有限公司 | Media information recognition methods, exchange method and the equipment of TV |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3780623B2 (en) * | 1997-05-16 | 2006-05-31 | 株式会社日立製作所 | Video description method |
| JP3619426B2 (en) * | 1999-06-10 | 2005-02-09 | 松下電器産業株式会社 | Optical disc playback apparatus and optical disc playback method |
| JP2001266052A (en) * | 2000-03-15 | 2001-09-28 | Ricoh Co Ltd | Method and device for structuring information and generating application |
| JP2002007432A (en) * | 2000-06-23 | 2002-01-11 | Ntt Docomo Inc | Information retrieval system |
| JP3784289B2 (en) * | 2000-09-12 | 2006-06-07 | 松下電器産業株式会社 | Media editing method and apparatus |
| US7062084B2 (en) * | 2000-12-01 | 2006-06-13 | Sharp Laboratories Of America, Inc. | Method for image description using color and local spatial information |
-
2003
- 2003-12-05 CN CNB200380100383XA patent/CN100454997C/en not_active Expired - Fee Related
-
2008
- 2008-10-27 JP JP2008275304A patent/JP4692784B2/en not_active Expired - Fee Related
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112241258A (en) * | 2020-10-15 | 2021-01-19 | 科大国创软件股份有限公司 | Shape graph generation method and device for volatile data structure program verification |
Also Published As
| Publication number | Publication date |
|---|---|
| JP4692784B2 (en) | 2011-06-01 |
| JP2009048657A (en) | 2009-03-05 |
| CN100454997C (en) | 2009-01-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1195379C (en) | Image content providing reproducting method, system, equipment and its storage media | |
| CN1260665C (en) | Electronic apparatus for managing changeable storage medium, method thereof and storage medium | |
| CN1908936A (en) | Image processing apparatus and method, and program | |
| CN1959673A (en) | Information-processing apparatus, content reproduction apparatus, information-processing method, event-log creation method and computer programs | |
| CN101047827A (en) | Electronic apparatus, method and system for collecting broadcast program information, and storage medium | |
| CN101047826A (en) | Electronic apparatus, information browsing method thereof, and storage medium | |
| CN1610910A (en) | Contents consignment sale system and method for networking broadcasting | |
| CN1757020A (en) | Information processing device, information processing method, and computer program | |
| CN1698353A (en) | Information processing apparatus, information processing method, and computer program | |
| CN1596445A (en) | Authoring device and authoring method | |
| CN1822643A (en) | Information processing apparatus, information processing method, and information processing system | |
| CN1855284A (en) | Reproducing device, reproducing control method and program | |
| CN1777950A (en) | Reproduction apparatus, program, integrated circuit | |
| CN1728144A (en) | Content providing system, content providing server, information processing apparatus, and computer program | |
| CN1268710A (en) | Device and method for database searching | |
| CN1932965A (en) | Electronic apparatus, display control method for the electronic apparatus, graphical user interface, and display control program | |
| CN101046818A (en) | Information processing apparatus, method, and program product | |
| CN1790376A (en) | Information processing apparatus , input device and method, program and information processing system | |
| CN1817038A (en) | Contents storage system, home server apparatus, information supply apparatus, integrated circuit, and program | |
| CN1756337A (en) | Method, apparatus and program for recording and playing back content data | |
| CN1862698A (en) | Information processing system, information generating apparatus and method | |
| CN101036142A (en) | Organizing content | |
| CN1914898A (en) | Preference information managing apparatus and method | |
| CN101069181A (en) | Storage device and recording medium | |
| CN1726551A (en) | Recording/reproduction method and recording/reproduction device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20090121 Termination date: 20151205 |
|
| EXPY | Termination of patent right or utility model |