WO2002065339A1 - Dispositif de tri/reorganisation de donnees multimedia et procede de tri/reorganisation - Google Patents

Dispositif de tri/reorganisation de donnees multimedia et procede de tri/reorganisation Download PDF

Info

Publication number
WO2002065339A1
WO2002065339A1 PCT/JP2001/000049 JP0100049W WO02065339A1 WO 2002065339 A1 WO2002065339 A1 WO 2002065339A1 JP 0100049 W JP0100049 W JP 0100049W WO 02065339 A1 WO02065339 A1 WO 02065339A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
classification
information set
unit
icons
Prior art date
Application number
PCT/JP2001/000049
Other languages
English (en)
Japanese (ja)
Inventor
Daiki Masumoto
Susumu Endo
Yusuke Uehara
Naoki Sashida
Shuichi Shiitani
Original Assignee
Fujitsu Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujitsu Limited filed Critical Fujitsu Limited
Priority to PCT/JP2001/000049 priority Critical patent/WO2002065339A1/fr
Priority to JP2002565195A priority patent/JP4295510B2/ja
Publication of WO2002065339A1 publication Critical patent/WO2002065339A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/45—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data

Definitions

  • the present invention clarifies the classification and arrangement of multimedia information groups that are mixed not only with text information but also with various media information such as image information and audio information.
  • Multimedia information classification and placement system that can be performed efficiently and flexibly
  • the following are known as a conventional information retrieval system and information classification and arrangement processing technology in the retrieval system.
  • text key search systems are known.
  • One or more pieces of text information representing the contents are appropriately allocated to the multimedia data to be searched, and text such as a word is specified as a search key in the search, and the search key is used as the search key.
  • the same text information is It is known how to detect attached multimedia data. If the media information to be searched is text information, the contents of the text information will be searched using a text key, and the text key search will be performed. Can often be performed properly.
  • this text key search system there is a sort and a list display of data hit by a search.
  • text full-text search systems are known. This is a particularly effective search system when the multimedia data to be searched is text information.
  • Several methods have been proposed. For example, as preprocessing, all words representing the characteristics of the contents are automatically extracted from the entire text, and a search key file is generated and assigned. In the search, text such as a word is specified as a search key, and text information in which the same text as the search key exists in the search key file is detected.
  • the search method using the first text as a key, for media information other than text information, for example, image information, unless an appropriate text key is assigned to each image information, text It is difficult to perform an appropriate search with a search using.
  • examples of the classification and arrangement of information in this text full-text search system include a system that sorts and displays a list of the hits in the search.
  • the classification and arrangement of information is also one that sorts and displays a list of the hit data in the search. No.
  • acoustic search technology based on acoustic pattern matching is being studied as a technology for searching for acoustic information.
  • acoustic information retrieval system a system in which a user specifies acoustic data (e.g., octing) to be the key, the system searches for acoustic data similar to the acoustic signal, and presents information such as a song name to the user. No.
  • Video retrieval methods include segmenting the video in advance, recognizing the contents of each segment using techniques such as motion recognition algorithms, voice recognition, and telop recognition, and labeling according to the information content. and occurs at a live coverage of the direction of movement of the object schedule Tutsi home run and shot as c this video information retrieval system in which the method to search by creating a search key has been studied by, goals, etc., such as baseball and soccer Systems are being developed to search for highlight scenes and for specific actions such as smash or pollet in tenis.
  • the first conventional text key search system it is difficult to properly perform image information and audio information. In other words, even if multiple keywords are assigned to one piece of image information, it is difficult to accurately and flexibly express the features of the image in text. Performing appropriate and flexible image retrieval is difficult. Although it is possible to collect a large number of images and display them in order, there is a problem that it becomes difficult to find the desired image from the large number of images. Even in the case of the conventional second full-text search system, it is difficult to properly perform image information and audio information as in the case of the first text key search system. In other words, image information and audio information do not originally have text information, and extraction of text information cannot be expected.
  • the searcher must prepare and specify the partial image of the image to be searched. In some cases, it is not clear what kind of image the searcher most wants to have, and it is not always possible to prepare partial images to be used for search, making search difficult. Furthermore, the searcher may not have a clear picture of the image he or she wants to search. In other words, you can specify the general content of the image you want to search, but you may not be able to specify the image itself, even if it is partial. In addition, searchers sometimes want to search by trial and error for images that are vaguely available. In such cases, matching of partial images alone lacks flexibility and is insufficient.
  • the conventional information retrieval technology has a problem in that it is "difficult to specify an appropriate key to narrow down the information".
  • the search target is composed of information units containing images (images themselves, mixed images / text documents, etc.)
  • the user is able to view the large amount of information presented because of its excellent browsability for humans.
  • the following problems arise.
  • the first problem is that the search device operates on the server, and when displaying images on the client via the network, a large amount of images must be transferred, and the searcher has to wait for a long time.
  • the problem is that it will take over.
  • the second problem is that when an information unit includes information other than an image, such as a mixed image / text document, the image does not always accurately represent the content of the information unit. It is. In other words, the image included in the information unit is often used simply for decoration, etc. In such a case, in the browsing search based on the image, the searcher can accurately set a desired information set. May not be found.
  • the third problem is that if the search target is composed of information units that do not contain images (text-only document data, audio data only, video data only), the list is displayed even if the information is presented as it is.
  • the search efficiency of the user is not improved due to lack of efficiency.
  • the present invention reduces the amount of data to be transferred when displaying information on a client via a network, and at the same time, ensures that the information contents can be easily viewed and understood by a searcher. It is an object of the present invention to provide a multimedia information classification and arrangement device and a classification and arrangement method.
  • the present invention efficiently and flexibly searches multimedia information such as text information, image information, and audio information by using various features of multimedia information, and searches the search results.
  • Visual and divided It is an object of the present invention to provide a classifying and arranging apparatus and a classifying and arranging method which can classify, arrange, and display the multimedia information so that it can be easily displayed. Disclosure of the invention
  • the multimedia information classification and arrangement apparatus of the present invention groups homogeneous and heterogeneous media information related to the same object from a set of media information including image information, text information, and audio information.
  • the information set is referred to as "information set”, and the information set is used as a processing unit. Then, an icon for visually displaying the content is assigned to each piece of media information constituting the information set, and an “information set icon” for visually expressing the content of the information set by synthesizing the icons is provided. Use.
  • an "information set” as a unit of information processing and the use of an "information set icon” in the present invention are clearly different from the conventional information search technology.
  • the “information set” of the present application is not one in which each piece of media information is synthesized and integrated as in the conventional multimedia information, but is related by a common concept. It is distinctly different from media information.
  • the information processing unit is an information set, and the corresponding icon is assigned to each piece of media information such as text, image, and sound that constitutes the information set, and these icons are combined. It is an icon corresponding to the information set as the information processing unit.
  • the multimedia information classification and arrangement device includes: an information set acquisition unit that acquires media information in units of an information set; and an eye that visually displays the content of each piece of media information constituting the information set. And an information visualization unit that generates an information set icon that visually expresses the content of the information set by combining the icons and a feature that extracts a component of a feature amount from each piece of media information of each information set.
  • a quantity extraction unit and an attribute selected from attributes of feature quantities extracted from each media information included in each acquired information set are assigned as axes of the space where information set icons are arranged, and one or more axes are provided.
  • An axis setting unit for setting the classified arrangement space, and a feature amount attribute of the media information included in each information set and a component of the feature amount.
  • a classification arrangement section for classifying and arranging information set icons in a class arrangement space, and an information display section for displaying the classification arrangement space and the information set icons.
  • the amount of data to be transferred can be reduced by generating and using an information set icon that has a high compression efficiency and requires a small amount of data transfer as data used for displaying information. It is possible to reduce the waiting time until the information is displayed by the searcher, and also to ensure that the information content for the searcher can be easily viewed and understood.
  • the Search efficiency can be improved.
  • the processing by the multimedia information classification and placement device is divided into two phases, an information set icon generation phase and an information search phase using the information set, and the information set icon generation phase is executed in advance as preprocessing.
  • information of the information set used for search in the information search phase You can also create and prepare set icons at once. According to this configuration, in the information set search phase, it is not necessary to generate an information set icon each time from the information set obtained as a result of the search, and the information set icon corresponding to the searched information set is classified and arranged in the space. It is only necessary to classify and dispose them, and the processing speed can be increased.
  • another configuration of the multimedia information classification and placement device of the present invention is not to compose an icon corresponding to each piece of media information constituting an information set to create an information set icon,
  • the icons corresponding to the media information are grouped, and the icons are classified and arranged in the classified arrangement space according to the feature amount, and the icons are displayed at the arrangement positions while being temporally switched according to the display order at the arrangement positions.
  • an information set acquisition unit that acquires media information in units of the information set, and an icon that visually displays the content of each piece of media information constituting the information set in units of the information set
  • An information visualization unit that allocates the information, a feature amount extraction unit that extracts the component of the feature amount from each piece of media information in each information set, and an attribute of the feature amount extracted from each piece of media information included in each acquired information set
  • the attribute selected from the above is assigned as the axis of the space where the icons are arranged, the axis setting unit that sets the classification arrangement space with one or more axes, and the attributes of the media information features of each information set and their attributes.
  • a classification arrangement unit that classifies and arranges icons for each piece of media information included in the information set in the classification arrangement space based on the component of the feature amount;
  • An information display unit for displaying a space and the icons, wherein the information display unit determines the display order of the icons for each piece of media information included in the information set, and displays the icons at the arrangement positions while temporally switching the icons. It is characterized by having.
  • the information set icon generation phase and the information search phase using the information set are divided into two phases, and the information set icon generation phase is executed in advance as pre-processing, and the search in the information search phase is performed.
  • the information set icons of the information set to be used can be generated and prepared collectively.
  • the classification and placement unit determines a plurality of components based on each of the plurality of components of the feature amount. If the information set icons or icons to be switched and displayed in accordance with the display order are classified and arranged at individual positions, even if a plurality of components are present in the feature amount, they can be appropriately classified and arranged in the feature space.
  • the icons and information set icons for each medium can be made easy to grasp and reflect the contents of the information and suitable for searching.
  • the axis setting unit resets the assignment of the attribute of the feature amount to each axis of the classification and placement space axis, and reconfigures the classification and placement space having one or more axes.
  • An axis resetting function for setting wherein the feature amount extracting unit extracts a component of the feature amount in accordance with the axis resetting by the axis setting unit.
  • the classification and arrangement unit classifies and arranges each information set icon in the classification and arrangement space in accordance with the component of the extracted feature amount, and the information display unit provides a viewpoint to the re-set classification and arrangement space. It is preferable to display the corresponding media information.
  • the searcher can view the search results executed by the searcher and interactively re-search to narrow down the information flexibly, or use the re-set classification arrangement spatial axis to search one after another from different perspectives. You can continue. In other words, by retrieving the search results, it is possible to perform a trial and error to specify another feature value and reset the axis of the classification arrangement space, and to set the information set icon using the classification arrangement space using the reset axis. It can be rearranged and redisplayed.
  • the axis setting unit resets the assignment of the attribute of the feature amount to each axis of the classification and placement space, and resets the already displayed classification and placement space
  • the classification and placement unit reclassifies and arranges each information set icon in the reconfigured classification and placement space
  • the information display unit displays a state in which the information set icons are reclassified and arranged. It is preferable to display the displayed media information by moving the displayed media information at predetermined intervals from a position displayed before the classification and arrangement to a position displayed after the re-classification and arrangement.
  • the classification and arrangement unit has a function of fixing an information set icon selected by the user to a specific position in the classification and arrangement space specified by the user, and a classification whose axis is reset. Placement space
  • the information set icons are reclassified and arranged, the information set icon selected by the user is fixed at the specific position, and only the other information set icons are reclassified and arranged according to the classified arrangement space. It is preferable to have a function of
  • the display position of the information set icon of interest is fixed at a specific position, so that the information set icon of interest can be easily found, and the axis of the information set icon of interest can be set. Since information set icons similar in the feature amount are arranged in the vicinity, it is easy to grasp the relationship between the information sets. In addition, it is possible to execute trial and error re-classification display while paying attention to the specific information set.
  • the multimedia information classification and arrangement device of the present invention records the processing program of the multimedia information classification and arrangement device of the present invention on a computer-readable recording medium and provides the same.
  • the multimedia information classification and arrangement device of the present invention can be realized.
  • FIG. 1 (a) is a diagram simply showing the concept of the "information set" of the present invention
  • FIG. 1 (b) is a diagram showing an example of multimedia information in the prior art
  • FIG. 2 is a diagram showing an example of an information set icon and an example of a procedure for generating the information set icon.
  • FIG. 3 is a diagram showing a device configuration example of a multimedia information classification and placement device according to the first embodiment of the present invention.
  • FIG. 4 is a flowchart illustrating an example of a processing flow of the multimedia information classification and arrangement device according to the first embodiment of the present invention.
  • FIG. 5 is a diagram showing an example in which six information set icons collected with the keyword “musician” are classified and arranged in a classification arrangement space.
  • FIG. 6 is a diagram showing a web page of the information set corresponding to the selected information set icon.
  • FIG. 5 is a flowchart showing an example of a processing flow of the multimedia information classification and arrangement device according to the second embodiment of the present invention.
  • FIG. 8 is a diagram showing a device configuration example of a multimedia information classification and placement device according to a third embodiment of the present invention.
  • FIG. 9 is a diagram showing a device configuration example of a multimedia information classification and placement device according to a fourth embodiment of the present invention.
  • FIG. 10 is a simplified illustration of a device configuration of a multimedia information classification and placement device according to a fifth embodiment of the present invention.
  • FIG. 11 is a diagram showing a display example after the re-sorting and arrangement displayed by the information display unit 60 according to the fifth embodiment of the present invention.
  • FIG. 12 is a diagram showing a state where a specific information set icon is selected and designated using the arrangement position fixed designation unit 52 before the re-sorting and arrangement.
  • FIG. 13 is a diagram showing a state in which the information set icon selected and designated in FIG. 12 is fixed at a specific position (the center of the screen) and other information set icons are re-classified and arranged.
  • FIG. 14 is a diagram showing an example in which the multimedia information classification and arrangement device according to the sixth embodiment of the present invention is constructed in a client-server configuration.
  • FIG. 15 is a diagram showing another example in which the multimedia information classification and arrangement device according to the sixth embodiment of the present invention is constructed in a client-server configuration.
  • FIG. 16 is a diagram showing an example of a recording medium on which a program in which processing contents of the multimedia information classification and arrangement device according to the seventh embodiment of the present invention are processed is recorded.
  • BEST MODE FOR CARRYING OUT THE INVENTION embodiments of a multimedia information classification and arrangement device and a classification and arrangement method of the present invention will be described with reference to the drawings.
  • the multimedia information classification and arrangement device is a device for duplicating homogeneous and heterogeneous media information related to the same object from a set of media information including image information, text information, and audio information.
  • Information set "and use that information set as a processing unit. Then, an icon for visually displaying the content is assigned to each piece of media information constituting the information set, and an “information set icon” is used to combine the icons to visually express the content of the information set. .
  • the multimedia information classification and arrangement device of the present invention uses an information set as a unit of information processing.
  • This information set is a processing unit that groups homogeneous and heterogeneous media information related to the same object from a set of media information including image information, text information, and audio information.
  • Figure 1 is a diagram that briefly illustrates the concept of the "information set" of the present invention.
  • one information set 1 is a group of a plurality of mutually similar or similar media information that are mutually related.
  • four pieces of the same or different types of media information namely, the image information 1a, the image information 1b, the audio information 1c, and the keyword information 1d are associated as a group.
  • These four types of similar or different media information are, for example, It is gathered under the concept of related information called "sonal computer.”
  • FIG. 1 also shows how the information set of the present invention is generated from multimedia information and the like in the prior art.
  • Reference numeral 500 denotes multimedia information in the related art, in which a plurality of pieces of media information are integrated and combined. For example, HTML file data.
  • the information set acquisition unit 10 is for acquiring the information set of the present application, and for example, can acquire conventional multimedia information and generate an information set.
  • the procedure for generating the information set is as follows.
  • the information set obtaining unit 10 extracts and collects an image portion, and further collects other media information such as text information, other image information, moving image information, and audio information related to the image. For example, if it is text information, it is obtained by analyzing an HTML file. As an example of the analysis method, text near the image is assumed to be related to the image, and the number of line breaks between the two is used as a measure of the positional proximity between the image and the text. I do. It is also possible to infer the degree of relevance between images and text from the HTML tag structure.
  • the degree of relevance between them is low.
  • the degree of relevance is low when there are tags such as ⁇ P> and ⁇ H1> ⁇ HR> that indicate breaks in the sentence. it can.
  • the ALT attribute which is a tag for displaying an image and is a text used as a substitute for the image attached to IMG>, has a high relevance to the image, it can be assigned a high degree of relevance. In this way, the type and number of tags included between an image and text are used as a guide.
  • the information set acquisition unit 10 collects media information related to an image.
  • the information set acquisition unit 10 associates the collected media information as a set of media information related to the same target and collectively sets the information set. And
  • image information 1a was extracted, and information set 1 was generated by associating text information lb, voice information lc, and keyword information 1d related to image information la with a common concept.
  • these image information la, text information lb, voice information lc, and keyword information 1d are not associated with a common concept. Are managed separately.
  • the information set is not a composite of each media, but a set of media information related to the same object, so it is possible to flexibly add and delete media information. If new media information is added to the multimedia information in the conventional technology, a process for embedding the media information to be added to the multimedia information, embedding the information, and combining the multimedia information is necessary. The media information itself needs to be reworked. On the other hand, when the “information set” of the present application is used, when new media information to be added, for example, audio data on “Personal computer of Company F” is obtained, the audio data of The only thing that needs to be done is to increase the association. The information set of the present application does not combine and integrate each media information itself, but only associates them with a common concept, and does not perform any special processing on the contents of the media information itself. This is distinctly different from conventional monolithic multimedia information.
  • the information set is described.
  • text information for example, tf t
  • the video part is extracted as moving image information
  • the audio data is extracted as audio information
  • the surrounding text in which the movie data is embedded is extracted as text information and associated with each other.
  • information sets may be generated by extracting links from HTML data linked to other data and extracting image information, text information, audio information, and the like at the link destination.
  • image information and text information that exist in the form of a single file from the beginning, such as HTML documents
  • multiple pieces of text information can be associated with one piece of image information or vice versa.
  • a plurality of pieces of image information may be associated with each other.
  • a URL Universal Resource Locator
  • each piece of media information itself does not have to be edited and processed, while image information remains as image information and audio information remains as voice information, while various features of each piece of media information exist.
  • the quantity can be processed as a feature quantity of the entire information set.
  • various feature amounts of each piece of media information belonging to the information set are processed as feature amounts of the entire information set, and the information set icons are classified. The placement position is determined.
  • Prior art multimedia Deer processing mainly refers to the following two processes.
  • the first meaning of the conventional multimedia processing means that one device can handle a plurality of media such as image information, text information, and audio information. In this case, a plurality of media can be obtained with one device, but the processing unit itself is each media information, and each media information such as image information, text information, and audio information is individually processed.
  • the second meaning of the conventional multimedia processing means that the processing unit itself is multimedia data in which a plurality of media are integrated. In this case, for example, image information or audio information is embedded in the text information or a link is established, and the data itself is integrated.
  • the information set of the present invention uses, as a processing unit, an information set in which, among media information including image information, text information, and audio information, homogeneous and heterogeneous media information having a relevance to each other are collectively associated. This is different from the conventional multimedia information in the first meaning, in which each piece of media information is processed individually without being associated with each other. Further, in the present invention, although a set of a plurality of pieces of media information data is associated with each other, the data of the plurality of pieces of media themselves are not integrated and processed as in the above-described conventional multimedia information in the second meaning. Each media information is collected and acquired and maintained.
  • the new media information it is easy to make the existing information set belong to the relevant information set, or if there is no relevant existing information set, the new information set Can be generated. If the data of a plurality of media are integrated and added as in the conventional multimedia information of the second meaning, it is not easy to add new media information.
  • the “information set icon” used in the present invention will be described.
  • the point of using the "information set icon” is that Are distinctly different.
  • the information set icon is an information set whose information processing unit is an information set.
  • the corresponding icon is assigned to each piece of media information such as text, image, and sound that constitutes the information set, and these icons are combined to process the information processing unit. Is an icon corresponding to the information set as.
  • FIG. 2 is a diagram showing an example of an information set icon and an example of a procedure for generating the information set icon.
  • This information set 3 is an information processing unit related by a common concept of “Mozart” and includes three media information related to musician “Mozart”.
  • sound data 3a, image data 3b, and text data 3c are included. These pieces of media information are not integrated or combined, but are only associated as information set 3.
  • the feature amount of each piece of media information is extracted. It is assumed that the characteristic amount “music section, minor, piano” is obtained as a feature of the acoustic data 3a. Assume that a feature amount of “background: black, target: human face (area skin color)” is obtained as a feature of image data 3b. It is assumed that the text data 3c has a feature amount of “Moalto, Piano Concerto No. 20”. The method of extracting a feature from each piece of media information will be described later.
  • an icon corresponding to the feature amount is assigned to each piece of media information.
  • the assigned icon is modified according to the feature amount.
  • a piano-key icon is assigned to the acoustic data 3a.
  • image data 3b a flesh-color area indicating a human face Assign an icon that is clipped to the center and that has been subjected to image data compression such as DCT.
  • a text icon consisting of a text sentence “Mozart, Piano Concerto No. 20” is assigned to text data 3c.
  • an icon assigned to each piece of media information is synthesized to generate one information set icon.
  • the information set icon 3d was obtained for the information set 3.
  • the characteristic of the sound data is “only music section, minor, guitar”, and the content characteristic of the text data is “only music section, minor, guitar”.
  • the compression efficiency is high and the data transfer amount is small, and at the same time, the kind of media information included in the information set can be easily grasped at a glance. This makes it possible to ensure that the information content is easy to understand and easy to grasp.
  • the multimedia information classification and placement device includes an information set acquisition unit 10, a feature amount extraction unit 20, an information visualization unit 30, an axis setting unit 40, a classification and placement unit 50, and an information display unit 6. It has 0.
  • Reference numeral 70 denotes an accessible network such as the Internet, and the multimedia information source 80 is accessible via the network 70. It is assumed that various multimedia information such as information, text information, and voice information are stored, and in this example, an information set which is a set of mutually related media information is also stored.
  • the information set acquisition unit 10 includes an information set reading unit 11, an information set collection unit 12, an information set collection key input unit 13 and a storage medium 13 as a configuration.
  • the information set collection unit 12 is a part that collects information sets, and collects information sets existing in the multimedia information source 80 via the network 70.
  • the information set reading unit 11 can also accept input of an information set directly from a storage medium such as a CD-ROM.
  • the information set acquisition unit 10 is configured to appropriately include one or both of the information set reading unit 11 and the information set collection unit 12, so that the multimedia set can be transmitted via the network 70.
  • the function of collecting the information set existing in the information source 80 and the function of directly accepting the input of the information set from a storage medium such as a CD-ROM can be selectively retained.
  • the configuration example of the information set collection unit 12 described in the first embodiment includes a storage medium 13 such as an eighteenth disk, and an information set collection key input unit 14. It communicates with the multimedia information source 80 on the network 70 via the communication interface.
  • the storage medium 13 can be used when storing the collected information set.
  • the information set collection key input section 14 is a part that specifies the range to be collected using keywords when collecting information sets. It is. If there is a large number of information sets stored in the multimedia information source 80 on the network, collecting the information sets without specifying anything will greatly increase the data capacity to be collected. Therefore, when collecting information sets, if a keyword is input from the information set collection key input unit 14 to narrow down the range in advance, the information set collection unit 12 To collect.
  • the feature amount extraction unit 20 is a part that extracts a feature amount from each media information of the information set. Although not shown in FIG. 3, the feature amount extraction unit 20 has a function for extracting various feature amounts from each piece of media information. For example, for image information, there are a DCT transform coefficient feature, a wavelet transform coefficient feature, an HSI color histogram feature, and the like. By setting the DCT transform coefficient feature amount on the classification arrangement spatial axis, it is possible to classify and arrange information according to the features of the spatial frequency components of the image information.
  • the feature transformation coefficient feature By setting the feature transformation coefficient feature in the classification arrangement spatial axis, the feature of the image information, especially in the low frequency part of the spatial frequency, that is, the information based on the feature of the outline of the outline of the object reflected in the image Can be classified and arranged.
  • the wavelet transform is also a waveform-frequency transform like the DCT transform, but it can be transformed while keeping the position (time) information.
  • the HSI color histogram feature on the classification arrangement space axis, it is possible to classify and arrange information based on image color information. According to the HSI color histogram, it is possible to grasp the characteristics of the image such as how much the human skin region is included.
  • a feature quantity indicating the presence of a specific word for example, a feature quantity indicating the presence of a specific word, a feature quantity indicating the number of times a specific word is used, and the like can be given.
  • By measuring the relative importance of words and enumerating them feature vectors having the tf-idf value of the word as an element are extracted from each text.
  • an audio frequency feature using a short-time frequency analysis (Fourier transform CM F C C), an amplitude feature of a voice, a time shift feature of a voice, and the like can be given.
  • CM F C C short-time frequency analysis
  • a voice frequency it is possible to extract features based on the features of the speech frequency, that is, the level of the speech, the sound quality, and the like.
  • the voice frequency it is possible to represent the characteristics of voice information such as the type of a sounding body, the sound of an animal, the voice of a man, the voice of a woman, and if the accuracy is improved, the person of the voice.
  • the feature amount of audio information can be extracted according to the size of the audio by the amplitude feature amount and time shift feature amount of the voice.
  • the feature amount extraction unit 20 can add, update, or delete a function of extracting a feature amount from each piece of media information. If the feature extraction function is provided as a microprogram by a digital signal processor (DSP), the content can be easily rewritten and added as needed.
  • DSP digital signal processor
  • the information visualization unit 30 assigns icons for visually displaying the contents to each piece of media information constituting each information set stored in the multimedia information source 80, and synthesizes the icons to obtain information. It is preferable that the information set icon, which is a part for generating the information set icon that visually represents the contents of the set, be generated before the information classification and placement process is performed by the user.
  • the procedure for generating an information set icon by the information visualization unit 30 is as described above with reference to FIG.
  • the second method is to use the “rules for converting content features into icons” that have been manually described in advance. It provides a program that includes rules for converting content features to icons.
  • the feature value has multiple values (components) or the information content is divided into multiple parts, multiple feature vectors are recognized and another icon is prepared for each feature vector. By copying or copying the same icon, one information unit can be represented by multiple icons.
  • the axis setting unit 40 is a part for setting a classification arrangement space axis.
  • the axis setting unit 40 sets assignment of a feature amount extracted from each piece of media information to each axis of the classification arrangement space, and performs classification including one or more axes.
  • Set the placement space For example, three axes of X axis, Y axis, and Z axis are specified, and the space spanned by the XYZ axes is set as the classification arrangement space.
  • the display is performed such that the display display surface of the information display unit 60 described later is an XY plane and the screen depth is in the Z-axis direction.
  • the feature amount that can be set on the classification arrangement space axis may be a feature amount such as image information, text information, and audio information that can be extracted according to the media.
  • a characteristic amount indicating the existence of a specific word a characteristic amount indicating the number of times a specific word is used, and the like can be given.
  • a DCT transform coefficient feature there are a DCT transform coefficient feature, a ⁇ ⁇ transform coefficient feature, an HSI color histogram feature, and the like.
  • DCT transform coefficient feature amount on the classification arrangement spatial axis, it is possible to classify and arrange information according to the features of the spatial frequency components of the image information.
  • wavelet transform coefficient features By setting the wavelet transform coefficient features to the classification arrangement spatial axis, the features of the image information, especially in the low-frequency part of the spatial frequency, that is, the information classification based on the features of the outline of the large frame of the object appearing in the image , Arrangement is possible.
  • the wavelet transform is also a waveform / frequency transform like the DCT transform, but it can be done while keeping the position (time) information.
  • the HSI color histogram feature value By setting the HSI color histogram feature value on the classification arrangement space axis, it is possible to classify and arrange information based on the color information of the image. According to the HSI color histogram, it is possible to grasp the characteristics of the image such as how much the human skin region is included.
  • audio frequency features By setting the audio frequency features on the classification arrangement space axis, the audio information can be classified and arranged according to the characteristics of the audio frequency, that is, the level of the audio, the sound quality, and the like. According to the audio frequency, it is possible to express the characteristics of audio information such as the type of a sounding body, the sound of an animal, the voice of a man, the voice of a woman, and, if the accuracy is improved, the person of the voice.
  • the amplitude feature and the time-shift feature of the speech By setting the amplitude feature and the time-shift feature of the speech on the classification arrangement space axis, it is possible to classify and arrange the speech information according to the magnitude of the speech.
  • the feature setting to one spatial axis by the axis setting unit 40 can be performed by combining a plurality of feature values. Combinations of two or more features require conversion and adjustment of each other's units, but they can be combined by replacing them with scores such as scores. For example, for one spatial axis, if the specified color component of the HSI color histogram is included as a first feature quantity at a rate equal to or greater than the threshold, it is set to "1"; otherwise, it is set to "0". The second feature value is set to “1” if it exists as a feature value indicating the presence of a specific word in the text information, and is set to “0” if it does not exist, and the first feature value and the second feature value are set.
  • one feature can be assigned to multiple axes.
  • assigning a feature to two or more axes One of such methods is to consider the feature as a vector and assign a dimension with a large variance for each dimension of the vector.
  • the classification arrangement section 50 is a section that classifies and arranges the information set icons in the classification arrangement space based on the feature amounts extracted by the feature amount extraction section 20. For example, if the classification arrangement space is set by the XYZ three axes, the information set icons will be classified and arranged in the three-dimensional classification arrangement space.
  • the information display section 60 is a section for displaying an information set icon for the information set classified and arranged in the classification arrangement space by the classification arrangement section 50, and displays the information set icon at a position corresponding to the viewpoint with respect to the classification arrangement space. You.
  • the XY plane is If it is the front, and the Z-axis direction is the depth direction, the display surface is an XY plane and the screen depth is displayed as if it were in the Z-axis direction.
  • the processing of the multimedia information classifying and arranging apparatus according to the first embodiment of the present invention is roughly classified into the following information set acquisition processing (S10).
  • the re-axis setting processing is indispensable. Not something. Also, in the above processing flow, it is assumed that the information set exists in an accessible state from the beginning, but as described later in the second embodiment, the multimedia information
  • the information set generation process of defining and generating an information set by collecting mutually relevant information among media information including image information, text information, and audio information may be executed as preprocessing.
  • the multimedia information classification and arrangement device according to the present invention is used for checking by a user.
  • Information set acquisition processing (S101) is executed by the information set acquisition unit 10 by executing a search.
  • the information set acquisition unit 10 provides a multimedia information source 80 such as a WWW server that is accessible on a network 70 such as an Internet via a communication interface. Collect information set from and get information set.
  • a multimedia information source 80 such as a WWW server that is accessible on a network 70 such as an Internet via a communication interface.
  • information set acquisition processing S101
  • a plurality of information sets in which image information and text information are associated are acquired.
  • only those hit by keyword search may be acquired.
  • the multimedia information classification and arrangement device generates an information set icon corresponding to each information set by the information visualization unit 30 (S102).
  • the procedure is as shown in Figure 2.
  • the compressed image is combined with an icon indicating sound data above the compressed image and an icon indicating text content below the compressed image.
  • the feature amount extraction unit 20 may be used in performing the feature amount extraction process for generating the information set icon.
  • the multimedia information classification and placement device receives an instruction regarding information classification and placement from the user and starts the classification and placement processing.
  • the axis setting unit 40 sets a feature amount to be assigned to an axis of the classification layout space, and executes an axis setting process (S103) for defining the classification layout space.
  • S103 axis setting process
  • the multimedia information classification and placement apparatus uses the feature amount extraction unit 20 to extract feature amounts assigned to each spatial axis from each media information of the collected information set.
  • the processing (S104) is executed.
  • the feature amount extraction unit 20 extracts the feature amount specified in the axis setting process S103 from each information set. In this example, the feature amount of the sound color of the sound data and the DCT conversion coefficient feature amount of the image data are extracted.
  • multiple feature vectors can be output.
  • the multimedia information classification and arrangement device executes an information classification and arrangement process (S105) for classifying and arranging the information set icons in the classification and arrangement space according to the extracted feature amounts by using the classification and arrangement unit 50. I do.
  • the information display section 60 executes an information display process (S106) for displaying the classified arrangement space and the information set icons arranged and arranged in the classified arrangement space from a set viewpoint.
  • Figure 5 shows an example in which six information set icons collected with the keyword "musician" are classified and arranged in the classification arrangement space.
  • the X-axis is the timbre feature
  • the piano is the X-axis left
  • the violin is near the center
  • the harp is near the right
  • the Y-axis is the DCT transform coefficient of the image data.
  • the information set icons in the example of Fig. 5 are sound data.For example, icons such as piano keyboard, violin, and harp are assigned to the icon part of the sound data in accordance with the tone. It is decorated in such a way that it is colored according to.
  • separate information set icons are displayed at the classification and placement position for each feature vector, or the same information set icon is copied. To represent one information unit with a plurality of icons.
  • Step S107: N0 the search processing may be terminated, but if the original search processing purpose has not been achieved and re-axis setting processing is required (Step S107: YES) ),
  • the multimedia information classification and arrangement device of the present invention can loop to step S103 in order to perform the re-axis setting process.
  • the feature amount to be assigned to the axis of the classification layout space is reset, and the re-axis setting process for redefining the classification layout space is executed.
  • the feature amount to be assigned to the classification arrangement space axis is reset until the required image information is obtained (S103), and the reset feature amount is re-extracted from each media information of the information set (S1).
  • the information set icons are reclassified and rearranged in the classification arrangement space based on the re-extracted feature amounts (S 105), and the information relocated to the classification arrangement space by the information display section 60. Redisplay the set icon (S106).
  • the axis setting unit 40 can change the X-axis from timbre features to ⁇ ⁇ Able transform coefficient features, and the Y-axis from DCT transform coefficient features to HSI color histogram features. is there. Such a series of axis resetting processes can be repeated until a desired search result is obtained.
  • the web page of the information set corresponding to the selected information set icon by looking at the display result, which is the classification and arrangement of information set icons such as image information. . If a URL that contains the information set such as the image information is also stored as part of the information set icon, the URL information will be displayed when the image information is selected by the user on the information display section 60.
  • the web page can be displayed based on the URL. For example, select image information and select “Web
  • an information set icon generation phase is referred to as an information set icon generation phase. It is executed as a process, and collectively generates information set icons for the information sets used in the information set search phase.
  • information set search phase information set search and acquisition processing (701), axis setting processing (S 70 2), feature quantity extraction processing (S 703), information classification and placement processing (S 704), information display processing (S 705), and re-axis setting to be executed as necessary Execute processing (loop to step S702).
  • the information set search phase it is not necessary to generate an information set icon from the information set obtained as a result of the search each time, and the information set icons corresponding to the searched information set are classified and arranged. Since it is only necessary to classify and arrange in the space, the processing can be sped up.
  • the feature amount extraction processing 703 may be executed as preprocessing in the information set generation phase, and the information set icon and each feature amount may be extracted from the information set and stored in association with each other. good.
  • the information set search phase it is not necessary to extract a feature amount from the information set obtained as a result of the search each time, so that the processing can be further speeded up.
  • the amount of data to be transferred can be reduced by using an icon that has a high compression efficiency and requires a small amount of data transfer as data used for displaying information. It is possible to reduce the waiting time until the searcher displays the information, and also to ensure that the content of the information for the searcher can be easily viewed and understood.
  • by classifying and arranging information set icons in the feature space according to their feature amounts it is possible to arrange related information set icons in the vicinity or to arrange images having specific features in specific places. Thus, user search efficiency can be improved.
  • the multimedia information classifying and arranging apparatus does not create an information set icon by synthesizing an icon corresponding to each piece of media information constituting an information set. Icons are prepared for each piece of information, and in the classification and arrangement of the information set, the icons are displayed in the corresponding arrangement position for each piece of media information while being temporally switched according to the display order.
  • the device configuration example of the multimedia information classification and placement device of the second embodiment is the same as that of FIG. However, the processing contents of the components are different.
  • the information visualization unit 30 is a unit for each media information constituting the information set. An icon that visually displays the content is assigned, but the information set icon is not generated by synthesizing the icon for each piece of media information.
  • the classification arrangement unit 50 classifies and arranges each piece of media information included in the information set in the classification arrangement space based on the attribute of the characteristic amount of the media information included in each information set and the component of the characteristic amount. Is what you do. Instead of arranging the information set icons, the icons corresponding to the media information included in the information set are classified and arranged as a group.
  • the information display section 60 displays the classification arrangement space and the icons, determines the table order of the icons for each piece of media information included in the information set, and switches the icons temporally and displays them at the arrangement position. .
  • the icons are repeated in the order of an image icon, an audio data icon, and a text information icon, and are switched and displayed every second.
  • the information set acquisition unit 10, the feature amount extraction unit 20, the axis setting unit 40, the network 70, and the multimedia information source 80 may be the same as those described in the first embodiment.
  • the multimedia information classification and arrangement device includes an information set acquisition process (S801) and an icon generation process for each media information (S811). 0 2), axis setting processing of the classification arrangement space (S 803), feature quantity extraction processing of the information set (S 804), classification of icons of each media information into the classification arrangement space, It has an arrangement process (S805), a process of switching and displaying icons of the classification arrangement space and each media information (S806), and a re-axis setting process (loop to step S803).
  • the classification arrangement empty set by the first axis setting process (S803)
  • the re-axis setting process (S806) is not essential.
  • the multimedia information classification and arrangement device performs information set acquisition processing (S80)
  • the information visualization unit 30 After executing 1), the information visualization unit 30 generates an icon corresponding to each piece of media information included in each information set stored in the multimedia information source 80 (S802). The procedure is the same as that shown in Fig. 2, but it is not combined in the form of the information set icon. Note that the feature amount extraction unit 20 may be used to execute the feature amount extraction process in the icon generation of each piece of media information.
  • the axis setting process (S 803) of the classification arrangement space and the feature amount extraction process (S 804) of the information set are the same as S 103 to S 104 in FIG. 4 of the first embodiment. Yes, the explanation is omitted.
  • the multimedia information classifying and arranging device classifies and arranges the icons of each piece of media information in the classifying and arranging space according to the extracted feature amount by using the classifying and arranging unit 50 (S805). Then, the information display section 60 sorts, arranges, and displays the icons in the classification arrangement space while switching the icons of the classification arrangement space and each media information (S806).
  • one information unit is represented by multiple icons by displaying icons at the classification and placement positions for each feature vector.
  • Step S807 N
  • step S807 YES
  • the multimedia information of the present invention is used.
  • the sorting and placement device performs re-axis setting processing. It is possible to loop to step S803.
  • the entire process is divided into two phases, an icon generation phase and an information set search phase, and the icon generation process is executed as a pre-process as an icon generation phase, and the information set search is performed.
  • An icon for each media information used in the phase may be generated collectively.
  • the amount of data to be transferred can be reduced by using icons having high compression efficiency and requiring a small amount of data transfer as data used for displaying information. It is possible to reduce the waiting time until the searcher's information is displayed, and to switch between the icons according to the display order and display them. Nature can be secured.
  • the multimedia information classification and arrangement device applies a self-organizing map to the classification and arrangement into a classification and arrangement space based on the feature amount of the media information by the classification and arrangement unit.
  • FIG. 9 shows a device configuration example of the multimedia information classification and placement device according to the third embodiment.
  • the classification arrangement unit 50 includes a self-organizing map processing unit 51.
  • the 60 network 70 and the multimedia information source 80 may be the same as the multimedia information classification and arrangement device configuration example of FIG. 3 described in the first embodiment.
  • Self-organizing maps are unsupervised learning models using neural networks.
  • a self-organizing map a high-dimensional feature vector space Perform mapping in low-dimensional space. At that time, those with similar feature vectors are located nearby even in low-dimensional space.
  • This self-organizing map is applied to media information classification and arrangement processing, and the information set icon is used as in Embodiment 1 using self-organizing map processing based on the features extracted by the feature extracting unit 20. Alternatively, an icon for each piece of media information is arranged as in the second embodiment.
  • the self-organizing map processing unit 51 executes the self-organizing map generation process on the feature amount extracted by the feature amount extracting unit 20.
  • the classification arrangement unit 50 of the third embodiment classifies and arranges in the classification arrangement space based on the arrangement information obtained by the self-organizing map generated by the self-organizing map processing unit 51.
  • the depth information is displayed in descending order of relevance based on the keyword returned by the text search server and the relevance of the Web page. You can switch between them and display them.
  • the multimedia information classification and placement apparatus of the third embodiment by applying the self-organizing map processing, an image whose contents are considered to be close in space is close and the contents are far. What you think can be placed far away from each other.
  • FIG. 10 shows a device configuration example of the multimedia information classification and placement device of the fourth embodiment.
  • the information display unit 60 includes a display viewpoint moving unit 61. Note that, other than the information display unit 60, the information set acquisition unit 10, the information visualization unit 30, the axis setting unit 40, the feature amount extraction unit 20, the classification arrangement unit 50, the network 70, and the multimedia information
  • the source 80 may be the same as that of the multimedia information classification and arrangement device configuration example of FIG. 3 described in the first embodiment.
  • the display viewpoint moving unit 61 when displaying the classified arrangement space in which the information set icons are classified and arranged by the classification arrangement unit 50 in the information display unit 60, sets the viewpoint setting position for displaying the space. It has a function to move.
  • the information display unit 60 displays the classification arrangement space viewed from the display viewpoint set by the display viewpoint moving unit 61.
  • the display viewpoint of FIG. 5 shown in the first embodiment is set as a default, and the display viewpoint moving unit 61 can dynamically change the display viewpoint.
  • the information set icons can virtually move freely in the classified layout space where the information set icons have been classified and arranged, and the information set icons can be displayed as being classified and arranged in the classified layout space as seen from the moving position.
  • the display surface is basically a two-dimensional plane, so it can be displayed in perspective, but the information set icons at deep positions tend to be difficult to see.
  • the display viewpoint can be dynamically changed, and according to the operation of the searcher, the classification of the information set icons that the searcher wants to see in more detail.
  • the arrangement can be displayed near the display surface of the display.
  • the multimedia information classification and arrangement device includes a classification arrangement space for each axis.
  • the attribute allocation of the feature amount to the attribute is reset, the already displayed classification layout space is reset, and each information set icon is re-classified and allocated to the reset classification layout space, and each information set is
  • the displayed media information is moved in predetermined steps from the position displayed before re-sorting and arrangement to the position displayed after re-sorting and arrangement. Is displayed.
  • the function to fix the information set icon selected by the user to a specific position in the sorting arrangement space specified by the user, and the re-setting of each information set icon to the sorting arrangement space with the axis re-set When categorizing and arranging, it has a function to fix the information set icon selected by the user at a specific position and re-classify and arrange only the other information set icons according to the categorizing and arrangement space.
  • FIG. 11 is a simplified illustration of a device configuration of a multimedia information classification and placement device according to a fifth embodiment.
  • the information display unit 60 includes a moving image processing unit 62.
  • the classification arrangement section 50 includes an arrangement position fixed designation section 52.
  • the information set acquisition unit 10, the information visualization unit 30, the axis setting unit 40, the feature amount extraction unit 20, the network 70, and the multimedia information source other than the classification arrangement unit 50 and the information display unit 60 80 may be the same as that of the configuration example of the multimedia information classification and arrangement device in FIG. 3 described in the first embodiment.
  • the moving image processing unit 62 of the information display unit 60 displays the information set icons before re-sorting and arranging when displaying the state of re-sorting and arranging each information set icon in the classification and arrangement space by resetting the axis. It has a function of moving the display from the position where it was set to a position displayed after the re-classification and arrangement at predetermined intervals, and displaying it. For example, the position of each information set icon before the reclassification and arrangement is stored, and the position of each information set icon after the reclassification and arrangement is stored in the classification arrangement section 5.
  • the predetermined step may be a movement of a predetermined distance regardless of the number of movement steps, or may be a step of adjusting the movement distance so that the movement is completed in the predetermined number of steps.
  • the classification arrangement section 50 includes an arrangement position fixed designating section 52, and the user can designate through the arrangement position fixed designating section 52 to fix a specific information set icon at a specific position.
  • the classification arrangement unit 50 keeps the specified information set icon fixed at a specific position, and sets another information set. It has a function to re-classify and arrange only icons according to the classification layout space.
  • FIG. 5 described in the first embodiment is used as an example of the display state of the information set icons in the classification layout space before the re-classification layout.
  • the timbre is not set on the X axis (horizontal direction)
  • the DCT conversion coefficient for image information is set on the Y axis (vertical direction)
  • no feature is set on the Z axis (depth direction).
  • the state of the re-sorting and arrangement will be described.
  • the HSI color histogram feature is assigned as the axis in the XY plane direction, and it is assumed that all the "0" values are assigned without assigning any feature to the Z-axis.
  • FIG. 12 shows a display example after the re-sorting and arrangement displayed by the information display section 60.
  • the information set icons are re-classified and arranged according to the feature amounts newly assigned to the axes by the re-axis setting.
  • the moving image processing unit 62 moves and displays each information set icon in a predetermined increment from a position displayed before the re-classification and arrangement to a position displayed after the re-classification and arrangement.
  • each information set icon is displayed in predetermined steps from the position before the re-classification and arrangement to the position after the re-classification and arrangement, so that the information set icon can be tracked visually. Display the movement.
  • the moving image processing unit 62 stores the position of each information set icon shown in FIG.
  • the user selects one or a plurality of information set icons from the displayed screen to grasp the relationship between a specific information set and another information set via the arrangement position fixed designation section 52. .
  • one information set icon is selected, the information set icon is fixed at the center, and the other information set icons are arranged in the classification arrangement space by the self-organizing map method.
  • select a plurality of information set icons for example, four information set icons, fix the positional relationship of these information set icons to a specific positional relationship, for example, to the four corners of the screen, and then use the other information set icons to self-organize. It is also possible to place a specific information set icon in a specific positional relationship and classify and place it in the classification layout space by the generalized map method.
  • FIG. 13 and FIG. 14 are diagrams illustrating the concept of the re-sorting arrangement in which a specific information set icon is fixed at a specific position using the arrangement position fixing specifying unit 52. Fig.
  • FIG. 13 is a diagram showing a state where a specific information set icon is selected and designated using the arrangement position fixed designation unit 52 before re-classification and arrangement.
  • Fig. 14 shows the information set icon selected and designated in Fig. 13
  • FIG. 13 is a diagram showing a state where another information set icon is re-sorted and arranged while being fixed at a specific position (the center of the screen).
  • the information set icon 5 is an information set icon selected and designated by the user via the arrangement position fixed designation section 52. If this information set icon 5 is fixed at the center and reclassified and arranged according to attributes different from those before reclassification, for example, color information, the information set icon with a color close to the information set icon 5 as shown in Fig. 14 will be in the center. They will gather near.
  • a reclassification and placement function in which each information set icon is moved at predetermined intervals is used. It is also possible to display the information set icons in such a way that each information set icon moves to the position after reclassification at a predetermined interval when the specific information set icon is fixed at a specific position. It is also possible.
  • the classification arrangement space can be reset, and the information set icon of interest can be re-classified in a fixed state at a specific position, and how the arrangement position of each information set icon has changed can be determined.
  • the status can be visually checked, and information set icon search and classification processing can be used. Convenience is improved.
  • a multimedia information classification and placement device according to a sixth embodiment of the present invention will be described.
  • the multimedia information classification and arrangement device is obtained by constructing the multimedia information classification and arrangement device according to the first to fifth embodiments with a client-server system via a computer network.
  • the server-computer 100 has an information visualization unit 30, an information set acquisition unit 10, and a feature value extraction unit 20, and the client computer 101 has an axis setting unit 40 and a classification arrangement.
  • Figure 16 shows a server-computer 100 with an information visualization unit 30, an information set acquisition unit 10, a feature value extraction unit 20, and a classification arrangement unit 50, with the client computer 101 having an axis
  • a setting unit 40 and an information display unit 60 are provided.
  • FIGS. 15 to 16 there are several methods for acquiring the information set.
  • a server computer 100 is used as a mouth pot to automatically collect and automatically update an information set having predetermined contents from a network or the like.
  • the information set acquisition unit 10 is illustrated in the server computer 100, but the information set acquisition unit 10 of the information set acquisition unit 10 collects information sets from the information set acquisition unit 12.
  • the key input unit 14 is separated and provided in the client computer 101 (not shown).
  • the user of the client computer 101 uses the information set collection key input unit 14 to The information set collection key is input, and the input information set collection key is passed to the information set acquisition unit 10 of the super computer 100, and the server computer 100 uses the information set collection key.
  • the multimedia information classification of the present invention can be performed by the client server system.
  • An arrangement device can be constructed.
  • the multimedia information classification and arrangement device of the present invention provides various programs by recording a program describing processing steps for realizing the processes described in the above-described embodiments on a computer-readable recording medium and providing the program.
  • the recording medium storing the program having the processing steps for realizing the multimedia information classification and arrangement device of the present invention is a CD-ROM 202 or a flexible disk.
  • a portable recording medium such as 203 but also a recording medium 200 in a recording device on a network and a recording medium 205 such as a hard disk or a RAM of a computer.
  • the program is dictated on the computer 204 and executed on the main memory.
  • the amount of data to be transferred is reduced by using an information set icon, which has a high compression efficiency and requires a small amount of data transfer, as data used for displaying information. This makes it possible to reduce the waiting time for the searcher to display the information, and also to ensure that the information content of the searcher can be easily viewed and understood. Wear.
  • the Search efficiency can be improved. Also, in the case where the icons of the respective media information are switched and displayed in accordance with the display order instead of the information set icons, it is possible to ensure that the information contents can be easily viewed and grasped by the searcher.
  • multimedia information classifying and arranging device of the present invention by holding the information set generation unit, image information, text information, and audio information are included as preprocessing prior to information set classification and arrangement processing.
  • Media information can be collected, the relationship between the collected media information can be analyzed, and the related media information can be combined to generate an information set icon.
  • the classification and arrangement space can be reset, and the information set icon of interest can be re-classified in a fixed state at a specific position.
  • the situation of how the position has changed can be visually checked, and the convenience of information set search and classification processing improves.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

L'invention concerne un dispositif de tri/réorganisation de données qui contribue à réduire la quantité de données transférées et permet une présentation rapide des contenus de ces données en les rendant visibles d'un seul coup d'oeil, et une perception aisée des données affichées sur un dispositif client. Une unité (10) d'acquisition d'ensembles de données acquiert un ensemble de données. Une unité (30) de visualisation de données crée une icône pour cet ensemble de données. L'ensemble de données est un groupe de données multimédia concernant un même objet et sélectionnées parmi divers sous-ensembles de données multimédia. L'icône créée pour l'ensemble de données est une icône qui représente visuellement le contenu de l'ensemble de données, créée par l'association d'icônes attribuées aux sous-ensembles de données média et représentant visuellement ces sous-ensembles. Une unité (40) de définition d'axe attribue une caractéristique à un axe de tri/réorganisation. Une unité (20) d'extraction de caractéristique extrait une caractéristique de chaque sous-ensemble de données multimédia. Une unité (50) de tri/réorganisation réorganise ensuite les icônes des ensembles de données dans un espace de tri/réorganisation conformément aux caractéristiques respectives et les icônes sont affichées par une unité (60) d'affichage de données.
PCT/JP2001/000049 2001-01-09 2001-01-09 Dispositif de tri/reorganisation de donnees multimedia et procede de tri/reorganisation WO2002065339A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/JP2001/000049 WO2002065339A1 (fr) 2001-01-09 2001-01-09 Dispositif de tri/reorganisation de donnees multimedia et procede de tri/reorganisation
JP2002565195A JP4295510B2 (ja) 2001-01-09 2001-01-09 マルチメディア情報分類配置装置および分類配置方法

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2001/000049 WO2002065339A1 (fr) 2001-01-09 2001-01-09 Dispositif de tri/reorganisation de donnees multimedia et procede de tri/reorganisation

Publications (1)

Publication Number Publication Date
WO2002065339A1 true WO2002065339A1 (fr) 2002-08-22

Family

ID=11736891

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2001/000049 WO2002065339A1 (fr) 2001-01-09 2001-01-09 Dispositif de tri/reorganisation de donnees multimedia et procede de tri/reorganisation

Country Status (2)

Country Link
JP (1) JP4295510B2 (fr)
WO (1) WO2002065339A1 (fr)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007524898A (ja) * 2003-04-22 2007-08-30 マイクロソフト コーポレーション リレーションシップビュー
JP2008287125A (ja) * 2007-05-21 2008-11-27 Sony Corp コンテンツの表示方法、コンテンツの表示装置、記録媒体、およびサーバ装置
US7668853B2 (en) 2002-11-27 2010-02-23 Sony United Kingdom Limited Information storage and retrieval
JP2011023029A (ja) * 2010-09-13 2011-02-03 Kyocera Corp データ検索装置及び検索方法
US9424563B2 (en) 2005-03-11 2016-08-23 Microsoft Technology Licensing, Llc Accessing medial context information using contextual links
JP2020537198A (ja) * 2017-10-03 2020-12-17 グーグル エルエルシー 音楽を特定の歌曲として識別する

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH01126693A (ja) * 1987-11-11 1989-05-18 Canon Inc 情報処理装置
JPH06103018A (ja) * 1992-09-21 1994-04-15 Toshiba Corp 表示方式
JPH08227297A (ja) * 1994-12-21 1996-09-03 Fujitsu Ltd サウンド・データのマッピング方法及び表示装置
JPH08329099A (ja) * 1995-05-30 1996-12-13 Minolta Co Ltd 画像情報処理装置
JPH09259130A (ja) * 1996-03-19 1997-10-03 Fujitsu Ltd 情報探索方法及び装置
JPH09298699A (ja) * 1996-05-07 1997-11-18 Matsushita Electric Ind Co Ltd アイコンメニュー表示方法及び表示装置
JPH11352960A (ja) * 1998-06-08 1999-12-24 Yamaha Corp 演奏システムの視覚的表示方法および演奏システムの視覚的表示プログラムが記録された記録媒体
JP2000305688A (ja) * 1999-03-03 2000-11-02 Dassault Systemes アプリケーション制御方法、グラフィカルユーザーインタフェースおよび記録媒体

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH01126693A (ja) * 1987-11-11 1989-05-18 Canon Inc 情報処理装置
JPH06103018A (ja) * 1992-09-21 1994-04-15 Toshiba Corp 表示方式
JPH08227297A (ja) * 1994-12-21 1996-09-03 Fujitsu Ltd サウンド・データのマッピング方法及び表示装置
JPH08329099A (ja) * 1995-05-30 1996-12-13 Minolta Co Ltd 画像情報処理装置
JPH09259130A (ja) * 1996-03-19 1997-10-03 Fujitsu Ltd 情報探索方法及び装置
JPH09298699A (ja) * 1996-05-07 1997-11-18 Matsushita Electric Ind Co Ltd アイコンメニュー表示方法及び表示装置
JPH11352960A (ja) * 1998-06-08 1999-12-24 Yamaha Corp 演奏システムの視覚的表示方法および演奏システムの視覚的表示プログラムが記録された記録媒体
JP2000305688A (ja) * 1999-03-03 2000-11-02 Dassault Systemes アプリケーション制御方法、グラフィカルユーザーインタフェースおよび記録媒体

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
HATANO et al., "Eizou Database no Dou-teki Clustering to Sozai Kensaku Kikou ni tsuite," Technical Research Report of Information Processing Society of Japan (IPSJ) (96-DBS-109-18), Vol. 96, No. 68, 24 July, 1996 (24.07.96), pp. 105-110 *
HAYAKAWA et al., "User no Riyou Rireki ni motozuku WWW Server no Chizu-gata Directory," Technical Research Report of Information Processing Society of Japan(IPSJ) (97-HI-70-3), Vol. 97, No. 2, 16 January, 1997 (19.01.97), pp. 17-24 *
ISHIWAKA et al., "Toukei-teki Inshou Kuukan wo mochiita Gazou Inshou Sousa Houshiki," Transactions of Information Processing Society of Japan (IPSJ) Symposium, Vol. 96, No. 10, 02 December, 1996 (02.12.96), pp. 131-137 *
SANO et al., "Jiko Soshiki-ka Map wo mochiita Web Bunsho no Taiwa-teki Bunrui to Sono Shikaku-ka," Technical Research Report of Information Processing Society of Japan (IPSJ) (98-DBS-116(1)-5), Vol. 98, No. 57, 08 July, 1998 (08.07.98), pp. 33-40 *
TANABE et al., "Tajigen Shinri Kuukan wo mochiiru Ruiji Gazou Kensaku-hou," Transactions D-II of the Institute of Electronics, Information and CommunicationEngineers, Vol. J75-D-II, No. 11, 25 November, 1992 (25.11.92), pp. 1856-1865 *
UEHARA et al., "MIRACLES: Multimedia Jouhou no Panoramic Kensaku System; Web Kensaku e no Ouyou", Technical Research Report of the Institute of Electronics, Information and Communication Engineers (DE2000-3), Vol. 100, No. 31, 02 May, 2000 (02.05.00), pp. 17-24 *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7668853B2 (en) 2002-11-27 2010-02-23 Sony United Kingdom Limited Information storage and retrieval
JP2007524898A (ja) * 2003-04-22 2007-08-30 マイクロソフト コーポレーション リレーションシップビュー
US7689525B2 (en) 2003-04-22 2010-03-30 Microsoft Corporation Relationship view
US9424563B2 (en) 2005-03-11 2016-08-23 Microsoft Technology Licensing, Llc Accessing medial context information using contextual links
US11481086B2 (en) 2005-03-11 2022-10-25 Microsoft Technology Licensing, Llc Accessing media context information using contextual links
JP2008287125A (ja) * 2007-05-21 2008-11-27 Sony Corp コンテンツの表示方法、コンテンツの表示装置、記録媒体、およびサーバ装置
US8402389B2 (en) 2007-05-21 2013-03-19 Sony Corporation Methods and apparatus for displaying contents using icon images
JP2011023029A (ja) * 2010-09-13 2011-02-03 Kyocera Corp データ検索装置及び検索方法
JP2020537198A (ja) * 2017-10-03 2020-12-17 グーグル エルエルシー 音楽を特定の歌曲として識別する
JP7143327B2 (ja) 2017-10-03 2022-09-28 グーグル エルエルシー コンピューティング装置によって実施される方法、コンピュータシステム、コンピューティングシステム、およびプログラム

Also Published As

Publication number Publication date
JPWO2002065339A1 (ja) 2004-06-17
JP4295510B2 (ja) 2009-07-15

Similar Documents

Publication Publication Date Title
WO2001031502A1 (fr) Dispositif et procede de classement et de rangement d'informations multimedia
JP4579404B2 (ja) 画像処理方法及びその装置
JP6190887B2 (ja) 画像検索システムおよび情報記録媒体
JP4569955B2 (ja) 情報格納及び検索方法
JP5340517B2 (ja) マルチメディア情報に対するメタ・ディスクリプタ
US20030123737A1 (en) Perceptual method for browsing, searching, querying and visualizing collections of digital images
US20040215660A1 (en) Image search method and apparatus
JP2009509215A (ja) 画像データベースでのナビゲーション、視覚化、及びクラスタ化のための相互ランク類似度空間
KR20100072070A (ko) 콘텐트 아이템들의 수집과 연관하여 메타데이터 생성
JP2006216031A (ja) 画像特徴抽出方法及び装置
JPWO2010053160A1 (ja) コンテンツ検索装置およびコンテンツ検索プログラム
JP2004030122A (ja) 図面検索支援装置および図面検索方法
JP2011070408A (ja) サンプル画像取得方法、サンプル画像取得装置及び画像分類装置
Brazil et al. Enhancing sonic browsing using audio information retrieval
Blaß et al. Content-based music retrieval and visualization system for ethnomusicological music archives
JP4295510B2 (ja) マルチメディア情報分類配置装置および分類配置方法
Bonnel et al. Effective organization and visualization of web search results
JP2004240887A (ja) 検索情報表示システム及び検索キーワード情報表示方法及び検索キーワード情報表示プログラム
JP4305836B2 (ja) コンテンツ検索表示装置およびコンテンツ検索表示方法
JP2014102625A (ja) 情報検索システム、プログラム、および方法
Murata Visualizing the structure of web communities based on data acquired from a search engine
JP4120597B2 (ja) 部分コンテンツ作成方法及び装置及びプログラム
JP2003316819A (ja) オブジェクト分類検索装置及びそれを実行するためのプログラム
Tietz et al. Knowledge Graph based Analysis and Exploration of Historical Theatre Photographs.
KR101520572B1 (ko) 음악에 대한 복합 의미 인식 방법 및 그 장치

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): GB JP US

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
WWE Wipo information: entry into national phase

Ref document number: 2002565195

Country of ref document: JP