WO2007131311A2 - System and method for net searches activated by digital and/or analogical signals - Google Patents

System and method for net searches activated by digital and/or analogical signals Download PDF

Info

Publication number
WO2007131311A2
WO2007131311A2 PCT/BR2007/000115 BR2007000115W WO2007131311A2 WO 2007131311 A2 WO2007131311 A2 WO 2007131311A2 BR 2007000115 W BR2007000115 W BR 2007000115W WO 2007131311 A2 WO2007131311 A2 WO 2007131311A2
Authority
WO
WIPO (PCT)
Prior art keywords
descriptors
signals
image
databank
search
Prior art date
Application number
PCT/BR2007/000115
Other languages
French (fr)
Other versions
WO2007131311A3 (en
Inventor
Roney Leon Thompson
Original Assignee
Roney Leon Thompson
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Roney Leon Thompson filed Critical Roney Leon Thompson
Publication of WO2007131311A2 publication Critical patent/WO2007131311A2/en
Publication of WO2007131311A3 publication Critical patent/WO2007131311A3/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/951Indexing; Web crawling techniques

Definitions

  • the present invention relates to a system and method for conducting searches in net and/or other computational environments, said search being activated by digital and/or analogical signals, thereby providing an alternative and advantageous approach to the existing systems and methods for search activated by alphanumeric entries.
  • a signal corresponding to an image is used as net search activating element. Indexes related to said image are obtained and compared, in net environments, with indexes of other images, so as to provide a similarity rate between the input image - defined as reference image or reference signal - and the other images found over the net searches, followed by the selection and displaying of similar images.
  • the system and method of the invention thereby provides a search method in net environments, which is activated by signals such as, but not restricted to, an image.
  • the Internet can be seen as a huge database in view of the countless information it can provide access to.
  • the same parallel can be applied to other kinds of nets. Therefore, one of the major current challenges in this field is to manage such amount of information.
  • precisely identify the kind of information wanted by the current and potential users of net environments is a substantial practical and technical challenge.
  • search engines such as Google ® , Yahoo ® and others.
  • the demand for more efficient search engines is still high in some specialized segments.
  • a search engine is more efficient when it can rapidly and precisely provide the information wanted by the user considering the specific requirements of the user in a particular moment.
  • the current systems and methods for net searches generally use the following mechanism: the user inputs a search engine with one or more words that are used as parameters for comparing with texts comprising said word(s).
  • the outcome or response of said systems is obtained in the form of texts, files containing text etc., which are available in other sites seemingly related to the input word or expression.
  • Google recently launched another product for search engines the Google-image ® .
  • This product is implemented by a specific method/system, in which the user chooses and inputs a word or text to the system, and obtains as result a series of images.
  • the categorization of such images occurs indirectly, since the search activation criterion is an alphanumeric text, that is, the resulting images are related to the name of the file or directory in which said images are categorized.
  • the search method seeks for the directory "university” in which several images were filed for being previously considered related to the input name "university”.
  • Said method for being indirect, is poorly efficient in several situations and can generate sound distortions, such as displaying images having no relation with the input.
  • Patent literature comprises some technologies related to general field of the present invention, although neither anticipating nor suggesting it.
  • Document WO 2005/066844 discloses an universal search engine for use in net environments. The method underneath said search engine provides a search activated by an alphanumeric entry, the results being categorized according to a similarity index attributed to the documents found to be related to said entry. Said method also displays the ranking of result lists comparatively, that is, it creates a document in which the search results are organized at least in part according to said ranking.
  • Said search engine comprises: a search component configured to identify documents related to a search entry within databases containing a plurality of document categories; a search component which provides lists of search results corresponding to at least one category; a ranking component configured to organize said lists of search results in a comparative fashion; and a correlation generating component configured to provide a document having the search results organized according to said lists.
  • said system and method provides an alphanumeric search engine.
  • Patent US 6,865,575 discloses a method and apparatus to use modified indexes to provide search results originated from ambiguous search criteria.
  • the method uses the conventional alphanumeric index and converts it into a second index rendered ambiguous as is the original search entry.
  • the original and ambiguous search entry is then compared with the secondary ambiguous index, and the resulting documents are listed as search results.
  • said system and method provides improvements over an alphanumeric search engine.
  • Document WO 2005/111896 discloses a method for attributing indexes to internet-available documents comprising or not comprising images, said method aiming to categorize and filter its distribution over net environments.
  • Said method can be used for the classification of market documents or banners such as those comprising images, and consists of using one or more indexes associated to a given document containing an image by means of optical recognition technologies.
  • the document is therefore approved or not for distribution according to the attributed index(es) and the corresponding attributed category given to said document.
  • said method does not provide results of net searches based to their similarity with an entry image or entry document comprising an image.
  • a system and method for conducting searches in net and/or other computational environments in which a set of signals is entered or captured by any capturing/obtaining means, said set of signals being submitted to system and method of the invention, which comprises at least some of the following steps/features:
  • a system and method for conducting searches in net and/or other computational environments in which an image entry signal is processed so as indexes regarding said image are obtained and compared with indexes obtained from other images available in net environments; lists files containing images with similar indexes to those of the reference image and/or produces lists that comprise images, texts, sounds, movies, links or any other document previously associated to the image containing similar indexes to those of the reference image.
  • a system and method for conducting searches in mobile nets which are activated by non-conventional signals. It is therefore another object of the invention to provide system and method for conducting searches in mobile nets which, starting from an image entry signal, provides the obtention of descriptors of said image and obtains, by means of mobile net searches, lists of files containing similar descriptors to those of the reference image, so as the result list be displayed in the mobile device of the user, said result list containing images, texts, sounds, movies, links and/or internet sites related to the reference image.
  • FIGURES Figure 1 Schematic representation of the general diagram of the invention.
  • FIG. 2 Schematic representation of the one of the preferred embodiments of the invention.
  • FIG. 3 Schematic representation of the one of the preferred embodiments of the invention, in which the invention serves as a tourist guide.
  • the present invention provides a system and method for conducting searches in net and/or other computational environments, said search being activated by a set of digital and/or analogical signals, so as to provide an alternative approach to the existing, alphanumeric-activated, systems and methods for conducting searches.
  • Figure 1 shows a schematic representation of the general diagram of the invention, in which a set of signals is entered and/or captured/obtained by any known means therefor, such as digital cameras, recorders, images digitalization devices, mobile communication devices etc, said set of entry signals comprising Images, sounds, radio waves etc, as well as combinations thereof; the set of entry signals is then processed by the system and method of the invention, which comprises the following steps: the set of entry signals is optionally transformed and/or filtered by routines such as JAVA, neural networks and the like.
  • routines such as JAVA, neural networks and the like.
  • Each transformed element of the set of entry signals constitutes a reference signal; the reference signals are processed by means for obtaining/extracting descriptors of the reference signals such as JAVA, neural networks and the like; the descriptors of the reference signals are stored in a descriptor databank, and are selected by means of comparing said descriptors with descriptors of other signals using methods such as the Euclidian distances method with non-supervised neural networks, supervised neural networks and so forth; the descriptors of the reference signals are compared with those of other signals which are available in a descriptor databank or with those of a signal databank (case in which such signals are submitted to one or more means for obtaining/extracting their descriptors); a signal information databank is formed; the selection of information databank sub-directories corresponding to the selected descriptors is provided; the results, optionally further ordered and/or categorized, are then displayed to the user.
  • the results may comprise images, sounds, texts, movies, links with other sites, or combinations thereof.
  • the filtering and/or transforming and/or processing of the entry signal can be performed by several means.
  • the entry signal is converted into a set of integer numbers representing the features of said signal.
  • the entry signal is converted into a set of integer numbers representing the features of said signal.
  • the JAVA language functions being preferred in the present invention.
  • the set of integer numbers is altered in a specific fashion, the JAVA language functions also being preferred in the present invention.
  • the next step is reconverting the new set of integers obtained by said transformation into a new signal, now referred to as reference signal.
  • the so obtained reference signal may therefore enhance or diminish certain features of the original entry signal, so as to drive or improve the foregoing searches.
  • the process for obtaining reference signal descriptors comprises two steps. First, the reference signal is converted into a set of integer numbers representing the features of the reference signal. Those skilled in the art will know that several ways of performing such conversion are available, the JAVA language functions being preferred in the present invention. Second, the descriptors are calculated using the integer numbers of the first step. Those skilled in the art will know that several sets of descriptors can be used, such as mean, standard deviation, kurtosis etc.
  • the set of descriptors of the reference signal is linked to its corresponding signal.
  • the process for biunivocal association of a reference signal with its corresponding set of descriptors comprises the association of the reference signal name which is in the same descriptors directory.
  • the processes for comparing the reference descriptors with the descriptors of descriptor databanks, and the selection of descriptors considered similar are described as follows.
  • the search and selection of signal(s) corresponding to the reference signal derive from the similarity between the reference descriptors and the descriptors of descriptor databanks.
  • the literature comprises several examples of techniques employed for obtaining similarity indexes between descriptors such as, e.g., supervised neural networks and non-supervised neural networks.
  • the method of the Euclidian distances with non-supervised neural networks is used along with a setup factor in the implemented algorithm.
  • the process for selecting the sub-directories of the databank comprising information on the selected descriptors aims to provide the ordering and displaying of the results.
  • Those skilled in the art will know that several ways of performing such process are available, being preferred in the present invention the process for biunivocal association described above, repeatedly, so as to maintain the level of similarity obtained with the comparison between the descriptors.
  • the process for displaying the results of the user searches may be performed by several methods known in the art, the creation of a .JSP page with file download support being preferred in the present invention.
  • the system and method of the invention can be implemented in several environments, including fixed and mobile workstations, as well as combinations thereof.
  • a digital or digitalized image is used as activating element of a search in net environments. Indexes related to said image are obtained and compared with indexes of other images, so as to provide a similarity index between the entry image and the other images, therefore selecting and displaying similar images.
  • the present invention has as one of its innovative features the capacity of providing searches within net environments, optionally mobile network environments, in which the entry signal and/or the activating signal is preferably an image, and in which the results may comprise images, sounds, texts, movies, links with other sites, or combinations thereof.
  • the present invention consists of a technical solution which combines several independently known technologies, such as: computer, Internet Server, satellite based signal transmission, signal processing techniques, JAVA language, digital cameras, cell phones etc. Therefore, no new technology is required for employing the present invention. Consequently, the present invention can be readily appreciated and used in daily life.
  • system and method of the invention further comprises the steps of:
  • Each image of said image databank is processed so as to obtain integer numbers associated with their pixel color and opacity, said processing being preferably performed by JAVA language functions.
  • the image descriptors are then calculated from these integer numbers.
  • the descriptors constitute a form of representation of the features of each image, and can comprise the mean, standard deviation, kurtosis, and so forth of the pixels ' color and opacity.
  • Each set of descriptors is associated with its corresponding image;
  • Images available in the internet or retrieved/obtained from equipments such as digital cameras, cell phones etc. that are read and stored in the user ' s computer are transmitted for Internet servers and/or satellite and/or cable and/or radio, by means of JAVA language functions;
  • the entry image is filtered by means of transforming functions which can modulate the brightness, colors, as well as bend, twist, invert or perform linear or angular deformations, and/or select parts of the image.
  • Other kinds of signal transformation can also be used, such as ageing/rejuvenating, degradation, altering the characterization of human recognition such as color or haircut modification, presence or absence of moustache, beard and so forth.
  • Still also other kinds of signal transformation can be used such as weather condition changes and so forth. 6) category processing choice
  • the signal category processing can be chosen as another signal, now as an alphanumeric, digitized and/or pronounced signal, so as to speed up the whole process.
  • the activating category can be chosen as another signal, now as an alphanumeric, digitized and/or pronounced signal, so as to restrict the search space and therefore speed up the whole process.
  • the calculated descriptors which are associated to a new read/entered image, that is, to the reference image, are then compared with the descriptors of previously filed images. This comparison is performed with the aid of proper mathematical methods such as, e.g., the Euclidian distances method.
  • the images are considered similar to the reference image when having low Euclidian distances (below a given threshold) between their descriptors and the descriptors of the reference image.
  • the threshold may be influenced/determined by the user.
  • the output may comprise images which are similar to the reference image, or may comprise texts, sounds, movies and/or links related to the reference image.
  • the user just have to be connected to any net environment to receive the output/results of the search, such internet, wireless nets and/or mobile phones. 11) output/results category choice:
  • the output/results category can be selected with another signal, now as an alphanumeric, digitized and/or pronounced signal, so as to adjust the kind of result intended by the user.
  • the method is implemented with JAVA language, aiming to better communicate with net environments such as the Internet, cell phones, digital cameras, since this is nowadays the proper language for satellite and/or several cell phones communication. Accordingly, a given image, such as a cell phone photographed image, can be transferred via satellite and/or other wireless means, to/by a JAVA program.
  • the process of searching images of one preferred embodiments of the present invention consists of searching images by recognition and selection, that is, the search activating signal is an image.
  • This process is substantially different from that currently performed in other search engines such as the Google Image, which uses a directory based search using an alphanumeric base as search activating signal.
  • the present invention can also be employed together with existing search engines such as Google, by generating a text out of the entry image and then searching related texts and/or images using Google methods.
  • the present invention can be used in several segments and technical applications. The foregoing examples are preferred embodiments and are not to be understood as limitations of the scope of the invention.
  • an Internet site 100 contains a fixed directory 200, named as terminal, where three databanks are first stored: an image databank 300, a descriptor databank 400 biunivocally associated with the image databank 300 by means of the process 009 described below, and a subdirectory databank 500 biunivocally associated (process 009) with the image databank 300 and the descriptor databank 400.
  • the descriptor databank 400 is formed by process 002 described below.
  • each sub-directory is biunivocally associated (process 009) with an image of the image databank 300 and contains a set of data from image, text, sound, movies and/or links which are related with the image to which said sub-directory is associated.
  • a user is connected to a fixed Workstation 600 containing a directory 620, named origin, which in turn contains an image file 650, named reference image.
  • This image 650 is generated by an image processing applied to an original image 670, which includes the possibility of neutral processing, that is, allows no modification of the entry image 670.
  • the image processing transforms/filters the entry image 670 by process 004 described below.
  • Image 670 can be obtained, e.g., by: a digital camera 700 and then transmitted to the Workstation 600; or by a mobile (cell) phone 800 and then transmitted to the Workstation 600; or can be downloaded from an e-mail server 900; or even from other internet site 1000.
  • the user connects site 100 and then inputs the image file 650 by means of process 010 described below.
  • This image is stored in a sub-directory 1100 of terminal 200. Then its descriptors are calculated (process 002) and stored in another sub-directory 1200.
  • process 006 the reference image descriptors are compared to each descriptor stored in the descriptor databank 400.
  • the election, selection and ordering of the set of descriptors of the descriptor databank 400 having higher similarity indexes in relation to those descriptors of sub-directory 1200 (of reference image 650) constitute the process 015 described below.
  • the sub-directories containing information on corresponding images, texts, sounds, movies and/or links are identified in the subdirectory databank 500, maintaining the order obtained by process 015. Then these results are displayed to the user in Workstation 600, by means of process 017 described below.
  • Example 2 - mobile with internet the system and method of the invention neither require site 1000 nor server 900 nor digital camera 700.
  • the functions of device 800 (e.g., digital camera and internet Access) and of the Workstation 600 are both available in a single mobile device.
  • An internet site 100 contains a fixed directory 200 named terminal, where three databanks are firstly stored: an image databank 300, a descriptors databank 400 biunivocally associated with the image databank 300 by means of process 009 described below, and a sub-directories databank 500 biunivocally associated (process 009) with the image databank 300 and with the descriptors databank 400.
  • the descriptors databank 400 is built/formed by process 002 described below.
  • each subdirectory is biunivocally associated (process 009) with an image from databank 300 and contains a set of data of images, texts, sounds, movies and links related to/with the corresponding image.
  • the user connects the internet site 100 by means of mobile Workstation 600 and enters by means of process 010 described below.
  • This image is stored in a sub-directory 1100 within terminal 200.
  • the descriptors of image 650 are calculated (process 002) and stored in another sub-directory 1200.
  • process 006 described below the descriptors of reference image 650 are compared with each and every descriptor stored in the descriptor databank 400.
  • the election, selection and ordering of the sets of descriptors of descriptor databank 400 having the higher similarity indexes to those descriptors contained in subdirectory databank 1200 (of reference image 650) consist the core of process 015 described below. Afterwards, and by means of process 016, the subdirectories containing information on corresponding images, texts, sounds, movies and/or links are identified within sub-directory databank 500, the ordering of process 015 being maintained. The ordered results are then displayed to the user at the mobile Workstation 600, by means of process 017 described below.
  • Example 3 mobile device without internet
  • the execution environment of the system and method of the invention is completely incorporated within the mobile communication device. Accordingly, the execution environment lies within a chip 100 instead of an Internet site 100 and therefore the mobile device needs not to transmit any signal to its exterior.
  • chip 100 has a very high memory capacity and contains a fixed directory 200 named terminal, where three databanks are firstly stored: an image databank 300, a descriptors databank 400 biunivocally associated with the image databank 300 by means of process 009 described below, and a sub-directories databank 500 biunivocally associated (process 009) with the image databank 300 and with the descriptors databank 400.
  • each sub-directory is biunivocally associated (process 009) with an image from databank 300 and contains a set of data of images, texts, sounds, movies and links related to/with the corresponding image.
  • the election, selection and ordering of the sets of descriptors of descriptor databank 400 having the higher similarity indexes to those descriptors contained in sub-directory databank 1200 (of reference image 650) consist the core of process 015 described below. Afterwards, and by means of process 016, the sub-directories containing information on corresponding images, texts, sounds, movies and/or links are identified within sub-directory databank 500, the ordering of process 015 being maintained. The ordered results are then displayed to the user at the mobile Workstation 600, by means of process 017 described below.
  • an oncology society builds an internet site 100 containing a fixed directory 200, where three databanks are firstly stored: a computer tomography image databank 300 comprising non-tumor, benign tumor and malign tumor images; a descriptors databank 400 biunivocally associated with the image databank 300 by means of process 009 described below, and a sub-directories databank 500 biunivocally associated (process 009) with the image databank 300 and with the descriptors databank 400.
  • the descriptors databank 400 is built/formed by process 002 described below.
  • each sub-directory is biunivocally associated (process 009) with a computer tomography image databank 300 and contains a set of data of images, texts, sounds, movies and links related to/with the corresponding breast image.
  • a physician such as those working far away from big cities, connected to a computer 600 performs a computer tomography in a female patient; the corresponding image 650 is stored in a directory 620.
  • Said user connects site 100 and then enters the image file 650 by means of process 010 described below.
  • This image is stored in a sub-directory 1100 of terminal 200.
  • its descriptors are calculated (process 002) and stored in another sub-directory 1200.
  • process 006 described below the reference image descriptors 650 are compared to each descriptor stored in the descriptor databank 400.
  • the election, selection and ordering of the set of descriptors of the descriptor databank 400 having higher similarity indexes in relation to those descriptors of sub-directory 1200 (of reference image 650) constitute the process 015 described below.
  • the sub-directories containing information on corresponding tomography-related images, texts, sounds, movies and/or links, so as to indicate the most likely diagnostic/prognosctic are identified in the subdirectory databank 500, maintaining the order obtained by process 015.
  • these results are displayed to the user in Workstation 600, by means of process 017 described below.
  • Process 002 - consists of a process for obtaining image descriptors and comprises two steps.
  • the reference signal an image in the examples above
  • the reference signal is converted into a set of integer numbers representing the features of the reference signal (image features such as color, opacity etc).
  • image features such as color, opacity etc.
  • the descriptors are calculated using the integer numbers of the first step.
  • Process 004 - consists of a process for filtering and/or transforming and/or processing of the entry signal can be performed by several means and comprises three steps.
  • the entry signal is converted into a set of integer numbers representing the features of said signal.
  • the JAVA language functions being preferred in the present invention.
  • the set of integer numbers is altered in a specific fashion, the JAVA language functions also being preferred in the present invention.
  • the new set of integers obtained by said transformation is reconverted into a new signal, now referred to as reference signal (in the preceding examples a reference image).
  • the JAVA language functions are also preferred in the present invention.
  • the so obtained reference signal may therefore enhance or diminish certain features of the original entry signal, so as to drive or improve the foregoing searches.
  • Process 009 - consists of a process for biunivocal association between an image and the corresponding set of descriptors. This association is obtained by the association of the image name in the same descriptors directory. Those skilled in the art will know that several ways of performing such association are available.
  • Process 010 - consists of a process for sending the image to the site. Those skilled in the art will know that several ways of performing such association are available, the creation of a file upload supported JSP page being preferred in the invention.
  • Processes 006 and 015 - consist of processes for comparing the reference descriptors with the descriptors of the descriptors databank, and the selection of the most related/similar descriptors.
  • the search and selection of image(s) are performed according to the similarity between the reference descriptors with the descriptors available in the descriptors databank.
  • Several techniques can be employed for obtaining similarity indexes between descriptors, such as, e.g., the method of the Euclidian distances with non-supervised neural networks, or supervised neural networks.
  • the preferred embodiments of the invention the method of the Euclidian distances with non-supervised neural networks, along with a setup factor in the implemented algorithm.
  • Process 016 - consists of a process for grouping corresponding sub-directories maintaining the similarity index obtained by the comparison between descriptors, as described above. Those skilled in the art will know that several ways of performing such process are available, the successive repetition of process 009 described above being preferred in the invention.
  • Process 017 - consists of a process for displaying the search results for the user. Those skilled in the art will know that several ways of performing such association are available, the creation of a file download supported JSP page being preferred in the invention.
  • Example 5 police Activity Using any of the preceding examples 1-3 described above, a police unit builds a fingerprint image databank comprising the corresponding descriptors and criminal files. During police activity cops immobilize a suspect and obtain their fingerprints. By using a mobile communication device having camera and internet access the cops send the fingerprint images the police unit databank. These images may be optionally filtered by process 004, so as to include a processing category such as "black ink". The fingerprints are then easily and rapidly found in the fingerprint image databank, the criminal file of the suspect being sent back to the cop ' s mobile communication device. The cop can then arrest or not the suspect based in such information.
  • a processing category such as "black ink”.
  • Example 6 Information about an unknown person Using any of the preceding examples 1-3 described above, an Internet site such as Orkut contains personal information and/or photographs from people. The process for obtaining descriptors is used on said photographs.
  • a person A finds a person B and person A wants to know more information about person B, such as profession, habits etc.
  • Person A photographs person B using a mobile phone: using the system and method of the invention, the image is optionally filtered so as only the face will be the reference image. Then the system accesses the Internet and performs the comparison between the descriptors of the reference image with the descriptors of the images available in a given or a series of Internet site(s); the system can optionally filter the site category for "search category Orkuf; the output may also be filtered so as only "profile category" results are displayed for the user.
  • an Internet site contains personal information and/or photographs from missing children.
  • a photograph of a child which is missing for years is entered into the system.
  • This image is transformed by means of process 004 which render the image older (the same number of years passed since the original photograph was taken).
  • This process can be activated by entering "process category: ageing image X years”.
  • the "aged" image is then the reference image which activates the search, and the person can be found by processes similar to those described above .
  • Example 8 Localizing mobile objects by using satellite images
  • an Internet site contains satellite based images (such as Google-Earth) which stores low distance images (in which vehicle colors, for example, can be distinguished) and updates said images in short periods of time or even continuously.
  • a person A wants to meet person B in a particular location. Person A says he/she will be driving a red car and sends a message with a photograph of his/her car taken from above to person B by its mobile device. Person B also has a mobile device, connects to said site and enters the photography of person A ' s car. The system thus searches the images on Earth having the corresponding category processing criteria. Person B can then follow person A ' s trajectory and can reach the meet location without getting late, or can meet person A at any point of his/her trajectory.
  • satellite based images such as Google-Earth
  • an Internet site contains product based images and their corresponding barcodes.
  • a buyer photographs a product or its barcode, and the system provides pricing information in the nearby shops, nutritional facts, shelf life etc.
  • FIG. 3 shows the case where the user photographs a building.
  • the operations described in more detail in examples 1-3 are performed so as the system results include a list of images, movies, texts and/or links related to the reference image.
  • the user can select any kind of listed information, as well as refine the displayed results by entering "category results: movies".
  • categories results movies

Abstract

The present invention provides a system and method for conducting searches in net and/or other computational environments, said search being activated by digital and/or analogical signals, said system and method being suitable for operation in mobile devices. The system executes, in net and/or other computational environments and starting from a set of input signals transformed into a set of reference signals, searches of images, sounds, texts, movies or links documents and/or web pages related to the reference signal. The system can be used in search tools as Google, Yahoo and the like, as well as in mobile phones, for the search and/or identification of several kinds of non- conventional signals.

Description

Description
SYSTEM AND METHOD FOR NET SEARCHES ACTIVATED BY DIGITAL AND/OR ANALOGICAL SIGNALS
FIELD OF THE INVENTION
The present invention relates to a system and method for conducting searches in net and/or other computational environments, said search being activated by digital and/or analogical signals, thereby providing an alternative and advantageous approach to the existing systems and methods for search activated by alphanumeric entries. In one of the preferred embodiments of the invention, a signal corresponding to an image is used as net search activating element. Indexes related to said image are obtained and compared, in net environments, with indexes of other images, so as to provide a similarity rate between the input image - defined as reference image or reference signal - and the other images found over the net searches, followed by the selection and displaying of similar images. The system and method of the invention thereby provides a search method in net environments, which is activated by signals such as, but not restricted to, an image.
DESCRIPTION OF PRIOR ART
The widespread use of microcomputers in universities and companies growed overwhelmingly in the 80's.By the end of the 80's, more efficient microcontrollers (softwares and hardwares) and several kinds of high level languages (softwares) were developed. With these products, companies and universities revolutionized several areas such as engineering, medicine, environment, finances, law, economy etc. The software implemented systems and methods gained additional capacity in the first years of the 90's, when the internet became popular. The merge of the internet technology with the previously existing ones gave rise to endless products and patents providing applications in several areas of knowledge.
The Internet can be seen as a huge database in view of the countless information it can provide access to. The same parallel can be applied to other kinds of nets. Therefore, one of the major current challenges in this field is to manage such amount of information. In this scenario, precisely identify the kind of information wanted by the current and potential users of net environments is a substantial practical and technical challenge. This is one of the reasons for the huge popularity of search engines such as Google®, Yahoo® and others. However, the demand for more efficient search engines is still high in some specialized segments. A search engine is more efficient when it can rapidly and precisely provide the information wanted by the user considering the specific requirements of the user in a particular moment. The current systems and methods for net searches generally use the following mechanism: the user inputs a search engine with one or more words that are used as parameters for comparing with texts comprising said word(s). The outcome or response of said systems is obtained in the form of texts, files containing text etc., which are available in other sites seemingly related to the input word or expression.
Google recently launched another product for search engines: the Google-image®. This product is implemented by a specific method/system, in which the user chooses and inputs a word or text to the system, and obtains as result a series of images. However, the categorization of such images occurs indirectly, since the search activation criterion is an alphanumeric text, that is, the resulting images are related to the name of the file or directory in which said images are categorized. Accordingly, when the user inputs a word such as "university" the search method seeks for the directory "university" in which several images were filed for being previously considered related to the input name "university". Said method, for being indirect, is poorly efficient in several situations and can generate sound distortions, such as displaying images having no relation with the input.
Patent literature comprises some technologies related to general field of the present invention, although neither anticipating nor suggesting it. Document WO 2005/066844 discloses an universal search engine for use in net environments. The method underneath said search engine provides a search activated by an alphanumeric entry, the results being categorized according to a similarity index attributed to the documents found to be related to said entry. Said method also displays the ranking of result lists comparatively, that is, it creates a document in which the search results are organized at least in part according to said ranking. Said search engine comprises: a search component configured to identify documents related to a search entry within databases containing a plurality of document categories; a search component which provides lists of search results corresponding to at least one category; a ranking component configured to organize said lists of search results in a comparative fashion; and a correlation generating component configured to provide a document having the search results organized according to said lists. In short, said system and method provides an alphanumeric search engine.
Patent US 6,865,575 discloses a method and apparatus to use modified indexes to provide search results originated from ambiguous search criteria. The method uses the conventional alphanumeric index and converts it into a second index rendered ambiguous as is the original search entry. The original and ambiguous search entry is then compared with the secondary ambiguous index, and the resulting documents are listed as search results. In short, said system and method provides improvements over an alphanumeric search engine.
Document WO 2005/111896 discloses a method for attributing indexes to internet-available documents comprising or not comprising images, said method aiming to categorize and filter its distribution over net environments. Said method can be used for the classification of market documents or banners such as those comprising images, and consists of using one or more indexes associated to a given document containing an image by means of optical recognition technologies. The document is therefore approved or not for distribution according to the attributed index(es) and the corresponding attributed category given to said document. However, said method does not provide results of net searches based to their similarity with an entry image or entry document comprising an image.
The above-cited documents and others documents of patent literature regarding the subject matter of the present invention are all based on net searches activated by alphanumeric entries. Consequently, although: methods for obtaining conventional alphanumeric indexes for categorization purposes are known; search engines working in net environments and using conventional alphanumeric indexes; and methods for obtaining indexes from documents comprising images, or even from other data convertible into signal are independently known; so far no technical solution combining all these features is available or known, so as to provide a tool which allows the inclusion of non- conventional mechanisms for activating searches in net environments, followed by the categorized listing of documents and/or information related to the search activating signal. These and other objects constitute the spirit of the invention, which between other advantages provides: the search and listing of images which are similar or related to an entry image; the search and listing of alphanumeric data which are related to an entry image; the search and listing of movies which are related to an entry image; the search and listing of links which are related to an entry image; and so forth.
SUMMARY OF THE INVENTION
It is one object of the present invention to provide a new system and method for conducting searches in net and/or other computational environments, said search being activated by digital and/or analogical signals.
In one aspect, being, therefore, another object of the invention, there is provided a system and method for conducting searches in net and/or other computational environments in which a set of signals is entered or captured by any capturing/obtaining means, said set of signals being submitted to system and method of the invention, which comprises at least some of the following steps/features:
a) means for obtaining/extracting descriptors of the reference signal; a.1) the above means for obtaining/extracting descriptors of the reference signal being optionally preceded by means for filtering/transforming the entry signals so as the corresponding filtered/transformed signals are reference signals which activate the searches; b) means for storing said descriptors in a descriptor databank; c) means for comparing said descriptors with descriptors of other signals; d) means for selecting the set of descriptors which are considered similar/related; e) means for storing information about said signals; f) means for selecting sub-directories from descriptor databank corresponding to the selected descriptors; g) means for displaying the search results to the user.
In another aspect, being, therefore, another object of the invention, there is provided a system and method for conducting searches in net and/or other computational environments in which an image entry signal is processed so as indexes regarding said image are obtained and compared with indexes obtained from other images available in net environments; lists files containing images with similar indexes to those of the reference image and/or produces lists that comprise images, texts, sounds, movies, links or any other document previously associated to the image containing similar indexes to those of the reference image.
In another aspect, being, therefore, another object of the invention, there is provided a system and method for conducting searches in net and/or other computational environments which creates and/or uses an image databank. In another aspect, being, therefore, another object of the invention, there is provided a system and method for conducting searches in net and/or other computational environments which provides the creation of a databank of image related descriptors.
In another aspect, being, therefore, another object of the invention, there is provided a system and method for conducting searches in net and/or other computational environments which provides the satellite-based image recognition and categorization.
In another aspect, being, therefore, another object of the invention, there is provided a system and method for conducting searches in mobile nets which are activated by non-conventional signals. It is therefore another object of the invention to provide system and method for conducting searches in mobile nets which, starting from an image entry signal, provides the obtention of descriptors of said image and obtains, by means of mobile net searches, lists of files containing similar descriptors to those of the reference image, so as the result list be displayed in the mobile device of the user, said result list containing images, texts, sounds, movies, links and/or internet sites related to the reference image. In another aspect, being, therefore, another object of the invention, there is provided a system and method for conducting searches in net and/or other computational environments which provide the choice: of the entry signal category processing; of the category of the search space; and/or of the category of the kind of output signal will be displayed. These and other objects of the invention will be better understood and appreciated with the following detailed description and with the appended claims.
BRIEF DESCRIPTION OF THE FIGURES Figure 1 - Schematic representation of the general diagram of the invention.
Figure 2 - Schematic representation of the one of the preferred embodiments of the invention.
Figure 3 - Schematic representation of the one of the preferred embodiments of the invention, in which the invention serves as a tourist guide.
DETAILED DESCRIPTION OF THE INVENTION
The present invention provides a system and method for conducting searches in net and/or other computational environments, said search being activated by a set of digital and/or analogical signals, so as to provide an alternative approach to the existing, alphanumeric-activated, systems and methods for conducting searches.
Figure 1 shows a schematic representation of the general diagram of the invention, in which a set of signals is entered and/or captured/obtained by any known means therefor, such as digital cameras, recorders, images digitalization devices, mobile communication devices etc, said set of entry signals comprising Images, sounds, radio waves etc, as well as combinations thereof; the set of entry signals is then processed by the system and method of the invention, which comprises the following steps: the set of entry signals is optionally transformed and/or filtered by routines such as JAVA, neural networks and the like. Each transformed element of the set of entry signals constitutes a reference signal; the reference signals are processed by means for obtaining/extracting descriptors of the reference signals such as JAVA, neural networks and the like; the descriptors of the reference signals are stored in a descriptor databank, and are selected by means of comparing said descriptors with descriptors of other signals using methods such as the Euclidian distances method with non-supervised neural networks, supervised neural networks and so forth; the descriptors of the reference signals are compared with those of other signals which are available in a descriptor databank or with those of a signal databank (case in which such signals are submitted to one or more means for obtaining/extracting their descriptors); a signal information databank is formed; the selection of information databank sub-directories corresponding to the selected descriptors is provided; the results, optionally further ordered and/or categorized, are then displayed to the user. The results may comprise images, sounds, texts, movies, links with other sites, or combinations thereof.
The filtering and/or transforming and/or processing of the entry signal can be performed by several means. In a preferred embodiment, the entry signal is converted into a set of integer numbers representing the features of said signal. Those skilled in the art will know that several ways of performing such conversion are available, the JAVA language functions being preferred in the present invention. Thereafter the set of integer numbers is altered in a specific fashion, the JAVA language functions also being preferred in the present invention. The next step is reconverting the new set of integers obtained by said transformation into a new signal, now referred to as reference signal. The so obtained reference signal may therefore enhance or diminish certain features of the original entry signal, so as to drive or improve the foregoing searches.
The process for obtaining reference signal descriptors comprises two steps. First, the reference signal is converted into a set of integer numbers representing the features of the reference signal. Those skilled in the art will know that several ways of performing such conversion are available, the JAVA language functions being preferred in the present invention. Second, the descriptors are calculated using the integer numbers of the first step. Those skilled in the art will know that several sets of descriptors can be used, such as mean, standard deviation, kurtosis etc.
The set of descriptors of the reference signal is linked to its corresponding signal. The process for biunivocal association of a reference signal with its corresponding set of descriptors comprises the association of the reference signal name which is in the same descriptors directory. Those skilled in the art will know that several ways of performing such process are available.
The process for sending the reference signal to the environment in which part of the system and method of the invention is conducted is described next.
Those skilled in the art will know that several ways of performing such process are available, the creation of a JSP page with file upload support being preferred in the present invention.
The processes for comparing the reference descriptors with the descriptors of descriptor databanks, and the selection of descriptors considered similar are described as follows. The search and selection of signal(s) corresponding to the reference signal derive from the similarity between the reference descriptors and the descriptors of descriptor databanks. The literature comprises several examples of techniques employed for obtaining similarity indexes between descriptors such as, e.g., supervised neural networks and non-supervised neural networks. In a preferred embodiment of the invention, the method of the Euclidian distances with non-supervised neural networks is used along with a setup factor in the implemented algorithm.
The process for selecting the sub-directories of the databank comprising information on the selected descriptors aims to provide the ordering and displaying of the results. Those skilled in the art will know that several ways of performing such process are available, being preferred in the present invention the process for biunivocal association described above, repeatedly, so as to maintain the level of similarity obtained with the comparison between the descriptors.
The process for displaying the results of the user searches may be performed by several methods known in the art, the creation of a .JSP page with file download support being preferred in the present invention. The system and method of the invention can be implemented in several environments, including fixed and mobile workstations, as well as combinations thereof. In a preferred embodiment of the invention, a digital or digitalized image is used as activating element of a search in net environments. Indexes related to said image are obtained and compared with indexes of other images, so as to provide a similarity index between the entry image and the other images, therefore selecting and displaying similar images. The present invention has as one of its innovative features the capacity of providing searches within net environments, optionally mobile network environments, in which the entry signal and/or the activating signal is preferably an image, and in which the results may comprise images, sounds, texts, movies, links with other sites, or combinations thereof.
The present invention consists of a technical solution which combines several independently known technologies, such as: computer, Internet Server, satellite based signal transmission, signal processing techniques, JAVA language, digital cameras, cell phones etc. Therefore, no new technology is required for employing the present invention. Consequently, the present invention can be readily appreciated and used in daily life.
In a preferred embodiment, the system and method of the invention further comprises the steps of:
1) creation and/or use of image databanks;
2) creation of databank(s) containing a set of descriptors corresponding to each image;
Each image of said image databank is processed so as to obtain integer numbers associated with their pixel color and opacity, said processing being preferably performed by JAVA language functions. The image descriptors are then calculated from these integer numbers. The descriptors constitute a form of representation of the features of each image, and can comprise the mean, standard deviation, kurtosis, and so forth of the pixels' color and opacity. Each set of descriptors is associated with its corresponding image;
3) creation of a databank containing a set of information from other images , texts, sounds, movies and/or links with other internet pages, which are related to each image;: 4) reading a of new image - entered by the user:
Images available in the internet or retrieved/obtained from equipments such as digital cameras, cell phones etc. that are read and stored in the user's computer are transmitted for Internet servers and/or satellite and/or cable and/or radio, by means of JAVA language functions;
5) image signal processing
The entry image is filtered by means of transforming functions which can modulate the brightness, colors, as well as bend, twist, invert or perform linear or angular deformations, and/or select parts of the image. Other kinds of signal transformation can also be used, such as ageing/rejuvenating, degradation, altering the characterization of human recognition such as color or haircut modification, presence or absence of moustache, beard and so forth. Still also other kinds of signal transformation can be used such as weather condition changes and so forth. 6) category processing choice
The signal category processing can be chosen as another signal, now as an alphanumeric, digitized and/or pronounced signal, so as to speed up the whole process.
7) obtention of descriptors corresponding to the new image: Once the new image is read/entered, the descriptors are calculated as specified in item 2 above.
8) entry category choice
The activating category can be chosen as another signal, now as an alphanumeric, digitized and/or pronounced signal, so as to restrict the search space and therefore speed up the whole process.
9) image recognition:
The calculated descriptors which are associated to a new read/entered image, that is, to the reference image, are then compared with the descriptors of previously filed images. This comparison is performed with the aid of proper mathematical methods such as, e.g., the Euclidian distances method. The images are considered similar to the reference image when having low Euclidian distances (below a given threshold) between their descriptors and the descriptors of the reference image. The threshold may be influenced/determined by the user.
10) displaying the results to the user:
After selecting the most similar descriptors by means of, e.g., JAVA functions, the output may comprise images which are similar to the reference image, or may comprise texts, sounds, movies and/or links related to the reference image. The user just have to be connected to any net environment to receive the output/results of the search, such internet, wireless nets and/or mobile phones. 11) output/results category choice:
The output/results category can be selected with another signal, now as an alphanumeric, digitized and/or pronounced signal, so as to adjust the kind of result intended by the user.
In one preferred embodiment of the invention, the method is implemented with JAVA language, aiming to better communicate with net environments such as the Internet, cell phones, digital cameras, since this is nowadays the proper language for satellite and/or several cell phones communication. Accordingly, a given image, such as a cell phone photographed image, can be transferred via satellite and/or other wireless means, to/by a JAVA program.
The process of searching images of one preferred embodiments of the present invention consists of searching images by recognition and selection, that is, the search activating signal is an image. This process is substantially different from that currently performed in other search engines such as the Google Image, which uses a directory based search using an alphanumeric base as search activating signal. Notwithstanding, the present invention can also be employed together with existing search engines such as Google, by generating a text out of the entry image and then searching related texts and/or images using Google methods. The present invention can be used in several segments and technical applications. The foregoing examples are preferred embodiments and are not to be understood as limitations of the scope of the invention. Example 1 - Fixed-internet
Making reference to figure 2, an Internet site 100 contains a fixed directory 200, named as terminal, where three databanks are first stored: an image databank 300, a descriptor databank 400 biunivocally associated with the image databank 300 by means of the process 009 described below, and a subdirectory databank 500 biunivocally associated (process 009) with the image databank 300 and the descriptor databank 400. The descriptor databank 400 is formed by process 002 described below. In the sub-directory databank 500 each sub-directory is biunivocally associated (process 009) with an image of the image databank 300 and contains a set of data from image, text, sound, movies and/or links which are related with the image to which said sub-directory is associated.
A user is connected to a fixed Workstation 600 containing a directory 620, named origin, which in turn contains an image file 650, named reference image. This image 650 is generated by an image processing applied to an original image 670, which includes the possibility of neutral processing, that is, allows no modification of the entry image 670. The image processing transforms/filters the entry image 670 by process 004 described below. Image 670 can be obtained, e.g., by: a digital camera 700 and then transmitted to the Workstation 600; or by a mobile (cell) phone 800 and then transmitted to the Workstation 600; or can be downloaded from an e-mail server 900; or even from other internet site 1000. The user connects site 100 and then inputs the image file 650 by means of process 010 described below.
This image is stored in a sub-directory 1100 of terminal 200. Then its descriptors are calculated (process 002) and stored in another sub-directory 1200. By means of process 006 described below the reference image descriptors are compared to each descriptor stored in the descriptor databank 400. The election, selection and ordering of the set of descriptors of the descriptor databank 400 having higher similarity indexes in relation to those descriptors of sub-directory 1200 (of reference image 650) constitute the process 015 described below. Afterwards, and by means of process 016, the sub-directories containing information on corresponding images, texts, sounds, movies and/or links are identified in the subdirectory databank 500, maintaining the order obtained by process 015. Then these results are displayed to the user in Workstation 600, by means of process 017 described below.
Example 2 - mobile with internet In one preferred embodiment, making reference to figure 2, the system and method of the invention neither require site 1000 nor server 900 nor digital camera 700. Preferably, the functions of device 800 (e.g., digital camera and internet Access) and of the Workstation 600 are both available in a single mobile device. An internet site 100 contains a fixed directory 200 named terminal, where three databanks are firstly stored: an image databank 300, a descriptors databank 400 biunivocally associated with the image databank 300 by means of process 009 described below, and a sub-directories databank 500 biunivocally associated (process 009) with the image databank 300 and with the descriptors databank 400. The descriptors databank 400 is built/formed by process 002 described below. In the sub-directories databank 500, each subdirectory is biunivocally associated (process 009) with an image from databank 300 and contains a set of data of images, texts, sounds, movies and links related to/with the corresponding image.
An user connected to a mobile Workstation 600 having Internet Access and digital camera photographs an outdoor scene and the corresponding image file 670 is processed (process 004) and transformed into image 650, which is stored in a directory 620, named origin, within device 600. The user connects the internet site 100 by means of mobile Workstation 600 and enters by means of process 010 described below. This image is stored in a sub-directory 1100 within terminal 200. Then the descriptors of image 650 are calculated (process 002) and stored in another sub-directory 1200. By means of process 006 described below the descriptors of reference image 650 are compared with each and every descriptor stored in the descriptor databank 400. The election, selection and ordering of the sets of descriptors of descriptor databank 400 having the higher similarity indexes to those descriptors contained in subdirectory databank 1200 (of reference image 650) consist the core of process 015 described below. Afterwards, and by means of process 016, the subdirectories containing information on corresponding images, texts, sounds, movies and/or links are identified within sub-directory databank 500, the ordering of process 015 being maintained. The ordered results are then displayed to the user at the mobile Workstation 600, by means of process 017 described below.
Example 3 - mobile device without internet
In another preferred embodiment, also making reference to figure 2, the execution environment of the system and method of the invention is completely incorporated within the mobile communication device. Accordingly, the execution environment lies within a chip 100 instead of an Internet site 100 and therefore the mobile device needs not to transmit any signal to its exterior. In this preferred example chip 100 has a very high memory capacity and contains a fixed directory 200 named terminal, where three databanks are firstly stored: an image databank 300, a descriptors databank 400 biunivocally associated with the image databank 300 by means of process 009 described below, and a sub-directories databank 500 biunivocally associated (process 009) with the image databank 300 and with the descriptors databank 400. The descriptors databank 400 is built/formed by process 002 described below. In the subdirectories databank 500, each sub-directory is biunivocally associated (process 009) with an image from databank 300 and contains a set of data of images, texts, sounds, movies and links related to/with the corresponding image.
An user connected to a mobile Workstation 600 having chip 100 and digital camera photographs an outdoor scene and the corresponding image file 670 is processed (process 004) and transformed into image 650, which is stored in a directory 620, named origin, within device 600. Then the descriptors of image 650 are calculated (process 002) and stored in another sub-directory 1200 within Workstation 600. By means of process 006 described below the descriptors of reference image 650 are compared with each and every descriptor stored in the descriptor databank 400. The election, selection and ordering of the sets of descriptors of descriptor databank 400 having the higher similarity indexes to those descriptors contained in sub-directory databank 1200 (of reference image 650) consist the core of process 015 described below. Afterwards, and by means of process 016, the sub-directories containing information on corresponding images, texts, sounds, movies and/or links are identified within sub-directory databank 500, the ordering of process 015 being maintained. The ordered results are then displayed to the user at the mobile Workstation 600, by means of process 017 described below.
Example 4 - Medicine - identification of breast tumors
Using any of the preceding examples described above, an oncology society builds an internet site 100 containing a fixed directory 200, where three databanks are firstly stored: a computer tomography image databank 300 comprising non-tumor, benign tumor and malign tumor images; a descriptors databank 400 biunivocally associated with the image databank 300 by means of process 009 described below, and a sub-directories databank 500 biunivocally associated (process 009) with the image databank 300 and with the descriptors databank 400. The descriptors databank 400 is built/formed by process 002 described below. In the sub-directories databank 500, each sub-directory is biunivocally associated (process 009) with a computer tomography image databank 300 and contains a set of data of images, texts, sounds, movies and links related to/with the corresponding breast image.
A physician, such as those working far away from big cities, connected to a computer 600 performs a computer tomography in a female patient; the corresponding image 650 is stored in a directory 620. Said user connects site 100 and then enters the image file 650 by means of process 010 described below. This image is stored in a sub-directory 1100 of terminal 200. Then its descriptors are calculated (process 002) and stored in another sub-directory 1200. By means of process 006 described below the reference image descriptors 650 are compared to each descriptor stored in the descriptor databank 400. The election, selection and ordering of the set of descriptors of the descriptor databank 400 having higher similarity indexes in relation to those descriptors of sub-directory 1200 (of reference image 650) constitute the process 015 described below. Afterwards, and by means of process 016, the sub-directories containing information on corresponding tomography-related images, texts, sounds, movies and/or links, so as to indicate the most likely diagnostic/prognosctic are identified in the subdirectory databank 500, maintaining the order obtained by process 015. Then these results are displayed to the user in Workstation 600, by means of process 017 described below.
The processes referred to in examples 1, 2, 3 and 4 above are described in further details below:
Process 002 - consists of a process for obtaining image descriptors and comprises two steps. First, the reference signal (an image in the examples above) is converted into a set of integer numbers representing the features of the reference signal (image features such as color, opacity etc). Those skilled in the art will know that several ways of performing such conversion are available, the JAVA language functions being preferred in the present invention. Second, the descriptors are calculated using the integer numbers of the first step. Those skilled in the art will know that several sets of descriptors can be used, such as mean, standard deviation, kurtosis etc Process 004 - consists of a process for filtering and/or transforming and/or processing of the entry signal can be performed by several means and comprises three steps. First, the entry signal is converted into a set of integer numbers representing the features of said signal. Those skilled in the art will know that several ways of performing such conversion are available, the JAVA language functions being preferred in the present invention. Second, the set of integer numbers is altered in a specific fashion, the JAVA language functions also being preferred in the present invention. Third, the new set of integers obtained by said transformation is reconverted into a new signal, now referred to as reference signal (in the preceding examples a reference image). The JAVA language functions are also preferred in the present invention. The so obtained reference signal may therefore enhance or diminish certain features of the original entry signal, so as to drive or improve the foregoing searches. Process 009 - consists of a process for biunivocal association between an image and the corresponding set of descriptors. This association is obtained by the association of the image name in the same descriptors directory. Those skilled in the art will know that several ways of performing such association are available. Process 010 - consists of a process for sending the image to the site. Those skilled in the art will know that several ways of performing such association are available, the creation of a file upload supported JSP page being preferred in the invention.
Processes 006 and 015 - consist of processes for comparing the reference descriptors with the descriptors of the descriptors databank, and the selection of the most related/similar descriptors. The search and selection of image(s) are performed according to the similarity between the reference descriptors with the descriptors available in the descriptors databank. Several techniques can be employed for obtaining similarity indexes between descriptors, such as, e.g., the method of the Euclidian distances with non-supervised neural networks, or supervised neural networks. The preferred embodiments of the invention the method of the Euclidian distances with non-supervised neural networks, along with a setup factor in the implemented algorithm. Process 016 - consists of a process for grouping corresponding sub-directories maintaining the similarity index obtained by the comparison between descriptors, as described above. Those skilled in the art will know that several ways of performing such process are available, the successive repetition of process 009 described above being preferred in the invention. Process 017 - consists of a process for displaying the search results for the user. Those skilled in the art will know that several ways of performing such association are available, the creation of a file download supported JSP page being preferred in the invention.
Example 5 - Police Activity Using any of the preceding examples 1-3 described above, a police unit builds a fingerprint image databank comprising the corresponding descriptors and criminal files. During police activity cops immobilize a suspect and obtain their fingerprints. By using a mobile communication device having camera and internet access the cops send the fingerprint images the police unit databank. These images may be optionally filtered by process 004, so as to include a processing category such as "black ink". The fingerprints are then easily and rapidly found in the fingerprint image databank, the criminal file of the suspect being sent back to the cop's mobile communication device. The cop can then arrest or not the suspect based in such information.
Example 6 - Information about an unknown person Using any of the preceding examples 1-3 described above, an Internet site such as Orkut contains personal information and/or photographs from people. The process for obtaining descriptors is used on said photographs.
A person A finds a person B and person A wants to know more information about person B, such as profession, habits etc. Person A photographs person B using a mobile phone: using the system and method of the invention, the image is optionally filtered so as only the face will be the reference image. Then the system accesses the Internet and performs the comparison between the descriptors of the reference image with the descriptors of the images available in a given or a series of Internet site(s); the system can optionally filter the site category for "search category Orkuf; the output may also be filtered so as only "profile category" results are displayed for the user.
Example 7 - Localizing long missing children
Using any of the preceding examples 1-3 described above, an Internet site contains personal information and/or photographs from missing children.
A photograph of a child which is missing for years is entered into the system. This image is transformed by means of process 004 which render the image older (the same number of years passed since the original photograph was taken). This process can be activated by entering "process category: ageing image X years". The "aged" image is then the reference image which activates the search, and the person can be found by processes similar to those described above .
Example 8 - Localizing mobile objects by using satellite images Using any of the preceding examples 1-3 described above, an Internet site contains satellite based images (such as Google-Earth) which stores low distance images (in which vehicle colors, for example, can be distinguished) and updates said images in short periods of time or even continuously. A person A wants to meet person B in a particular location. Person A says he/she will be driving a red car and sends a message with a photograph of his/her car taken from above to person B by its mobile device. Person B also has a mobile device, connects to said site and enters the photography of person A's car. The system thus searches the images on Earth having the corresponding category processing criteria. Person B can then follow person A's trajectory and can reach the meet location without getting late, or can meet person A at any point of his/her trajectory.
Example 9 - Supermarket
Using any of the preceding examples 1-3 described above, an Internet site contains product based images and their corresponding barcodes. A buyer photographs a product or its barcode, and the system provides pricing information in the nearby shops, nutritional facts, shelf life etc.
Example 10 - Tourist Guide
Using any of the preceding examples 1-3 described above the present invention is applied for the obtention of information about people and/or buildings and/or objects. Figure 3 shows the case where the user photographs a building. The operations described in more detail in examples 1-3 are performed so as the system results include a list of images, movies, texts and/or links related to the reference image. The user can select any kind of listed information, as well as refine the displayed results by entering "category results: movies". The skilled in the art will readily appreciate the value and advantages of the invention, and will understand that it is applicable to several fields in addition to the ones exemplified above. Those skilled in the art will also understand that other ways of performing the invention are enabled from the teachings of the above description, and such variations are to be considered as within the spirit of the invention and within the scope of the appended claims.

Claims

Claims
SYSTEM AND METHOD FOR NET SEARCHES ACTIVATED BY DIGITAL AND/OR ANALOGICAL SIGNALS
1- System and method for net searches activated by digital and/or analogical signals characterized by comprising: a) means for filtering/transforming the entry signals so as the corresponding filtered/transformed signals are reference signals which activate the searches; b) means for obtaining/extracting descriptors of the reference signal; c) means for storing said descriptors in a descriptor databank; d) means for comparing said descriptors with descriptors of other signals; e) means for selecting the set of descriptors which are considered similar/related; f) means for storing information about said signals; g) means for selecting sub-directories from descriptor databank corresponding to the selected descriptors; and h) means for displaying the search results to the user.
2 - System and method, according to claim 1 , characterized by further creating a databank comprising images, texts, sounds, movies and/or internet links which are related to the entry signals.
3 - System and method, according to claim 1 or 2, characterized by further training the system and testing the filed signals.
4 - System and method, according to claim 1 , characterized by the fact that said descriptors constitute the representation of the features of an image, including mean, variance and/or kurtosis of attributes of color and opacity of the corresponding pixels. 6 - System and method, according to claim 5, characterized in that said means for obtaining/extracting descriptors of the reference signal comprises JAVA language specific functions.
7 - System and method, according to claim 3, characterized in that said training is performed by Artificial Neural Networks.
8 - System and method, according to any preceding claim, characterized in that said means for comparing said descriptors with descriptors of other signals comprise the Euclidian Distances method.
9 - System and method, according to any preceding claim, characterized in that said set of reference signals is obtained by a mobile communication device.
10 - System and method, according to any preceding claim, characterized in that said search result is displayed to a mobile communication device.
11 - System and method, according to any preceding claim, characterized in that said search results comprise images, movies, sounds, texts, internet links or combinations thereof.
12 - System and method, according to any preceding claim, characterized in that the capture/obtention of reference signals and the search for corresponding information are both performed in the same mobile communication device.
13 - System and method, according to any preceding claim, characterized in that more than a signal can serve as reference signal, this multiple reference signals comprising one or more categories of signals, including image, sound, text, radio waves, or any other information which is convertible into digital and/or analogical signals.
PCT/BR2007/000115 2006-05-15 2007-05-15 System and method for net searches activated by digital and/or analogical signals WO2007131311A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
BRPI060.2471-8 2006-05-15
BRPI0602471-8A BRPI0602471A (en) 2006-05-15 2006-05-15 system and method for performing network search activated by digital and / or analog signals

Publications (2)

Publication Number Publication Date
WO2007131311A2 true WO2007131311A2 (en) 2007-11-22
WO2007131311A3 WO2007131311A3 (en) 2009-06-11

Family

ID=38694251

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/BR2007/000115 WO2007131311A2 (en) 2006-05-15 2007-05-15 System and method for net searches activated by digital and/or analogical signals

Country Status (2)

Country Link
BR (1) BRPI0602471A (en)
WO (1) WO2007131311A2 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9836669B2 (en) 2016-02-22 2017-12-05 International Business Machines Corporation Generating a reference digital image based on an indicated time frame and searching for other images using the reference digital image
WO2019014649A1 (en) * 2017-07-14 2019-01-17 Memorial Sloan Kettering Cancer Center Weakly supervised image classifier

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6026411A (en) * 1997-11-06 2000-02-15 International Business Machines Corporation Method, apparatus, and computer program product for generating an image index and for internet searching and querying by image colors
US20040229611A1 (en) * 2003-05-12 2004-11-18 Samsung Electronics Co., Ltd. System and method for providing real-time search information

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH09146907A (en) * 1995-11-28 1997-06-06 Nec Corp Pattern learning method

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6026411A (en) * 1997-11-06 2000-02-15 International Business Machines Corporation Method, apparatus, and computer program product for generating an image index and for internet searching and querying by image colors
US20040229611A1 (en) * 2003-05-12 2004-11-18 Samsung Electronics Co., Ltd. System and method for providing real-time search information

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9836669B2 (en) 2016-02-22 2017-12-05 International Business Machines Corporation Generating a reference digital image based on an indicated time frame and searching for other images using the reference digital image
US10607110B2 (en) 2016-02-22 2020-03-31 International Business Machines Corporation Generating a reference digital image based on an indicated time frame and searching for other images using the reference digital image
WO2019014649A1 (en) * 2017-07-14 2019-01-17 Memorial Sloan Kettering Cancer Center Weakly supervised image classifier
US10685255B2 (en) 2017-07-14 2020-06-16 Memorial Sloan Kettering Cancer Center Weakly supervised image classifier

Also Published As

Publication number Publication date
BRPI0602471A (en) 2008-03-25
WO2007131311A3 (en) 2009-06-11

Similar Documents

Publication Publication Date Title
US9189554B1 (en) Providing images of named resources in response to a search query
US9235733B2 (en) Mobile biometrics information collection and identification
US7653702B2 (en) Method for automatically associating contextual input data with available multimedia resources
CN104239408B (en) The data access of content based on the image recorded by mobile device
US20140003714A1 (en) Gesture-based visual search
EP2883158B1 (en) Identifying textual terms in response to a visual query
US8755837B2 (en) Methods and systems for content processing
US9104915B2 (en) Methods and systems for content processing
US8718383B2 (en) Image and website filter using image comparison
US20070288453A1 (en) System and Method for Searching Multimedia using Exemplar Images
US9043268B2 (en) Method and system for displaying links to search results with corresponding images
US20090061949A1 (en) System, method and mobile unit to sense objects or text and retrieve related information
US20130332451A1 (en) System and method for correlating personal identifiers with corresponding online presence
US20080065606A1 (en) Method and Apparatus for Searching Images through a Search Engine Interface Using Image Data and Constraints as Input
KR20140093957A (en) Interactive multi-modal image search
KR101835333B1 (en) Method for providing face recognition service in order to find out aging point
KR20060026924A (en) Tagging method and system for digital data
CA2711143A1 (en) Method, system, and computer program for identification and sharing of digital images with face signatures
US10380164B2 (en) System and method for using on-image gestures and multimedia content elements as search queries
WO2005050370A2 (en) System and method of searching for image data in a storage medium
US20140344238A1 (en) System And Method For Accessing Electronic Data Via An Image Search Engine
US20180330206A1 (en) Machine-based learning systems, methods, and apparatus for interactively mapping raw data objects to recognized data objects
EP3396566A1 (en) Method, information processing apparatus and program
CN111324768A (en) Video searching system and method
CN113869063A (en) Data recommendation method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 07719283

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 07719283

Country of ref document: EP

Kind code of ref document: A2