JP2005182495A - Information processor, information processing method, recording medium and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To narrow down information that a user desires, to retrieve and display it and to speedily grasp an information amount at every type of information. <P>SOLUTION: A picture extracting part 301 extracts a picture near a position detected by a pointer position detecting part 301a among inputted pictures, and supplies it to a recognition part 302. The recognition part 302 recognizes the picture by using recognition DB 303, and supplies a recognition result to a retrieval part 304. The retrieval part 304 retrieves a retrieval picture file and peculiar data, which correspond to the recognition result, by using retrieval DB 305, and supplies them to a display control part 501. An information amount calculation part 501a of the display control part 501 calculates the information amount at every information amount based on the retrieval result supplied from the retrieval part 304. The display control part 501 creates a window displaying the retrieval picture file and peculiar data based on the calculation result calculated by the information amount calculation part 501a so as to display it. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an information processing apparatus and method, a recording medium, and a program, and in particular, acquires information about the periphery, searches for information based on the acquired periphery information, and further presents the information. , A recording medium, and a program.

  2. Description of the Related Art Conventionally, a search engine that recognizes an image captured by a camera by some kind of image recognition technology and presents corresponding stored data has been proposed.

  In the above search engine, for example, a character string included in a captured image is recognized by an OCR (Optical Character Reader), and the meaning of a word is searched based on the recognized text character string. Then, the map data around the place name can be searched and presented, and further, if it is a URL (Univarsal Resource Locator), the corresponding HP (Home Page) can be opened.

  Also, for example, when a captured image includes a person's face, the search engine described above recognizes which person's face registered in advance is the person's face included in the image. There is a technique for presenting related information (see, for example, Patent Document 1).

JP 2002-157596 A

  However, in this method, the search engine can search only based on information on the image captured by the camera. For example, information is obtained based on various information included in the image captured by the camera. As a result of the search, a large amount of information may be searched, and as a result, there is a possibility that the information desired by the user cannot be accurately searched (found).

  In addition, even if the user knows that the searched information is enormous, the user cannot grasp how much information is, and whether the searched information should be searched with his own eyes, There was a problem that it was not possible to determine whether or not to narrow down by another method.

  The present invention has been made in view of such a situation. In particular, the present invention makes it possible to search and display information desired by the user, and to quickly grasp the amount of information for each type of information. It is to make.

  An information processing apparatus according to the present invention includes an acquisition unit that acquires input information, a recognition unit that recognizes input information acquired by the acquisition unit, a search unit that searches presentation information based on a recognition result of the recognition unit, and a search The present invention is characterized by comprising calculation means for calculating the information amount of the presentation information searched by the means, and screen generation means for generating a screen for displaying the presentation information based on the information amount calculated by the calculation means.

  The acquisition unit can acquire an image as input information, and the recognition unit can recognize the image acquired by the acquisition unit.

  Position detection means for detecting the position of the pointer on the image may be further provided, and the acquisition means may acquire an image near the position detected by the position detection means as input information. it can.

  The acquisition unit can acquire voice as input information, and the recognition unit can recognize the voice acquired by the acquisition unit.

  The screen generating means can generate a screen for displaying the presentation information with an icon having a size based on the amount of information calculated by the calculating means.

  The information processing method of the present invention searches for presentation information based on an acquisition step for acquiring input information, a recognition step for recognizing input information acquired in the processing of the acquisition step, and a recognition result in the processing of the recognition step. A search step, a calculation step for calculating the information amount of the presentation information searched in the processing of the search step, and a screen generation step for generating a screen for displaying the presentation information based on the information amount calculated in the processing of the calculation step; It is characterized by including.

  The recording medium program of the present invention includes an acquisition control step for controlling the acquisition of input information, a recognition step for recognizing input information whose acquisition is controlled in the process of the acquisition control step, and a recognition result in the process of the recognition step. A search step for searching for presentation information based on the calculation step, a calculation step for calculating the information amount of the presentation information searched in the processing of the search step, and a screen for displaying the presentation information based on the information amount calculated in the processing of the calculation step And a screen generation step for generating.

  The program of the present invention is presented based on an acquisition control step for controlling the acquisition of input information, a recognition step for recognizing input information whose acquisition is controlled in the process of the acquisition control step, and a recognition result in the process of the recognition step A search step for searching for information, a calculation step for calculating the information amount of the presentation information searched in the processing of the search step, and a screen for displaying the presentation information based on the information amount calculated in the processing of the calculation step are generated The screen generation step is executed by a computer.

  In the information processing apparatus, method, and program of the present invention, input information is acquired, the acquired input information is recognized, presentation information is searched based on the recognition result, and an information amount of the searched presentation information is calculated. Then, a screen on which the presentation information is displayed based on the calculated information amount is generated.

  The information processing apparatus of the present invention may be an independent apparatus or a block that performs information processing.

  According to the present invention, it is possible to quickly search and display information desired by a user, and to quickly grasp the amount of information for each type of information.

  Embodiments of the present invention will be described below. The correspondence relationship between the invention described in this specification and the embodiments of the invention is exemplified as follows. This description is intended to confirm that the embodiments supporting the invention described in this specification are described in this specification. Therefore, although there is an embodiment which is described in the embodiment of the invention but is not described here as corresponding to the invention, it means that the embodiment is not It does not mean that it does not correspond to the invention. Conversely, even if an embodiment is described herein as corresponding to an invention, that means that the embodiment does not correspond to an invention other than the invention. Absent.

  Further, this description does not mean all the inventions described in this specification. In other words, this description is for the invention described in the present specification, which is not claimed in this application, that is, for the invention that will be applied for in the future or that will appear and be added by amendment. It does not deny existence.

  That is, the information processing apparatus according to the present invention includes an acquisition unit (for example, the image extraction unit 301 in FIG. 22) that acquires input information and a recognition unit (for example, the recognition in FIG. 22) that recognizes the input information acquired by the acquisition unit. Unit 302), search means for searching for presentation information based on the recognition result of the recognition means (for example, search section 304 in FIG. 22), and calculation means for calculating the amount of information of the presentation information searched by the search means (for example, 22, and an information amount calculation unit 501a in FIG. 22 and a screen generation unit (for example, the display control unit 501 in FIG. 22) that generates a screen for displaying the presentation information based on the information amount calculated by the calculation unit. It is characterized by.

  The information processing method of the present invention includes an acquisition step for acquiring input information (for example, step S42 in the flowchart of FIG. 23) and a recognition step for recognizing input information acquired in the processing of the acquisition step (for example, FIG. 23). In step S45 in the flowchart), a search step for searching for presentation information based on the recognition result in the process in the recognition step (for example, step S48 in the flowchart in FIG. 23), and a search step. A calculation step for calculating the information amount of the presented information (for example, processing in step S49 in the flowchart of FIG. 23) and screen generation for generating a screen for displaying the presentation information based on the information amount calculated in the calculation step processing Step (for example, the process of step S50 in the flowchart of FIG. 23).

  The recording medium and the program invention are the same as the information processing method, and the description thereof is omitted.

  FIG. 1 shows a configuration example of a service providing system to which the present invention is applied.

  In the service providing system of FIG. 1, an access point 4 and image data management servers 5 to 7 are connected to a network 1 including the Internet. Further, the information processing apparatus 2 performs wireless communication with the access point 4 by a wireless local area network (LAN) and is connected to the network 1 via the access point 4. The information processing apparatus 3 performs Bluetooth communication with the information processing apparatus 2. Note that communication between the information processing device 3 and the information processing device 2 can be performed by a wireless LAN. Further, hereinafter, the information processing device 2 is also referred to as a parent device 2, and the information processing device 3 is also referred to as a child device 3.

  The image data management servers 5 to 7 store a database similar to a recognition DB (database) 303 and a search DB (database) 305, which will be described later, via the network 1, and the information processing apparatus 2 or 3 However, when performing recognition processing or search processing based on image data, the information processing apparatus 2 or 3 provides database information similar to the recognition DB (database) 303 and the search DB (database) 305. Note that the information included in the image data management servers 5 to 7 may be stored in the information processing apparatuses 2 and 3. Further, although three image data management servers 5 to 7 are shown in FIG. 1, the number of image data management servers 5 to 7 is not limited to this, and may be other than three.

  The parent device 2 transmits information related to the recognition DB or the search DB from the image data management server 5 to the child device 3.

  The subunit | mobile_unit 3 is the image imaged by the imaging part 121 (FIG. 7) mentioned later, the image acquired from the main | base station 2 (an image acquired from the other apparatus via the main | base station 2), or self The image stored in the storage unit 154 (FIG. 7) is displayed on the display 101. The subunit | mobile_unit 3 searches the information relevant to the image of the area | region of the vicinity designated with the pointer among the images displayed on the display 101 (FIG. 7), and displays it on the display 101. FIG.

  The handset 3 can be detached from the base unit 2. 2 and 3 show examples of attaching and detaching the slave unit 3. FIG. 2 shows an example in which the slave unit 3 is mounted inside the master unit 2. The master unit 2 and the slave unit 3 have, for example, a substantially hexagonal shape. The slave unit 3 has one of its corners overlapped with one corner of the master unit 2 as shown in FIG. The base unit 2 is attached.

  FIG. 3 shows an example in which the child device 3 is detached from the parent device 2. As shown in FIG. 3, the handset 3 can be detached from the base unit 2. The parent device 2 and the child device 3 are of a size that can be carried by the user. For example, the parent device 2 has a size that can be carried in a bag or the like, and the child device 3 has a size that fits in a palm.

  FIG. 4 shows an example of the size of the handset 3. As shown in FIG. 4, the cordless handset 3 has a length between the opposing sides of 86 mm and a thickness of about 30 mm, which is a size that fits in the palm. FIG. 5 shows an appearance of the slave unit 3 and an internal configuration example. An example of the external configuration of the slave unit 3 is shown on the left side of FIG. 5, and an internal configuration example is shown on the right side. As shown on the left side of FIG. 5, on the surface of the child device 3, various displays, a display 101 that displays images captured by the imaging unit 121 (FIG. 6), and sound are collected. Microphone 102, temperature / humidity sensor 103 that measures ambient temperature and humidity, IC communication unit 104 that transmits / receives information to / from an IC (Integrated Circuit) card reader / writer by contact or non-contact, and outputs sound A speaker 105 is arranged. The display 101 is configured by an LCD (Liquid Crystal Display), for example. The display 101 is a so-called touch panel. By touching the display 101 with a finger or the like, the pointer can be moved on the image on the display 101 corresponding to the touched position, and the position on the image is designated. It is possible. Therefore, the display 101 also functions as the operation unit 156.

  For example, the IC communication unit 104 performs payment of electronic money through communication with an IC card reader / writer. Although not shown, an LED (Light Emitting Diode) 159 (see FIG. 7) is also arranged on the surface of the slave unit 3 in addition to this. Note that, as shown on the right side of FIG. 5, in the following description, the surface on the side where the display 101 of the slave unit 3 is disposed is the front surface, and the opposite side is the back surface.

  FIG. 6 illustrates an arrangement example on the back side inside the slave unit 3. As shown in FIG. 6, the handset 3 includes an imaging unit 121, a wireless LAN communication unit 122, a GPS antenna 123, vibrators 124-1 and 124-2, a GPS detection unit 125, a Bluetooth communication unit 126, a magnetism. A sensor 127, a three-axis motion sensor 128, a battery 129, and a heartbeat detection sensor 130 are arranged.

  The imaging unit 121 includes an imaging device such as a CCD (Charged Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and images a surrounding subject. This image sensor is installed on the back surface of the slave unit 3. The wireless LAN communication unit 122 performs communication by wireless LAN. The GPS antenna 123 receives a satellite signal from a position measurement satellite and supplies it to the GPS detection unit 125. Vibrators 124-1 and 124-2 incorporate a small motor, are arranged so as to be orthogonal to each other, and vibrate in the front-rear and left-right directions. The GPS detection unit 125 calculates the latitude and longitude of the current position of the slave unit 3 based on the satellite signal from the GPS antenna 123. In the following description, information such as longitude and latitude detected by the GPS detection unit 125 is also referred to as GPS information.

  The Bluetooth communication unit 126 transmits and receives information to and from the parent device 2 and the like by Bluetooth. The magnetic sensor 127 reads information by magnetism. The three-axis motion sensor 128 detects the three-dimensional posture angle of the child device 3 in real time. The battery 129 supplies power to each unit of the child device 3. The heartbeat detection sensor 130 is a sensor for detecting a user's heart rate.

  FIG. 7 is a block diagram illustrating an example of an electrical configuration inside the handset 3. In addition, about the part already demonstrated in the above, description is abbreviate | omitted suitably.

  The CPU (Central Processing Unit) 151 performs the operation of each unit of the slave unit 3 according to a program stored in a ROM (Read Only Memory) 152 or a program loaded from a storage unit 154 to a RAM (Random Access Memory) 153. Control and execute various processes. The RAM 153 also appropriately stores data necessary for the CPU 151 to execute various processes. The CPU 151 also includes a magnetic disk 171 (including a flexible disk), an optical disk 172 (including a CD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versatile Disk)), a magneto-optical disk 173 connected to the drive 170. (Including MD (Mini-Disc)) or a program and data recorded in the semiconductor memory 174 are read and executed as appropriate, and a predetermined program and data are recorded.

  The CPU 151, ROM 152, and RAM 153 are connected to each other via a bus 155. The bus 155 also includes a storage unit 154, a wireless LAN communication unit 122, a Bluetooth communication unit 126, an IC communication unit 104, an operation unit 156, a microphone 102, a heart rate detection unit 157, a temperature / humidity sensor 103, a GPS detection unit 125, and an imaging unit. 121, a 3-axis motion sensor 128, a magnetic sensor 127, vibrators 124-1, 124-2, an audio output control unit 158, a display 101, an LED 159, an RTC 160, and a drive 171 are connected.

  The operation unit 156 includes buttons, a jog dial, and the like, and accepts an operation from the user. The heart rate detection unit 157 calculates the user's heart rate based on the signal from the heart rate detection sensor 130. The sound output control unit 158 causes the speaker 105 to output sound corresponding to the supplied sound information (electrical signal). For example, the LED 159 emits light to direct the user's attention to the slave unit 3 when presenting information to the user.

  An RTC (Real Time Clock) 160 calculates the current time, generates time information necessary for processing, and supplies it to various devices.

  Next, FIG. 8 illustrates an example of the size of the parent device 2. As shown in FIG. 8, the base unit 2 has a length between opposing sides of 155 mm and a thickness of about 30 mm. For example, the base unit 2 can be stored in a bag or the like and portable. .

  FIG. 9 illustrates an example of an electrical configuration inside the parent device 2. In FIG. 9, a CPU 201 is configured by, for example, a Pentium (registered trademark) processor manufactured by Intel, and is connected to a front side bus (FSB) 202. A north bridge 203 is further connected to the FSB 202, and the north bridge 203 has an AGP (Accelerated Graphics Port) 204 and is connected to the hub interface 210.

  The north bridge 203 includes, for example, Intel's AGP Host Bridge Controller 440BX and the like, and controls the CPU 201 and RAM (Random Access Memory) 209 (so-called main memory). Further, the north bridge 203 controls the video controller 205 via the AGP 204. The video controller 205 controls the LCD 206 or a VGA (Video Graphics Array) display (hereinafter referred to as VGA 207).

  The video controller 205 receives data (such as image data or text data) supplied from the CPU 201 and generates image data corresponding to the received data, or a video memory (not-ready) containing the received data as it is. Stored in the figure). The video controller 205 causes the LCD 206 or the VGA 207 to display an image corresponding to the image data stored in the video memory. The LCD 206 or VGA 207 displays an image or text based on the data supplied from the video controller 205.

  The north bridge 203 is further connected to a cache memory 208. The cache memory 208 is configured by a memory that can execute a higher-speed writing or reading operation than the RAM 209 such as an SRAM (Static RAM), and caches (temporarily stores) a program or data used by the CPU 201. .

  Note that the CPU 201 has a cache that can operate at a higher speed than the cache memory 208 that is primary, and that is controlled by the CPU 201 itself.

  The RAM 209 is composed of, for example, a DRAM (Dynamic RAM), and stores a program executed by the CPU 201 or data necessary for the operation of the CPU 201. Specifically, for example, the RAM 204 stores an operating system, an Internet program, and the like loaded from an HDD (Hard Disk Drive) 223 when the activation is completed.

  The operating system (OS (Operating System)) controls basic computer operations such as the so-called Windows (registered trademark) XP of Microsoft or the so-called Mac OS (registered trademark) of Apple Computer. It is a program to do.

  The north bridge 203 is also connected to the south bridge 211 via the hub interface 210. The south bridge 211 is composed of, for example, PIIX4E manufactured by Intel Corporation, and includes an AC link interface 211A, a USB (Universal Serial Bus) interface 211B, an IDE (Integrated Drive Electronics) interface 211C, and a PCI (Peripheral Component Interconnect) interface 211D. , An LPC (Low Pin Count) interface 211E, a LAN interface 211F, and the like are incorporated.

  The south bridge 211 controls various I / Os (Input / Output) such as control of devices connected via devices connected to the AC link bus 212, USB bus 217, and IDE bus 222.

  A modem 213 and a sound controller 214 are connected to the AC link bus 212. When the modem 213 is connected to the public line network, the modem 213 executes communication processing via the public line network or the Internet (both not shown). The sound controller 214 captures sound from the microphone 215, generates data corresponding to the sound, and outputs the data to the RAM 209. The sound controller 214 drives the speaker 216 to cause the speaker 216 to output sound.

  A USB connector 218 is connected to the USB bus 217 of the south bridge 211 so that various USB devices can be connected. In addition, a memory stick slot 219 and a Bluetooth communication unit 221 are provided via the USB bus 217. A Memory Stick (trademark) 220 is connected to the Memory Stick slot 219.

  The memory stick 220 is a type of flash memory card developed by Sony Corporation, the applicant of the present application. This memory stick 220 stores a flash memory device, which is a kind of EEPROM (Electrically Erasable and Programmable Read Only Memory), which is a nonvolatile memory that can be electrically rewritten and erased in a small and thin plastic case. Various data such as images, sounds, music, etc. can be written and read via the 10-pin terminal.

  The Bluetooth communication unit 221 performs communication according to the Bluetooth standard. The USB interface 221B transmits data to an external device connected via the USB bus 217 and receives data from the device.

  The IDE interface 221C includes two IDE controllers, a so-called primary IDE controller and a secondary IDE controller, a configuration register, and the like (both not shown).

  The HDD 223 is connected to the primary IDE controller via the IDE bus 222. Further, when a so-called IDE device such as a CD-ROM drive 224 or an HDD (not shown) is attached to another IDE bus, the attached IDE device is electrically connected to the secondary IDE controller. .

  The wireless LAN communication unit 225 is connected to the network by wireless LAN communication such as IEEE802.11a or IEEE802.11b. The LAN interface 211F transmits data to a network connected to the wireless LAN communication unit 225 and receives data.

  A BIOS (Basic Input Output System) 252, an I / O (Input / Output) interface 253, and a controller 256 are connected to the LPC bus 251. The BIOS 252 is a program group that collects basic operation instructions of the parent device 2, and is stored in, for example, a ROM (Read Only Memory). The BIOS 252 controls data transfer (input / output) between the OS or application program and peripheral devices.

  A serial terminal 254 and a parallel terminal 255 are connected to the I / O interface 253, and data is exchanged with devices connected to the respective terminals. A jog dial 259 and a battery 260 are connected to the controller 256. A keyboard 258, a mouse 257, and the like can also be connected to the controller 256.

  A PC card interface 229 and an IEEE1394 interface 227 are connected to the PCI bus 226. The PC card interface 229 supplies the data supplied from the device (card) connected to the slot 230 to the CPU 201 or the RAM 209 and outputs the data supplied from the CPU 201 to the card connected to the PC card slot. . A drive 231 can be connected to the PCI bus 226 via the slot 33 and the PC card interface 78.

  The drive 231 reads data recorded on the magnetic disk 241, the optical disk 242, the magneto-optical disk 243, or the semiconductor memory 244 that is loaded, and supplies the read data to the RAM 209. In addition, data generated by the processing of the CPU 201 can be stored in the magnetic disk 241, the optical disk 242, the magneto-optical disk 243, or the semiconductor memory 244 that is mounted on the drive 231.

  The IEEE1394 interface 227 transmits and receives data conforming to the IEEE1394 standard (data stored in a packet) via the IEEE1394 port 228.

  Next, functions realized by the information processing apparatus 3 described with reference to FIG. 7 will be described with reference to the functional block diagram of FIG.

  Note that the functions realized by the information processing apparatus 3 shown in FIG. 10 will be described on the assumption that each block is realized by software. However, the function is realized by hardware having the function of each block. Needless to say. Therefore, the functional block diagram shown in FIG. 10 may be recognized as a functional block realized by software, or may be recognized as a functional block realized by hardware.

  The image extraction unit 301 extracts an image in the vicinity of the detected position based on the position information on the image displayed on the display 101 by the pointer position detection unit 301 a and supplies the extracted image to the recognition unit 302. More specifically, the pointer position detection unit 301a detects coordinates indicating the pixel position of the pointer on the display 101 (for example, coordinates indicating the pixel position of the tip portion of the pointer). For example, when the coordinate position of the pixel detected by the pointer position detection unit 301a is (X, Y), the image extraction unit 301 satisfies (X−α <X <X + α, Y−β <Y <Y + β). Extract a range image. Here, α and β are the number of pixels indicating the range in the vicinity of the pointer on the display 101. For example, when a person's face is displayed on the display 101 and the pointer is present on the face part, the image extraction unit 301 has a size of 2α pixels × 2β pixels as an image in the vicinity of the pointer. Then, the face image of the person is extracted and supplied to the recognition unit 302. Since α and β can be set arbitrarily, the size of the extracted image can be set arbitrarily. Therefore, the size of the extracted image can be set to, for example, the size displayed as a thumbnail image or one window.

  The recognition unit 302 recognizes an image using the image data for recognition recorded in the recognition DB 303 based on the image supplied from the image extraction unit 301, and supplies the recognition result to the search unit 304.

  More specifically, the recognition DB 303 is composed of a plurality of DBs for each category. That is, for example, in the DB of the category “company”, information on a person corresponding to each image data is recorded together with image data such as a facial photograph of a person existing in the company. In addition, in the DB of the family category, information such as a person corresponding to each image data is recorded together with image data such as a facial photograph of a person existing at home.

  Based on the current time information supplied from the RTC 160 and the GPS information supplied from the GPS detection unit 125, the recognition unit 302 makes an inquiry to the recognition DB 303 to determine which category of DB to use. The image data recorded in the DB and the image data extracted by the image extraction unit 301 are compared, and image ID information corresponding to the matching image data is supplied to the search unit 304 as a recognition result.

  The search unit 304 searches for a search image file associated with the recognized information from the image data recorded in the search DB 305 based on the recognition result supplied from the recognition unit 302 and displays the search control unit 306. To supply. More specifically, the search DB 305 stores a search image file corresponding to the image data used for recognition and unique data related to the person, and a search image file corresponding to the person information as a recognition result, The unique data is retrieved, and the retrieved search image file and unique data are supplied to the display control unit 306.

  The display control unit 306 generates an image that displays the searched image file and the unique data as a window, for example, and displays the image overlapping the image on the display 101.

  Next, data display processing by the information processing apparatus (child device) 3 in FIG. 10 will be described with reference to the flowchart in FIG.

  In step S1, the pointer position detection unit 301a of the image extraction unit 301 uses, for example, an arrow-shaped pointer as coordinate information indicating the position of the pointer on the image based on information from the display 101 functioning as the operation unit 156. Coordinates (X, Y) indicating the position of the tip portion are detected.

  In step S <b> 2, the image extraction unit 301 uses, for example, an image in the vicinity of the coordinates of the pointer, for example, in the horizontal direction based on the coordinate information (X, Y) indicating the pixel position of the pointer detected by the pointer position detection unit 301 a. Thus, an image in the range of X−α <X <X + α and the range of Y−β <Y <Y + β with respect to the vertical direction is extracted and supplied to the recognition unit 302. That is, the image extraction unit 301 extracts an image of 2α pixels × 2β pixels as an image in the vicinity of the pointer with (X, Y) that is the coordinate position of the pointer as the center, and supplies the image to the recognition unit 302.

  In step S3, the recognizing unit 302 acquires the recognition DB specifying information, and in step S4, identifies which category of the recognition DB 303 is to be used among the recognition DBs 303 including a plurality of category DBs.

  More specifically, as shown in FIG. 12, the recognition DB 303 includes a company DB 303a in which reference face images of persons (supervisors, colleagues, subordinates, etc.) belonging to the company where the user works, and family reference faces. The family DB 303b or the like in which images are registered is configured as a collection of DBs in which reference face images of persons are registered by category for each location based on user relationships. In this case, the number of DBs included in the recognition DB 303 is two in FIG. 12. However, the number of DBs may be more than that, and a category may be set for each location. However, for example, it may be possible to set various categories such as for every time period often met or sex.

  These company DB 303a and family DB 303b are selected according to the situation in which recognition processing is performed, as will be described later. Each DB stores and manages a plurality of pieces of information including at least a recognition ID and reference face image data of a person associated therewith. In order to avoid a decrease in recognition rate, it is preferable to register reference face image data of about 10 people for one category of DB.

  Further, the recognizing unit 302 acquires the current time information from the RTC 160 as the recognizing DB specifying information in the recognizing DB 302 described with reference to FIG. 12, and the GPS detecting unit 125 obtains the GPS information of the current position. get. Then, the recognizing unit 302 determines which category of the DB for recognition 303 is to be used based on the acquired time information and GPS information. In the case of FIG. 12, the recognition unit 302 selects either the company DB 303a or the family DB 303b based on the acquired time information and GPS information.

  More specifically, for example, when the time zone of the time information is morning or night and the GPS information is in the vicinity of the home, the imaging unit 121 belongs to the family DB 303b because the possibility of contact with the family is high. Since there is a high possibility that a family image is taken, the family DB 303b is selected. On the other hand, when the time zone of the time information is daytime and the GPS information is in the vicinity of the company, there is a high possibility that the person will belong to the company DB 303a, so the imaging unit 121 may capture an image belonging to the company DB 303a. Therefore, the company DB 303a is selected (specified). In this way, by setting the DB by category, it is possible to improve the recognition rate and the recognition speed.

  In step S <b> 5, the recognition unit 302 finds a matching image based on the difference between the captured image and the image stored in the recognition DB 302 based on the image supplied from the image extraction unit 301. The image ID of the image that is recognized and found as the recognition result is supplied to the search unit 304. More specifically, the recognizing unit 302 realizes the two supplied images by determining whether the person in the images is the same person or another person. The determination is made based on the difference image between the two face images.

  That is, in identifying a face, first, a difference between two images of the same person is obtained in advance as a learning stage, and referring to this, whether or not the images are the same from the difference result of the two images ( In this case, it is determined whether or not they are the same person.

  In the learning stage, the difference between two images of the same subject (for example, the same person) is obtained, the same difference image group (intra-personal class) is generated, and the eigenvalue expansion is performed to obtain a feature space. Has acquired a unique space of intra-personal class. For example, as a technique of such learning method, a report of B. Moghaddam et al. (B. Moghaddam, W. Wahid, A. Pentland, "Beyond Eigenfaces: Probabilistic Matching for Face Recognition", MIT Media Lab. Perceptual Computing Section Technical Report No. 443, 1998.).

  The recognition process realized through such learning is as follows. First, in order to correctly obtain the difference between the two images, it is necessary to accurately obtain the position of the subject on the image (in this case, the position of the person's face). In particular, in the case of a face, the positions of the eyes and nose in the face differ depending on the person, so that these positions need to be normalized. For example, here, eye-nose position identification is performed by matching the proper eye with the proper nose on the obtained image, and affine transformation is performed so that the detected eye and nose are in a predetermined position. Performs rotation and expansion / contraction (morphing process).

  Further, through such a morphing operation, a difference between the image extracted by the image extraction unit 301 and the image used as the image data in the recognition DB 303 is generated.

  Next, which image is stored in the recognition DB 302 is specified based on the difference image obtained by the difference as described above and the eigenspace described above. Specifically, the distance between the difference image and the intra-personal class space (distance from feature space, DFFS) and the distance from the point where the difference image is projected to the intra-personal space and the origin (distance in feature space, DIFS) are calculated. To do. For example, assuming that difference images of the same person have a normal distribution in the intra-personal class space, the added value of DFFS and DIFS (DFFS + DIFS) represents the distance from the distribution. And the probability that it is the same person will become high, so that this distance is small. By using such a relationship, the image is recognized by determining the image having the highest probability of being the same as the image extracted by the image extraction unit 301. For example, when the difference value is small, the difference image is not sharp and is obtained as a so-called flat image.

  More specifically, as shown in FIG. 13, the image extraction unit 301 extracts an image G1 from the input image G0 by the process of step S2. Then, the recognition unit 302 masks the four corners of the image as in the image G2, removes the background portion, and further performs a morphing operation to acquire the image G3.

Then, as shown in FIG. 14, the recognition unit 302 stores the image G3 obtained by the morphing operation in advance in the selected category DB in the recognition DB 303, for example, the images DG1, DG2, and the like. A difference from DG3,... Is obtained. Thereby, difference images SD1, SD2, SD3,... Are obtained. In this example, the images whose faces are included in the images G1 to G3 have difference values P of P = 2.79e ⁻⁶⁴ (P = 2.79e−64 in the figure) and 1.53e ⁻⁶⁹ (FIG. In the figure, P = 1.53e−69) and 9.73e ⁻⁷² (in the figure, P = 9.73e ⁻⁷² ), and the same person as the third image DG3 in which P = 9.73e ⁻⁷² A certain probability is the highest result. Therefore, the recognition unit 302 recognizes that the person shown in the image extracted by the image extraction unit 301 is the person of the image DG3, and supplies the recognition result to the search unit 304. In the above example, the case of recognizing a person imaged in an image has been used as an example. However, it is also possible to recognize an object by a similar method. Moreover, it is also possible to recognize a character etc. as text data by the same method.

  Now, the description returns to the flowchart of FIG.

  In step S <b> 6, the search unit 304 determines whether the image can be recognized based on the presence / absence of the image ID supplied from the recognition unit 302. That is, the recognition unit 302 may be image data registered in advance in the recognition DB 303 in order to recognize a person (including not only a person but also an object or a character) captured in the image. As a premise, registered image data may not include image data that matches the extracted image supplied from the image extraction unit 301. In such a case, the recognition unit 302 cannot recognize the image, and as a result, there is no image ID as a recognition result. Therefore, the search unit 304 determines that the image has not been recognized. The process proceeds to step S7.

  In step S <b> 7, the search unit 304 registers the image supplied from the image extraction unit 301 to the recognition unit 302 in the DB of the category specified in step S <b> 4 in the recognition DB 302. Returning to step S1, the subsequent processing is repeated. More specifically, the search unit 304 controls the display control unit 306 to display a dialog box for registering unique data or the like for the input image, and allows the user to input such information. The unique data input by the user and the input image are registered in the search DB 305 in association with the recognition ID. In this way, new images can be accumulated in the search DB 305 described later.

  On the other hand, if it is determined in step S6 that the image can be recognized by the above-described method, the search unit 304 responds based on the image ID that is the recognition result supplied from the recognition unit 302 in step S8. The search image file to be searched is searched from the search DB 305.

  Here, the search DB 305 will be described. The search DB 305 is configured, for example, as shown in FIG. That is, in FIG. 15, an image ID column, a search image file column, and a unique data column are provided from the left.

  Furthermore, in the image ID column, IDs corresponding to the image data recorded in the recognition DB 303 are shown. In this case, 1, 2, 3 are shown as image IDs from the top. Further, in the search image file column, "ID1-1.jpg; ID1-2.jpg", "ID2-1.jpg; ID2-2.tiff; ID2-3.mpeg", and "picture01. jpg; picture02.jpg; picture03.jpg; picture04.jpg; picture05.jpg "are shown. Furthermore, in the specific data column, “Ichiro Tanaka; ichiro_tanaka@ABC.com”, “Keiko Yamada; ○ △ Commercial Accounting Department”, and “Taro Yamada; taro_yamada@zony.com; It is shown.

  That is, in FIG. 12, in the search DB 305, the search image file corresponding to the image ID = 1 is “ID1-1.jpg; ID1-2.jpg”, and the unique data corresponding to the image ID = 1 is , “Ichiro Tanaka; ichiro_tanaka@ABC.com”. Similarly, the search image file corresponding to the image ID = 2 is “ID2-1.jpg; ID2-2.tiff; ID2-3.mpeg”, and the unique data corresponding to the image ID = 2 is “ It is shown that it is Keiko Yamada; Further, the search image file corresponding to the image ID = 3 is “picture01.jpg; picture02.jpg; picture03.jpg; picture04.jpg”, and the unique data corresponding to the image ID = 2 is “Taro Yamada; taro_yamada @ zony.com; rented 1,000 yen ”.

  Here, a file with an extension of “.jpg” indicates image data of a still image compressed by the JPEG (Joint Photographic Experts Group) method. Similarly, an extension of “.tiff” is added. Indicates the still image data compressed with TIFF (Tagged Image File Format), and files with the extension “.mpeg” are mpeg (Moving Picture Experts Group) The image data of the moving image compressed in FIG. Needless to say, the format of the image is not limited to this method, and may be any other format. “Ichiro Tanaka”, “Keiko Yamada”, and “Taro Yamada” in the specific data are text data indicating the name of the specific data, and “ichiro_tanaka@ABC.com” and “taro_yamada@zony.com” are respectively The text data indicates an e-mail address. Further, “1,000 yen lent” and “◯ △ Commercial Accounting Department” are text data indicating memo information.

  More specifically, the information in the search DB 304 is described for each image ID as shown in FIG. 16, for example, in XML (Extesible Markup Language). This description example corresponds to the image ID = 3 in FIG.

  In the case of FIG. 16, “<record>” on the 01st line indicates the start of recording, and the description up to the end position indicated by “</ record>” on the 13th line is a description relating to the image ID = 3. Is shown.

  “<Name> Taro Yamada </ name>” in the second line indicates that the name of the person captured in the image corresponding to the image ID = 3 is “Taro Yamada”. “<Mail> taro_yamada@jp.zony.com </ mail>” on the third line indicates that the mail address is “taro_yamada@jp.zony.com”.

  “<Picture>” in the 04th line indicates that the range from the line until “</ picture>” is described is the range in which the search image file is described. In this case, “</ picture>” is described in the 09th line, which indicates that the search image file is described in the range from the 05th line to the 08th line. .

  In this case, in the 05th to 08th lines, "<path> c: \ my pictures \ picture01.jpg </ path>" and "<path> c: \ my pictures \ picture02.jpg < / path> "," <path> c: \ my pictures \ picture03.jpg </ path> ", and" <path> c: \ my pictures \ picture04.jpg </ path> " The path (information indicating the location of the file) of the search image files “picture01.jpg” to “picture04.jpg” is recorded.

  In addition, “<memo>” is described in the 10th line, and it is shown that memo information is recorded in the range up to “</ memo>” in the 12th line. In this case, “<content> rented 1,000 yen </ content>” is described in the 11th line, and “1,000 yen loaned” is described as a memo.

  The search unit 304 searches the search DB 305 for the search image file corresponding to the image ID supplied as the recognition result and the unique data, and supplies the search result to the display control unit 306 as the search result.

  Now, the description returns to the flowchart of FIG.

  In step S9, the display control unit 306 generates a display screen based on the search result supplied from the search unit 304, and displays it on the display 101 in step S10. The processing returns to step S1, and thereafter The process is repeated.

  That is, by the above processing, for example, when the display screen of the browser as shown in FIG. 17 is displayed on the display 101, the pointer 452 is displayed on the area 451 where the photograph is displayed. To do.

  In step S1, the position of the tip of the pointer 452 is detected as the position of the pointer, and for example, an extracted image 453 is extracted as an image in the vicinity of the pointer 452.

  Through the processing in steps S3 and S4, which DB of the recognition DB 303 is used is specified based on the extracted image 453, and in step S5, the image ID of the person included in the extracted image 453 is determined. Recognize

  The search image file and unique data corresponding to the image ID that is the recognition result are extracted from the search DB 305 by the process of step S8, and the searched search image file and unique data are further processed by the process of step S9. And it is displayed on the display 101 by step S10.

  More specifically, for example, the search image file and the unique data corresponding to the image ID are the search image data and the unique data corresponding to the image ID of FIG. 15 in this case, so that as shown in FIG. The unique data display field 461 and the search image file display field 462 are generated using these pieces of information.

  The unique data display column 461 is a window, and in this case, unique data corresponding to the image ID = 3 in FIG. 15 is shown, and “Taro Yamada”, “taro_yamada@zony.com”, and “1000 yen” Lent "is displayed. The search image file display field 462 is provided with a selection image display field 471 in which an image selected from the search images 474 to 477 corresponding to picture01.jpg to picture04.jpg is displayed. The search images 474 to 477 can be switched by buttons 472 and 473. In this case, the search image 475 is selected and displayed in the selected image display field 471. Therefore, the search image 474 is to be displayed. Sometimes, it can be switched by pressing a button 472. On the other hand, when it is desired to display the search image 476 from the state of FIG. 18, it can be switched by pressing the button 473, and when it is desired to display the search image 477, it can be switched by further pressing the button 473. It becomes.

  By moving the pointer to a predetermined position on the image as shown in FIG. 17 by the processing as described above, the information of the person included in the image near the pointer can be displayed as shown in FIG. For example, by searching for unique data as shown in the unique data display column 461, it is possible to search for a name, an e-mail address, etc., and to recall information that is often forgotten such as "rented 1,000 yen". It becomes possible. Further, by looking at the image displayed in the search image file display field 462, it is possible to confirm what has been done with the projected person. That is, by recognizing an image and retrieving and presenting information based on the recognition result, it is possible to rationally assist a process in which a person remembers by looking at the image.

  Further, in the above, when a human face is recognized based on an image and unique data and a search image file corresponding to the recognized person are displayed (when the recognition unit 302 has a function as a face recognition engine). ), For example, the image recognition engine is used to recognize the shape of a logo mark or the like in the image, information corresponding to the logo mark is presented, and the URL corresponding to the logo mark, for example, URL (Universal Resource Locator) may be accessed.

  FIG. 19 shows other functions realized by the information processing apparatus 3 that recognizes a symbol such as a logo mark included in an image and can execute a corresponding process. The same components as those of the information processing apparatus 3 in FIG. 10 are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

  19 is different from the information processing apparatus 3 in FIG. 10 in that a display control unit 311 is provided instead of the display control unit 306. The function of the display control unit 311 is the same as that of the display control unit 306, but if the URL is included in the unique data stored in the search DB 305, the network is connected via the base unit 2 and the access point 4. A function is provided to access the URL above 1 and display the corresponding browser.

  Next, data display processing by the information processing apparatus 3 in FIG. 19 will be described with reference to the flowchart in FIG. Note that the processing of steps S21 to S29 in the flowchart of FIG. 20 is the same as that of steps S1 to S9 in the flowchart of FIG.

  For example, in the case of FIG. 21, since the pointer 491 exists in the vicinity of the logo mark 493, the display control unit 311 in step S30, the search DB 305 corresponding to the image ID corresponding to the logo mark 493 on the display image 494. The information display column 492 “Access the MIKE Japan homepage” is displayed based on the unique data stored in. That is, the URL corresponding to the logo mark 493 is registered as unique data, and it is indicated that the URL can be accessed by selecting this image.

  In step S31, the display control unit 311 determines whether or not the logo mark 493 has been selected from the display 101 as the operation unit 156. If it is determined that the logo mark 493 has been selected, the display control unit 311 via the parent device 2 and the access point 4 The URL included in the unique data corresponding to the image ID of the logo mark is accessed, the corresponding screen is displayed on the display 101, and the process returns to step S21.

  If it is determined in step S31 that the logo mark 493 has not been selected, the process in step S32 is skipped, the process returns to step S21, and the subsequent processes are repeated.

  As a result of the above processing, the recognition unit 302 functions as an image recognition engine, and as a result, the corresponding information is presented without the user's awareness based on information such as a logo mark of an image near the pointer. Thus, it is possible to quickly provide the information desired by the user without the need for searching and the like.

  In the above description, the recognition unit 302 has a function as a face recognition engine or an image recognition engine, and as a result, the case where information corresponding to the recognition result is presented has been described. A function as a plurality of recognition engines is provided in the recognition unit 302, and switching is performed according to the recognition target so that various information is recognized and information is presented corresponding to the recognition content. Good.

  Furthermore, by setting the input image as an image picked up by the image pickup unit 121, it is possible to present not only information of the picked-up person but also information such as a store in real time. Even when walking in an unguided place, information on nearby stores can be acquired.

  In the above description, an example in which a search image file as a search result is displayed as in the unique data display field 461 and the search image file display field 462 shown in FIG. 18 has been described. If there is a huge amount of unique data or search image files, it will take time and effort to search for the information you want to find while checking the information you want to find, and the amount of information searched will not be known. There is a possibility that even such a situation cannot be predicted.

  In view of this, it is possible to make it easier to predict the time and effort required for the search by displaying the unique data and the information amount of the search image file so that they can be intuitively grasped.

  FIG. 22 shows still other functions realized by the information processing apparatus 3 that can intuitively grasp the searched unique data and the information amount of the search image file. The same components as those of the information processing apparatus 3 in FIG. 10 are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

  22 is different from the information processing apparatus 3 in FIG. 10 in that a display control unit 501 is provided instead of the display control unit 306. The function of the display control unit 501 is the same as that of the display control unit 306, but an information amount calculation unit 501a is further provided to calculate the specific data searched by the search unit 304 and the information amount of the search image file. Is done. Based on the information amount calculated by the information amount calculation unit 501a, the display control unit 501 generates and displays a display screen by changing the size of the icon indicating each unique data or search image file.

  Next, a data display process performed by the information processing apparatus (child device) 3 shown in FIG. 22 will be described with reference to the flowchart shown in FIG. In addition, since the process of step S41 thru | or S48 in the flowchart of FIG. 23 is the same as the process of step S1 thru | or S8 demonstrated with reference to the flowchart of FIG. 11, the description is abbreviate | omitted.

  In step S49, the information amount calculation unit 501a of the display control unit 501 calculates the information amount of the searched unique data and information of the search image file for each item. That is, as described above, in the search DB 305, as shown in FIG. 15, in addition to the search image file consisting of still images and moving images, the name, e-mail address, and memo are included as specific data. Therefore, the information amount calculation unit 501a calculates the amount of information for each piece of still data, moving image, personal data including names, e-mail addresses, and memo information. More specifically, the information amount calculation unit 501a calculates, for example, the number of images in the case of a still image, the recording time in the case of a moving image, personal data, and information on the memo as the amount of characters. To do. Of course, as an information amount, it goes without saying that the size of data (for example, how many bytes it is) may be used as it is.

  In step S50, the display control unit 501 shows, for example, in FIG. 18 based on the search image file and unique data supplied from the search unit 24 and the information amount calculation result calculated by the information amount calculation unit 501a. Instead of the unique data display field 461 and the search image file display field 462, a display image including a window 551 as shown in FIG. 24 is generated.

  That is, in the window 551 in FIG. 24, a balloon-like icon 552 indicating a still image is displayed in the search image file corresponding to the person 556 (target to be searched by the search engine) that is the subject of the image included in the input image. A balloon-like icon 553 indicating a still image of the search image file is displayed on the right side, and a balloon-like icon 555 indicating personal data among the unique data is further displayed on the right side. Shows a balloon-shaped icon 554 indicating memo information in the unique data. Further, an end button 557 for ending this display image is displayed on the upper right of the window 551.

  The icons 552 to 555 are generated so that the diameter of the balloon changes based on the information amount calculated by the above-described information amount calculation unit 501a. Therefore, the user can intuitively grasp the amount of the search image file and the amount of unique data only by looking at the size of the icons 552 to 555 displayed on the window 551. As a result, it is possible to quickly determine whether or not to search information by itself according to the amount of search image files and the amount of unique data.

  In step S52, the display control unit 501 determines whether or not a pointer is present on the icon 555 indicating personal data information among the icons 552 to 555 on the window 551. For example, if it is determined that the icon 561 is present on the icon 555 indicating the memo information as shown in FIG. 25, in step S53, as shown in FIG. 25, a window 562 for displaying personal data among the unique data. Is generated and personal data is displayed, and the process proceeds to step S56.

  In FIG. 25, “Keiko Yamada ○ △ Commercial Accounting Department”, which is personal data of image ID = 2 in the search DB 305 of FIG. A button 563 displayed as “OK” is pressed when the window 562 is closed. Since the user can refer to the personal data simply by moving the pointer 561 to the balloon-like icon indicating the personal data in this way, for example, the user displayed inadvertently forgets the person displayed in the input image. Even in the state, it is possible to remember by receiving the presentation of personal data such as name.

  On the other hand, if the pointer 561 does not exist on the balloon-like icon 555 of personal data in step S52, the display control unit 501 determines in step S54 whether the icons 552 to 554 other than the icon 555 have been selected by the pointer 561. Determine.

  In step S54, for example, as shown in FIG. 26, when the pointer 561 is selected by being clicked in a state of being on the icon 552, it is determined that any of the icons 552 to 554 has been selected. Then, the process proceeds to step S55.

  In step S55, the display control unit 501 displays information corresponding to the selected icon among the selected icons 552 to 554.

  More specifically, for example, as shown in FIG. 26, when the icon 552 for a still image is selected in the search image file, the display control unit 501 repels the selected balloon-shaped icon 552. Then, a window 571 for displaying a still image is displayed.

  In the window 571, a still image display field 572 for displaying one still image of the search image file is displayed. In addition, buttons 573 to 575 are provided below the still image display field 572. The button 573 is operated when returning to and displaying the still image arranged one before the still image displayed in the still image display field 572 in the search image file. The button 574 is operated when the window 571 is closed. Further, the button 575 is operated when the search image file is advanced to a still image arranged one after the still image displayed in the still image display field 572 and displayed.

  Although the case where the still image icon 552 is selected from the search image file has been described above, a window is generated and displayed in the same manner when a moving image is selected. A button, a stop button, a fast forward button, and a fast reverse button are further displayed. When the icon 553 corresponding to the memo is selected from the unique data, information corresponding to the memo is displayed from the unique data. For example, when the image ID of the search DB 305 in FIG. Information such as “I rented 1,000 yen” is displayed.

  In step S56, the display control unit 501 determines whether or not an instruction to end the display of the windows 551, 572, or 562 has been instructed. If it is determined that the end has not been instructed, the process proceeds to step S56. Returning to S52, if it is determined that the end has been instructed, the process returns to step S41, and the subsequent processes are repeated. If no balloon is selected in step S55, that is, if the pointer 561 is moved to any position of the icons 552 to 554 and is not selected, the process in step S55 is skipped. The process proceeds to step S56.

  Through the above processing, personal data can be presented simply by moving the pointer to the position of a specific person, for example, on the image displayed while viewing the image collected by the user. It becomes possible to display an image related to the person. In addition, since the balloon-like icon displayed for each type of information is displayed with the size changed according to the amount of information, it is possible to quickly grasp the amount of information for each type of information. Become. As a result, it is possible to easily determine whether the searched information should be searched with one's own eyes or whether the information should be narrowed down by another method.

  Furthermore, by making the input image an image captured by the image capturing unit 121, for example, by taking an image of a nearby landscape while carrying the information processing apparatus 3 in the city, it is inadvertently known who was an acquaintance. Even if it is forgotten, personal data such as a name is presented to a person registered in the recognition DB 302 and the search DB 304 so that it can be recalled.

  In addition to recognizable information, not only the face of a person but also the logo mark of the store etc. as described above can be recognized, so that the store name and logo mark etc. written on the signboard of the store etc. can be recognized. It is possible to display unique data correspondingly, to access the URL of the store, and to present related information. At this time, the coupon information of the store may be displayed on the above-described balloon-shaped icon, and the coupon may be displayed when the balloon is popped and bursts by selection.

  Furthermore, in the above example, the amount of information is presented by the size of a balloon-shaped icon for each type of information, but this is not limited to a balloon-shaped icon, and the size changes according to the amount of information. Any icon can be used as long as it can be displayed, and icons of other shapes may be used.

  In the above, the case where the information processing device 3 executes the above-described processing has been described, but the information processing device 2 that is the parent device may execute the same processing. When the above-described processing is executed by the information processing apparatus 2, the functions shown in FIGS. 10, 19, and 22 are realized by the information processing apparatus 2. Since each process is the same, description thereof is omitted.

  In the above, an example of recognizing an image has been described. For example, information is presented based on a voice recognition result by performing voice recognition processing using voice detected by the microphone 102 or 215. You may make it make it. That is, information necessary for speech recognition is stored in advance in the speech recognition DB 303, and the recognition unit 302 uses the recognition DB 303 to input a person from, for example, the type of person's voice. It can be identified or identified by identifying a person, animal, or machine using animal calls or machine operating sounds. Further, by storing search information corresponding to the recognition result in the search DB 304, the search unit 304 may search for information corresponding to the recognition result.

  In the above example, the case where the information processing apparatus 2 or 3 includes the recognition DB 302 and the search DB 305 has been described. However, the recognition DB 302 and the search DB 305 are provided by themselves. For example, the recognition unit 302 and the search unit 304 may be used via the network 1 from any one of the image data management servers 5 to 7.

  According to the above, it is possible to acquire input information, search and display the presentation information desired by the user based on the acquired input information, and display the presentation information according to the information amount of the presentation information. Since the display screen to be displayed is generated, it is possible to quickly grasp the amount of information for each type of information.

  The series of processes described above can be executed by hardware, but can also be executed by software. When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. For example, a general-purpose personal computer is installed from a network or a recording medium.

  7 and 9, this recording medium is distributed to provide a program to the user separately from the computer. Magnetic disks 171 and 241 (including a flexible disk) on which the program is recorded are distributed. Optical disks 172 and 242 (including CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc)), magneto-optical disks 173 and 243 (including MD (Mini-Disc)), or semiconductor memories 174 and 244 In addition to a package medium composed of, etc., it is composed of a ROM 152 on which a program is recorded, a hard disk included in the storage unit 154, the HDD 223, etc. provided to the user in a state pre-installed in the apparatus. .

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the described order, but is not necessarily performed in chronological order. It also includes processes that are executed individually.

  Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.

It is a figure which shows the structure of the service provision system of this invention. It is a figure explaining the example at the time of attaching a subunit | mobile_unit to the main | base station of FIG. It is a figure explaining the example at the time of removing a subunit | mobile_unit from the main | base station of FIG. It is a figure explaining the example of the shape and size of the subunit | mobile_unit of FIG. It is a figure explaining the external appearance of the subunit | mobile_unit of FIG. 1, and an example of an internal structure. It is a figure explaining the structural example of the external appearance of the subunit | mobile_unit of FIG. It is a block diagram which shows the electrical structural example inside the subunit | mobile_unit of FIG. It is a figure explaining the example of the shape and size of the main | base station of FIG. It is a block diagram which shows the electrical structural example inside the main | base station of FIG. It is a functional block diagram explaining the function implement | achieved by the information processing apparatus of FIG. It is a flowchart explaining the data display process by the function implement | achieved by the information processing apparatus of FIG. It is a figure explaining DB for recognition. It is a figure explaining image recognition. It is a figure explaining image recognition. It is a figure explaining DB for search. It is a figure explaining the example of description of DB for search. It is a figure explaining the example of a display of a window. It is a figure explaining the example of a display of a window. It is a functional block diagram explaining the other function implement | achieved by the information processing apparatus of FIG. It is a flowchart explaining the data display process by the function implement | achieved by the information processing apparatus of FIG. It is a figure explaining the example of a display of a window. FIG. 8 is a functional block diagram for explaining still another function realized by the information processing apparatus of FIG. 7. It is a flowchart explaining the data display process by the function implement | achieved by the information processing apparatus of FIG. It is a figure explaining the example of a display of a window. It is a figure explaining the example of a display of a window. It is a figure explaining the example of a display of a window.

Explanation of symbols

  301 Image Extraction Unit, 301a Pointer Position Detection Unit, 302 Recognition Unit, 303 Recognition DB, 304 Search Unit, 305 Search DB, 306, 311, 501 Display Control Unit, 501a Information Amount Calculation Unit

Claims (8)

An acquisition means for acquiring input information;
Recognition means for recognizing input information acquired by the acquisition means;
Search means for searching for presentation information based on a recognition result of the recognition means;
A calculation means for calculating an information amount of the presentation information searched by the search means;
An information processing apparatus comprising: screen generation means for generating a screen for displaying the presentation information based on the amount of information calculated by the calculation means. The acquisition means acquires an image as the input information,
The information processing apparatus according to claim 1, wherein the recognition unit recognizes the image acquired by the acquisition unit. Further comprising position detecting means for detecting the position of the pointer on the image;
The information processing apparatus according to claim 2, wherein the acquisition unit acquires an image in the vicinity of the position detected by the position detection unit as input information. The acquisition means acquires voice as the input information,
The information processing apparatus according to claim 1, wherein the recognition unit recognizes the voice acquired by the acquisition unit. The information processing apparatus according to claim 1, wherein the screen generation unit generates a screen that displays the presentation information with an icon having a size based on the amount of information calculated by the calculation unit. An acquisition step for acquiring input information;
A recognition step for recognizing input information acquired in the processing of the acquisition step;
A search step of searching for presentation information based on a recognition result in the processing of the recognition step;
A calculation step of calculating an information amount of the presentation information searched in the processing of the search step;
A screen generation step of generating a screen for displaying the presentation information based on the amount of information calculated in the processing of the calculation step. An acquisition control step for controlling the acquisition of input information;
A recognition step for recognizing input information whose acquisition is controlled in the processing of the acquisition control step;
A search step of searching for presentation information based on a recognition result in the processing of the recognition step;
A calculation step for calculating an information amount of the presentation information searched in the processing of the search step;
A screen generation step for generating a screen for displaying the presentation information based on the amount of information calculated in the processing of the calculation step. A recording medium on which a computer-readable program is recorded. An acquisition control step for controlling the acquisition of input information;
A recognition step for recognizing input information whose acquisition is controlled in the processing of the acquisition control step;
A search step of searching for presentation information based on a recognition result in the processing of the recognition step;
A calculation step of calculating an information amount of the presentation information searched in the processing of the search step;
A program causing a computer to execute a screen generation step of generating a screen for displaying the presentation information based on the amount of information calculated in the processing of the calculation step.

Images