JP2000113097A - Device and method for image recognition, and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve user's convenience by displaying image pickup information regarding a subject on a screen. SOLUTION: A signal processing part 5 processes an image signal sent from a CCD 4 variously and stores the signal in a main storage part 22 and on an external storage medium 14 such as a floppy disk, displays the picked-up image at a display part 6, and sends the variously processed image data to an image recognition part 7. The image recognition part 7 extracts feature quantities of the image from the sent image data, retrieves feature information in a memory 9 according to the extracted feature quantities to recognize the subject, and sends information on the recognized subject to an information retrieval part 8. The information retrieval part 8 retrieves various information specifying the subject from the memory 9 according to the send information on the subject, reads retrieved information specifying the subject, e.g. the explanation for the subject, and composites it with the image of the subject displayed at the display part 6 and display it.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image recognizing device for displaying information for specifying a subject picked up by an image pickup device, an image recognizing method, and a storage medium storing the image recognizing method.

[0002]

2. Description of the Related Art A digital camera which takes an image of a subject, converts the image into digital image data, and displays the image of the subject is easy to interface with a digital device, is suitable for input to a personal computer or the like, and is taken by a digital line. The feature is that image information can be transmitted without deterioration of image quality. A reading device that searches for product information using an image captured by this digital camera is disclosed in, for example,
No. 114913. This reading device captures an image of a photographing target including a plurality of products with a barcode by a digital camera, extracts image data corresponding to the barcode, recognizes the barcode, and stores product information corresponding to the recognized barcode data. Is retrieved from the product data memory and displayed to improve the work efficiency of product management.

As a method of providing information to a mobile terminal, for example, Japanese Patent Application Laid-Open No. Hei 9-130861 discloses a method in which positional information of a mobile terminal by GPS and a destination are communicated to an information server through a mobile phone, and the information server is moved. An area including the current location and the destination of the terminal is determined, and service information including position information corresponding to the determined area is transmitted via the network.

[0004]

However, as shown in Japanese Patent Application Laid-Open No. 9-114913, when a subject on which information for identifying a subject is displayed is taken by a digital camera, the information can be recognized. Even if a digital camera captures an object such as a natural object that does not display information for identifying the object, the digital camera cannot recognize the detailed information of the captured object. Could only be used for For this reason, information on the imaged subject is stored in a guidebook or in
It has to be confirmed using the mobile terminal disclosed in Japanese Patent No. 1303081.

[0005] In contrast, for example, when an image is taken with a digital camera at a tourist attraction of a travel destination and information on the tourist attraction is displayed on the imaged screen, the traveler who has been traveling around the tourist attraction with a guidebook in one hand so far. It serves as a very convenient guide for. Also, when images of alpine plants and the like are taken with a digital camera, their names and descriptions are displayed, so that it is possible to know in detail the information of the plants and the like taken without carrying a large picture book etc. It can be easily organized. When the information of the imaged subject is displayed on the digital camera when the image is captured by the digital camera in this way, the utility value of the digital camera can be greatly enhanced.

[0006] In order to meet such a demand, the present invention has stored an image recognition apparatus, an image recognition method, and an image recognition method which can display information on a photographed subject on a screen and improve user convenience. It is intended to provide a storage medium.

[0007]

An image recognition device according to the present invention has an image recognition unit and an information display unit, and the image recognition unit extracts and captures features of a digital image captured by an imaging unit. The subject is recognized, and the information display unit displays information of the recognized subject together with the image on the display unit on which the image of the subject is displayed.

A second image recognition device according to the present invention has an information storage and storage unit, an image recognition unit and an information search unit, and the information storage and storage unit stores in advance the characteristic information of the subject and various information for specifying the subject. The image recognizing unit extracts the feature of the digital image captured by the imaging unit, searches the feature information of the subject stored in the information storage unit based on the extracted feature, recognizes the captured subject, and recognizes the captured subject. Various types of information that specify the subject stored in the information storage unit are searched by the recognized subject, and the information that specifies the searched subject is displayed together with the image on the display unit on which the image of the subject is displayed.

The information storage unit is preferably a removable recording medium.

A third image recognition device according to the present invention has an information storage and storage unit, an information input unit, an image recognition unit and an information search unit, and the information storage and storage unit specifies the characteristic information of the subject and the subject. The information input unit stores various information and an information search processing program, and the information input unit inputs the characteristic information of the subject and various information specifying the subject from an external computer and the information search processing program, and stores the characteristic information of the subject stored in the information storage unit. The image recognizing unit extracts features of the digital image captured by the image capturing unit, and rewrites the information retrieval processing program stored in the information storage unit based on the extracted features. The information retrieval unit recognizes the captured object by retrieving the feature information, and the information retrieval unit stores the information retrieval processing program stored in the information storage unit based on the recognized subject. Searching various information for identifying a subject by ram, images of a subject information identifying the searched object and displaying together with the image on the display unit displayed.

[0011] The image recognizing unit is configured to store the characteristic information of the subject and the GP.
It is preferable to recognize the imaged subject based on the position information of the subject input from the S device.

A fourth image recognition apparatus according to the present invention has an image pickup means and a communication means, the image pickup means displays a digital image picked up on a display section, and the communication means outputs a digital image picked up by the image pickup means to a computer. The computer extracts the features of the transmitted digital image, recognizes the captured subject, searches for information identifying the recognized subject, transmits the information to the communication means, and the communication means identifies the transmitted subject. The information to be transmitted is sent to the image pickup means, and the image pickup means displays the information for specifying the sent object together with the image on the display section on which the image of the object is displayed.

Preferably, the information for specifying the subject is combined with an image of the subject and stored.

According to the image recognition method of the present invention, a feature of a digital image picked up by an image pick-up means is extracted, the picked-up subject is recognized, and information for specifying the recognized subject is displayed on a display unit on which the image of the subject is displayed. The information is displayed together with the image, and information of the recognized subject is combined with the image of the subject and stored.

The recording medium according to the present invention stores a program for extracting characteristics of a digital image picked up by an image pickup means, recognizing the picked-up object, and reading out information for specifying the recognized object, and storing characteristic information of the object. And various information for specifying the subject are stored.

In another recording medium according to the present invention, feature information of a subject, various kinds of information for specifying the subject, and features of a digital image captured by an imaging unit are extracted, and images are captured from the extracted features and feature information of the subject. And a program for recognizing information for specifying the subject and reading the information for specifying the recognized subject.

[0017]

DESCRIPTION OF THE PREFERRED EMBODIMENTS A digital camera according to the present invention has a photographing lens, an aperture, a CCD, a signal processing unit, a display unit comprising a liquid crystal panel, an image recognition unit, an information retrieval unit, and a memory. The memory stores the characteristic information of the subject and various information for specifying the subject.

When an image of a subject is picked up by the digital camera, the amount of light from the subject that has entered the photographing lens is adjusted by the aperture and formed on a CCD. The image formed on the CCD is converted into an image signal in accordance with the amount of light for each pixel, and is sequentially sent to a signal processing unit. The signal processing unit performs various kinds of processing of the transmitted image signal, stores the processed image signal in a built-in main storage unit such as a flash memory or an external storage medium such as a floppy disk, displays a captured image on a display unit, and performs various kinds of processing. The obtained image data is sent to the image recognition unit. The image recognizing unit extracts a feature amount of the image from the sent image data, searches feature information in a memory based on the extracted feature amount to recognize a subject, and sends information of the recognized subject to the information searching unit. The information search unit searches the memory for various kinds of information for identifying the subject based on the sent information of the subject, reads out information for identifying the searched subject, for example, a description of the subject, and combines the information with the image of the subject displayed on the display unit. To display.

[0019]

FIG. 1 is a block diagram showing the configuration of an embodiment of the present invention. As shown in the figure, the digital camera 1 includes a photographing lens 2, an aperture 3, a CCD 4, a signal processing unit 5, a display unit 6 including a liquid crystal panel, an image recognition unit 7, an information search unit 8, a main storage unit 22, and a memory 9. Have. The memory 9 has a feature information storage section 12 storing feature information of the subject and an information storage section 13 storing various information for specifying the subject.
The image recognizing unit 7 identifies a captured subject based on image data sent from the signal processing unit 5, and includes a feature extracting unit 10 and a subject identifying unit 11. The feature extracting unit 10 extracts the feature of the captured subject from the image data sent from the signal processing unit 5. The subject identification unit 11 identifies the subject by comparing the extracted feature of the subject with the feature information stored in the feature information storage unit 12 of the memory 9. The information search unit 8 searches the information storage unit 13 based on the identification information of the subject, reads out information for specifying the subject, and displays the information on the display unit 6.

When an image of a subject is taken by the digital camera 1, light from the subject incident on the taking lens 2 is
And the light amount is adjusted to form an image on the CCD 4. The image formed on the CCD 4 is converted into an image signal according to the amount of light for each pixel, and is sequentially sent to the signal processing unit 5 at regular time intervals. The signal processing unit 5 removes and amplifies noise components and the like of the transmitted image signal, converts the analog signal into a digital signal, converts the digital signal into an RGB signal and a luminance signal, and converts the image signal into a gamma signal. After performing image processing such as correction and compression, the main storage unit 22 including a flash memory or the like or the external storage medium 1
4 is stored in a floppy disk or the like, the captured image is displayed on the display unit 6, and image data subjected to various processes is sent to the image recognition unit 7.

A process for recognizing a subject by the image recognizing unit 7 and displaying information for specifying the subject recognized by the information retrieving unit 8 on the display unit 6 will be described with reference to a flowchart of FIG.

When image data is sent from the signal processing unit 5 to the image recognizing unit 7 while capturing an image of a subject (step S1), the feature extracting unit 10 extracts a feature amount of the image from the sent image data. And sends it to the subject identification unit 11 (step S2). The subject identification unit 11 searches the feature information storage unit 12 of the memory 9 based on the sent feature amount (step S).
3) It is determined whether feature information within a predetermined threshold value is stored in the feature information storage unit 12 for the sent feature amount (step S4), and when feature information within the threshold value is stored. Recognizes the subject based on the characteristic information, and sends information of the recognized subject, for example, a name or the like to the information search unit 8 (step S5). The information search unit 8 searches the information storage unit 13 of the memory 9 based on the sent information of the subject (step S6), reads information specifying the searched subject, for example, a description of the subject, and reads the subject displayed on the display unit 6. (Step S7). In addition, the memory 9 is stored in the memory 9 based on the feature amount sent from the subject identification unit 11.
When the feature information storage unit 12 is searched and no feature information within the threshold is stored (step S4), the fact that the subject cannot be recognized is sent to the information search unit 8. When receiving the signal indicating that the subject cannot be recognized, the information search unit 8 sends a message indicating that there is no information for identifying the subject to the display unit 6, combines the message with the image of the subject, displays the image, and stores the image in the external storage medium 14. (Step S8).

As described above, various information of a subject is stored in the memory 9 and information for specifying an image, such as a description of a taken image, is displayed together with the image. By lending the digital camera 1 in which information such as the information is stored in the memory 9 to a user, it is possible to confirm the description and the like while capturing an image of a building or a famous place.

In the above embodiment, the memory 9 of the digital camera 1 is used.
In the above description, various kinds of information of a subject are stored in advance.
Alternatively, the information may be stored in an external storage medium, and the information from the external storage medium may be used to recognize the subject and display various information for specifying the subject.

FIG. 3 is a block diagram showing the configuration of an embodiment for reading information of a subject from an external storage medium such as a CD-ROM. As shown in FIG.
An external storage medium input / output unit 15 for reading and outputting various information from an external storage medium 16 such as a ROM is provided. The external storage medium 16 stores various information of the subject, for example, the contents of a pictorial book showing information such as characteristic information of alpine plants and their description. The external storage medium 16 is mounted on the external storage medium input / output unit 15 of the digital camera 1a, and the digital camera 1a captures an image of an alpine plant found during mountain climbing. The captured image data of the alpine plant is sent from the signal processing unit 5 to the image recognizing unit 7, and the feature extracting unit 10 extracts the feature amount of the alpine plant from the image data. The external storage medium 16 is searched through the medium input / output unit 15 to recognize the imaged alpine plant, and the recognized information is sent to the information search unit 8. The information retrieval unit 8 retrieves the external storage medium 16 based on the transmitted information on the alpine plants, reads out the name, classification, and various descriptions of the captured alpine plants, and combines them with the alpine plant image displayed on the display unit 6. indicate.

As described above, the subject is recognized based on the information stored in the external storage medium 16 such as a CD-ROM, and information such as an explanation for identifying the subject is displayed together with the image of the subject. By replacing the medium 16, explanations of various subjects can be displayed, and the present invention can be applied to any purpose.

In the above-described embodiment, various kinds of information of a subject are recorded on a CD-R.
The case of inputting from an external storage medium 16 such as an OM has been described, but as shown in the block diagram of FIG. 4, a communication control unit 17 and a modem 18 are provided in the digital camera 1b, and a mobile phone 19 and an external computer are used. In advance, various information of the subject and an information retrieval processing program are input in advance, and the input various information of the subject is input and stored in the memory 9,
The information search unit 8 searches for and processes information according to the input information search processing program, or the subject identification unit 1 of the digital camera 1c as shown in the block diagram of FIG.
In 1, the imaged subject may be recognized from the feature information of the subject stored in the feature information storage unit 12 of the memory 9 and the position information of the subject obtained from the GPS device 20 used for car navigation or the like. In this way, by recognizing the subject using the position information of the subject, the recognition accuracy of the subject can be further improved.

In each of the above embodiments, a case has been described in which the digital camera 1 recognizes a subject, and various kinds of information for specifying the recognized subject are combined with an image and displayed. The information is sent to a personal computer or the like by a communication means such as a telephone, the subject captured by the personal computer or the like is recognized, various information for specifying the recognized subject is retrieved, sent back to the digital camera 1 via the communication means, and combined with the captured image. May be displayed.

An embodiment in this case will be described with reference to the block diagram of FIG. As shown in the figure, the digital camera 1d
Has a photographing lens 2, an aperture 3, a CCD 4, a signal processing unit 5, a display unit 6, and communication means 21 such as a mobile phone or an infrared transceiver, and stores image data of a captured object in a main storage unit 22 or a floppy disk. The image stored in the external storage medium 14 and captured is displayed on the display unit 6. The communication unit 21 exchanges various information and data with the computer 30 by wireless. The computer 30 includes a communication interface 31 for exchanging information with the digital camera 1e, a CPU 32, a memory 33, a display unit 34, and a recording medium such as a CD-ROM 3
6 has a recording medium driver 35 for reading various information recorded on the CD-ROM 36 by driving the CD-ROM 36. CD-R
The OM 36 stores characteristic information of a subject captured by the digital camera 1d and various types of subject information for specifying the subject. For example, a CD-R titled "Kyoto"
The OM 36 stores feature information of landscapes and buildings in Kyoto and various information describing the landscapes and buildings. Memory 3
3 extracts the characteristics of the imaged subject from the image data sent from the digital camera 1d, compares the extracted characteristics of the object with the characteristic information stored in the CD-ROM 36, and identifies the object. -ROM 36
A recognition program for retrieving and reading information for specifying the subject is stored in advance.

The digital camera 1 configured as described above
The processing when the subject is imaged in d will be described with reference to the flowchart in FIG.

An image of a subject is picked up by the digital camera 1d, the picked up image is displayed on the display unit 6, and the main storage unit 22
Or stored in the external storage medium 14 (step S1
1) When the user presses a switch or button of the digital camera 1d or selects information display with a cursor key or the like from a menu displayed on the display unit 6, the image data stored in the main storage unit 22 or the external storage medium 14 is deleted. The signal is read out by the signal processing unit 5 and transmitted to the communication interface 31 of the computer 30 via the communication unit 21 (steps S12 and S12).
13). The communication interface 31 stores the received image data in the memory 33. In this state, the CPU 32 extracts the feature amount of the captured image from the image data received by the recognition program stored in the memory 33 in advance (step S14), and the CD-R is extracted based on the extracted feature amount.
The feature information of the OM 36 is searched to determine whether feature information within a predetermined threshold value is stored for the extracted feature amount (steps S15 and S16). As a result of this determination, when feature information within the threshold is stored, the subject is recognized based on the feature information (step S).
17) CD-ROM 36 based on the information of the recognized subject
Is retrieved (step S18), information for identifying the retrieved subject, for example, a description of the subject, is read, and information such as the description of the subject is transmitted to the digital camera 1d via the communication interface 31 (step S1).
9). Further, the CPU 32 determines a CD-format based on the extracted feature amount.
If the feature information within the threshold value is not stored when the feature information in the ROM 36 is searched (step S16),
A message indicating that there is no information for specifying the subject is transmitted to the digital camera 1d via the communication interface 31 (step S20). The communication means 21 of the digital camera 1d writes the message indicating that there is no description of the subject or that there is no information for identifying the subject on the display unit 6 on which the image is displayed, displays the message, and stores it in the external storage medium 14 (step). S21).

In this way, when a subject is imaged by the digital camera 1d which does not use many memories or the like with a simple configuration, the information can be displayed. Also,
By exchanging the CD-ROM 36, the hard disk, and the like by remote control, when an arbitrary place or situation is imaged, the explanation and the like can be displayed together with the image.

In the above embodiment, the memory 3 of the computer 30 is used.
3 describes a case where a recognition program for previously extracting information such as a description of extracting a feature of a captured object from image data sent from the digital camera 1d and identifying the subject based on the feature of the subject and identifying the subject is stored in advance. However, the recognition program may be stored in the CD-ROM 36.

[0034]

As described above, according to the present invention, the features of a digital image taken by the image pickup means are extracted and the imaged object is recognized, and the information display section displays the information of the recognized object on the image of the object. Since the image is displayed on the display unit together with the image, it is possible to confirm the description of the imaged object while imaging the object with the image pickup means, and to increase the value of use of the image pickup means such as a digital camera.

Further, by storing in advance the characteristic information of the subject and various kinds of information for specifying the subject in the imaging means, the information of the subject imaged at an arbitrary place can be quickly displayed.

Further, the characteristic information of the subject and various information for specifying the subject are stored in a removable recording medium, and the characteristic information of the subject and various information for specifying the subject and an information retrieval processing program are obtained from an external computer. Can respond to various objects.

In addition, by recognizing the imaged subject in consideration of the position information of the subject, the recognition rate of the subject can be further improved.

The imaging means transmits the captured digital image to an external computer, recognizes a subject captured by the computer, extracts information for identifying the recognized subject, and transmits the information to the imaging means. Can be simplified, and information for specifying the subject can be displayed together with the image of the subject by the imaging means having a simple configuration.

A program for extracting the characteristics of the digital image captured by the imaging means, recognizing the captured object, and reading out the information of the recognized object, and the characteristic information of the object and various information for specifying the object are stored in the storage medium. By storing the information, information on various subjects can be easily displayed together with the image, and various objects can be handled.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of the present invention.

FIG. 2 is a flowchart showing a process of the embodiment.

FIG. 3 is a block diagram showing a configuration of a second embodiment.

FIG. 4 is a block diagram showing a configuration of a third embodiment.

FIG. 5 is a block diagram showing a configuration of a fourth embodiment.

FIG. 6 is a block diagram showing a configuration of a fifth embodiment.

FIG. 7 is a flowchart illustrating a process according to a fifth embodiment.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 digital camera 2 shooting lens 3 aperture 4 CCD 5 signal processing unit 6 display unit 7 image recognition unit 8 information search unit 9 memory 10 feature extraction unit 11 subject identification unit 12 feature information storage unit 13 information storage unit 14 external storage medium 22 main Memory

Claims (12)

[Claims] 1. An image recognizing unit comprising: an image recognizing unit and an information displaying unit, wherein the image recognizing unit extracts features of a digital image captured by an image capturing unit and recognizes a captured subject; An image recognition apparatus characterized in that is displayed together with an image on a display unit on which an image of a subject is displayed. 2. An information storage / storage unit, an image recognition unit, and an information search unit, wherein the information storage / storage unit stores in advance characteristic information of a subject and various information for specifying the subject, and the image recognition unit includes an image pickup unit. The features of the captured digital image are extracted, the feature information of the subject stored in the information storage unit is searched by the extracted features, and the captured subject is recognized, and the information search unit stores in the information storage unit based on the recognized subject. An image recognizing apparatus comprising: searching for various kinds of information identifying a searched subject; and displaying information identifying the searched subject together with an image on a display unit on which an image of the subject is displayed. 3. The image recognition device according to claim 2, wherein the information storage unit is a removable recording medium. 4. An information storage / storage unit, an information input unit, an image recognition unit, and an information search unit, wherein the information storage / storage unit stores characteristic information of the subject, various information for specifying the subject, and an information search processing program. The information input unit inputs the characteristic information of the subject and various information for specifying the subject from an external computer and an information search processing program, and stores the characteristic information of the subject and various information and information for specifying the subject stored in the information storage unit. The processing program is rewritten, and the image recognizing unit extracts the features of the digital image captured by the image capturing unit, searches the feature information of the subject by the information search processing program stored in the information storage unit based on the extracted features, and retrieves the captured subject. Recognized and the information retrieval unit uses the information retrieval processing program stored in the information storage unit based on the recognized subject to identify various types of information. Search, the image recognition apparatus characterized by displaying together images on a display unit on which an image is displayed in the object information identifying the searched object. 5. The image recognition unit according to claim 1, wherein said characteristic information is
The image recognition device according to claim 1, 2, 3, or 4, wherein the imaged object is recognized based on the position information of the object input from the PS device. 6. An image pickup means and a communication means, wherein the image pickup means displays a digital image picked up on a display unit, the communication means transmits a digital image picked up by the image pickup means to a computer, and the computer outputs the transmitted digital image. The features of the image are extracted to recognize the imaged subject, the information identifying the recognized subject is searched and transmitted to the communication means, and the communication means sends the transmitted information identifying the subject to the imaging means. Is an image recognition device for displaying information for identifying a sent subject together with the image on a display unit on which the image of the subject is displayed. 7. The image recognition apparatus according to claim 1, wherein the information for specifying the subject is combined with an image of the subject and stored. 8. A method for extracting features of a digital image captured by an imaging unit, recognizing a captured subject, and displaying information for identifying the recognized subject together with the image on a display unit on which the image of the subject is displayed. Image recognition method. 9. The image recognition method according to claim 8, wherein the information of the recognized subject is combined with an image of the subject and stored. 10. A storage medium storing a program for extracting a feature of a digital image captured by an imaging unit, recognizing a captured subject, and reading information for identifying the recognized subject. 11. A storage medium storing characteristic information of a subject and various information for specifying the subject. 12. Extracting characteristic information of a subject, various types of information for specifying the subject, and features of a digital image captured by the imaging means, and recognizing information for specifying the captured subject from the extracted features and the feature information of the subject. A program for reading information for identifying a recognized subject.