JP2009272931A - Imaging apparatus, and data providing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus for obtaining, within a short period of time, the data in relation to the interested object among the images imaged even in the area of insufficient communication environment, and to provide a data providing system utilizing such imaging apparatus. <P>SOLUTION: The imaging apparatus includes an image sensor 102 for acquiring image data by imaging an object, an interested object extracting circuit 108 for extracting the image data obtained of the interested object from the image data obtained through the image sensor 102, a characteristic amount extracting circuit 109 for extracting amount of characteristic in relation to the interested object from the image data obtained of the interested object extracted with the interested object extracting circuit 108, a data acquiring circuit 110 for acquiring reference data in relation to the reference object corresponding to the interested object from the amount of characteristic extracted with the characteristic amount extracting circuit 109, and a monitor 115 for displaying the reference data obtained with the data acquiring circuit 110. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging device capable of extracting a subject of interest from a captured image and providing reference information related to the extracted subject of interest, and an information providing system using such an imaging device.

In recent years, a predetermined subject in an image captured by a user is extracted as a subject of interest, the target subject is identified by referring to a predetermined database, and reference information related to the identified subject of interest is identified by the user. Information providing services have been started. In such an information providing service, a general-purpose terminal device such as a PC (Personal Computer) is used to access a database, or a dedicated information processing device is used. In Patent Document 1, detailed information of a subject of interest in a captured image can be acquired using a mobile terminal device such as a mobile phone.
JP 2003-323440 A

  Here, in the conventional method such as Patent Document 1, reference information is acquired by accessing a database on a server from a terminal device. Therefore, there may be a case where reference information related to the subject of interest cannot be acquired or it takes time to acquire the reference information depending on the communication status. In addition, power consumption increases as the communication time increases. For this reason, it is desirable to be able to acquire reference information without performing communication, or to be able to acquire reference information related to the subject of interest in as short a time as possible.

  The present invention has been made in view of the above circumstances, and an imaging apparatus capable of acquiring in a short time information related to a subject of interest in a captured image even in a place with poor communication conditions, and such an imaging apparatus. It aims at providing the used information service system.

  In order to achieve the above object, an imaging device according to a first aspect of the present invention includes an imaging unit that captures a subject and acquires a captured image, and a target subject extraction unit that extracts an image of the target subject from the captured image. A feature amount extraction unit that extracts a feature amount related to the target subject from the extracted image of the target subject, an information acquisition unit that acquires reference information corresponding to the target subject from the extracted feature amount, and the acquisition And an information providing unit for providing the reference information to the user.

  According to the present invention, it is possible to provide an imaging apparatus capable of acquiring information relating to a subject of interest in a captured image in a short time even in a place with poor communication conditions, and an information providing system using such an imaging apparatus. Can do.

[First Embodiment]
First, a first embodiment of the present invention will be described. FIG. 1 is a diagram illustrating a configuration of an imaging apparatus according to the first embodiment of the present invention. An imaging apparatus 100 shown in FIG. 1 includes an optical system 101, an image sensor 102, an analog / digital (A / D) conversion circuit 103, a preprocess circuit 104, a focus detection circuit 105, and a random access memory (RAM). 106, image processing circuit 107, target subject extraction circuit 108, feature amount extraction circuit 109, information acquisition circuit 110, image compression / decompression circuit 111, flash memory 112, recording medium 113, display circuit 114 , A monitor 115, an input circuit 116, a control circuit 117, and a data bus 118.

  The optical system 101 is an optical system for forming an optical image from a subject (not shown) on the image sensor 102. The image sensor 102 is configured such that, for example, a Bayer color filter is disposed on the front surface of a pixel that converts an optical image formed through the optical system 101 into an analog electric signal (captured image signal). Note that the image sensor 102 can use either a CCD image sensor or a CMOS image sensor.

  The A / D conversion circuit 103 digitizes the captured image signal output from the image sensor 102 to obtain captured image data. The preprocess circuit 104 performs preprocessing such as white balance adjustment processing on the captured image data obtained by the A / D conversion circuit 103. The white balance adjustment process is a process for adjusting the color balance of the captured image data by adjusting the gain of each color component (R, Gr, Gb, B in the case of the Bayer array) of the captured image data. The focus detection circuit 105 calculates a focus evaluation value to be used for focus adjustment of the optical system 101.

  The RAM 106 is composed of SDRAM or the like, and temporarily stores captured image data obtained by the A / D conversion circuit 103. The image processing circuit 107 performs various types of image processing on the captured image data obtained by the A / D conversion circuit 103 and temporarily stored in the RAM 106. As this image processing, processing for generating luminance data Y and color difference data Cb and Cr from captured image data, processing for removing noise in captured image data, and processing for correcting tone characteristics (gamma characteristics) in captured image data And processing for changing the angle of view of the captured image data.

  The subject-of-interest extraction circuit 108 extracts captured image data of the subject of interest considered to be focused by the user from the captured image data. The feature quantity extraction circuit 109 extracts a feature quantity such as shape and color from the captured image data of the subject of interest extracted by the subject of interest extraction circuit 108. The information acquisition circuit 110 compares the feature quantity extracted by the feature quantity extraction circuit 109 with the reference information registered in either the flash memory 112 or the recording medium 113 and flashes the reference information related to the subject of interest. Obtained from the memory 112 or the recording medium 113.

  The image compression / decompression circuit 111 compresses the data after the image processing in the image processing circuit 107 using the JPEG method, the MPEG method, or the like to generate compressed image data, and decompresses the compressed image data.

  The flash memory 112 stores various programs and parameters for controlling the imaging apparatus 100, and stores reference information of various reference objects referred to as the subject of interest as a database. Here, the reference information includes, as basic information, information that the user wants to know most, such as a generic name and an individual name of the reference object, and in addition, in order to identify an image or reference object indicating the reference object The identification code and the detailed information of the reference object may be included. The recording medium 113 is a memory card configured to be detachable from the imaging device of FIG. On this recording medium 113, the compressed image data obtained by the image compression / decompression circuit 111 is recorded, and reference information of various reference objects to be compared with the subject of interest is recorded as a database.

  The display circuit 114 performs various processes for performing image display based on the captured image data. As this processing, for example, processing for resizing captured image data in accordance with the display size of the monitor 115, processing for generating a synchronization signal for outputting captured image data or OSD data of reference information related to the subject of interest to the monitor 115. Etc. are included. The monitor 115 is a liquid crystal display, for example, and displays an image based on captured image data, reference information related to the subject of interest, and the like.

  The input circuit 116 is a circuit for the user to give various instructions to the control circuit 117, and includes operation members such as various buttons and a touch panel. This instruction includes an instruction to change the angle of view of the imaging data, an instruction to specify a method for extracting the subject of interest, and the like. The control circuit 117 includes a CPU, and controls various processes of the imaging device 100. The data bus 118 is a transfer path for transferring various data generated in the imaging apparatus 100.

Next, the operation of the imaging apparatus shown in FIG. 1 will be described.
An optical image obtained through the optical system 101 is received by the image sensor 102 and converted into a captured image signal. The captured image signal obtained in the image sensor 102 is output to the A / D conversion circuit 103 and digitized. The captured image data obtained by the A / D conversion circuit 103 is output to the preprocess circuit 104 and the focus detection circuit 105. The preprocess circuit 104 performs preprocessing on the input captured image data. The preprocessed captured image data is temporarily stored in the RAM 106 via the data bus 118.

  The focus detection circuit 105 calculates a focus evaluation value based on the input captured image data. This focus evaluation value is output to the control circuit 117. The control circuit 117 drives the optical system 101 according to the focus evaluation value, and performs focus adjustment of the optical system 101. As a result, the optical system 101 is focused on the subject in the imaging data obtained via the image sensor 102. At the time of shooting, the captured image data after focus adjustment is temporarily stored in the RAM 106.

  The captured image data stored in the RAM 106 is read out by the image processing circuit 107. The image processing circuit 107 performs various image processing such as noise removal, luminance / color difference conversion, and gamma correction on the read captured image data. The captured image data after image processing is temporarily stored in the RAM 106 via the data bus 118. The captured image data after image processing stored in the RAM 106 is read by the display circuit 114 or the image compression / decompression circuit 111. The display circuit 114 resizes (usually reduces) the read captured image data to a size suitable for the display size of the monitor 115. Furthermore, the display circuit 114 generates various synchronization signals synchronized with the display timing of the captured image, and displays an image on the monitor 115 according to the synchronization signal.

  The image compression / decompression circuit 111 compresses the read captured image data after image processing according to the JPEG method (in the case of a still image) or the MPEG method (in the case of a moving image), and the compressed image data obtained by the compression is recorded on the recording medium. 113 is recorded.

  Next, an information providing process for providing the user with reference information related to the subject of interest will be described. This series of information provision processes is executed, for example, when displaying an image after shooting. As described above, captured image data obtained by shooting is processed in the image processing circuit 107 and then temporarily stored in the RAM 106. Thereafter, an image based on the captured image data stored in the RAM 106 is displayed on the monitor 115 in accordance with the processing of the display circuit 114. At this time, the subject of interest is extracted. The target subject may be extracted manually by the user, or the target subject extraction circuit 108 may automatically extract the target subject.

  FIG. 2 is a diagram showing a screen display of the monitor 115 when the user manually selects the subject of interest. The user can select a subject of interest by designating a predetermined area in the image 301 displayed on the monitor 115 using the selection frame 302. The selection frame 302 can be moved, for example, by a cross key 116a included in the input circuit 116, a touch panel, or the like, and the size can be changed.

  In addition, as a method for automatically extracting a subject of interest, for example, a method of extracting a subject having a large feature amount such as color, brightness, or shape from the image 301 as a subject of interest, or a focus of the optical system 101 is used. A method of extracting a matching subject as a subject of interest can be considered. The user may be allowed to select which of the automatic extraction methods is used.

  When captured image data corresponding to the subject of interest is extracted, the image data of the subject of interest is extracted by the image processing circuit 107 into shape data registered in advance in a database in the flash memory 112 or the recording medium 113 (for details). It will be resized according to the size of (described later). Thereafter, the resized captured image data of the target subject portion is sent to the feature amount extraction circuit 109. In response to this, the feature quantity extraction circuit 109 extracts the feature quantity of the subject of interest. In the present embodiment, the shape of the subject of interest and the color distribution of the subject of interest are extracted as feature amounts. As for the shape of the subject of interest, a high contrast portion (edge portion) of the subject of interest is extracted using a high-pass filter or the like. The color distribution of the subject of interest is the distribution of captured image data of the portion of the subject of interest.

  The feature quantity extracted by the feature quantity extraction circuit 109 is sent to the information acquisition circuit 110. In response to this, the information acquisition circuit 110 acquires reference information related to the reference object corresponding to the subject of interest. That is, the information acquisition circuit 110 compares the feature amount sent from the feature amount extraction circuit 109 with the shape data registered in the database in the flash memory 112 or the recording medium 113, and based on the comparison result, the object of interest The reference information related to the reference object corresponding to is acquired.

  FIG. 3 is a diagram showing an example of shape data. The shape data is image data indicating a typical shape such as a circle, an ellipse, or a polygon. The shape data for estimating the outer shape of the subject of interest as shown in FIG. 3A and FIG. There are two types of shape data for estimating the shape of a part of the subject of interest as shown in FIG.

  For example, when the subject of interest is a flower, first, flower shape data is extracted, and the flower shape data is compared with each shape data shown in FIG. Then, the degree of similarity between the extracted flower shape data and each shape data (for example, the sum of the differences between the outline data and the shape data for each pixel in percentage) is calculated. Thereby, the external shape of a flower can be estimated. Further, the shape data of the flower is compared with the shape data shown in FIG. Then, the degree of similarity between the extracted flower shape data and each shape data is calculated. Thereby, the shape of the detail of a flower can be estimated.

  After the above comparison, the reference information related to the reference object corresponding to the target object is referred to as the reference information of the reference object whose similarity to the shape of the outer shape of the target object and the similarity to the detail shape of the target object is higher than a predetermined level. Get as. For example, in the case of a flower, what is the generic name of a flower having an outer shape corresponding to each shape data in FIG. 3A, among them, a leaf of a shape corresponding to each shape data in FIG. By registering in advance in the database information such as what the individual names of the flowers having petals are, it is possible to specify the generic names and individual names of the flowers by the above comparison. That is, according to the method of the present embodiment, a rough flower is specified by the outer shape and color distribution of flowers including leaves and stems, and a fine flower is specified by the petal shape, leaf shape and color distribution. Is possible.

  As described above, in the present embodiment, the captured image data obtained by actual photographing and the actual image data are not directly compared, so that the comparison can be performed in a relatively short time. Further, by comparing not only the outlines but also the details, it is possible to specify the subject of interest with relatively high accuracy.

  Here, a surface pattern (texture structure) or the like of the subject of interest may be extracted as the feature amount. Further, if the imaging apparatus 100 is provided with a position detection function by GPS or the like, the shooting position information and the altitude information at the time of shooting can be extracted as the feature amount. Further, if the imaging apparatus 100 is provided with a clock function, the shooting date and time can be used as a feature amount. Furthermore, if the image sensing device 100 is provided with a temperature detection function, the ambient temperature of the image sensing device 100 at the time of photographing can be used as a feature amount. For example, in the case of flowers, the habitat and the flowering time are often different depending on the type of flower, so registering information such as the habitat and the flowering time in the database as reference information provides higher accuracy. The subject of interest can be identified.

  FIG. 4 is a flowchart showing the flow of processing up to the acquisition of the reference information described above. Here, a series of processing shown in FIG. 4 is executed according to control of the control circuit 117 when displaying an image after photographing.

  After displaying the image after shooting, the control circuit 117 uses the target subject extraction circuit 108 to extract the captured image data of the portion corresponding to the target subject from the captured image data (step S1). After extracting the picked-up image data of the subject of interest, the control circuit 117 resizes the picked-up image data of the subject of interest in the image processing circuit 107 and sends it to the feature amount extraction circuit 109. In response to this, the feature quantity extraction circuit 109 extracts feature quantities (step S2). After the feature amount is extracted, the control circuit 117 sends the extracted feature amount to the information acquisition circuit 110. Thereafter, the information acquisition circuit 110 performs reference information acquisition processing related to the subject of interest as described above (step S3).

  After the reference information acquisition processing in the information acquisition circuit 110, the control circuit 117 has acquired the reference information in the information acquisition circuit 110, that is, the reference information corresponding to the feature amount extracted in step S2 is stored in the database. Is registered (step S4).

  In the determination of step S4, when the information acquisition circuit 110 can acquire the reference information, the control circuit 117 displays the basic information among the reference information related to the top 10 reference objects that are considered to have high similarity. (Generic name or personal name of the reference object) is temporarily stored in the RAM 106 (step S5). Thereafter, a process for presenting reference information is performed. This reference information presentation process will be described later.

  If the reference information cannot be acquired by the information acquisition circuit 110 in the determination in step S4, the control circuit 117 has the reference information corresponding to the feature amount extracted in step S2 registered in the database. Information indicating the absence is temporarily stored in the RAM 106 (step S6). Thereafter, a process for presenting reference information is performed.

  FIG. 5 is a diagram schematically showing the processing of FIG. Note that the example of FIG. 5 shows an example in which the subject of interest is a flower.

  As described above, in the database, association information between a combination of shape data and a reference object indicated by the combination is registered in advance. In the example of FIG. 5, the “generic name” and “lily” of the flower whose shape is the shape data (star shape) 401 are associated, and the shape of the petal position is the personal name of the flower of the shape data 403 and “Casablanca”. Are associated with each other. Furthermore, detailed information such as the place of origin and flowering time of “Casablanca” is also associated. Similar association is performed for other flowers.

  In the example of FIG. 5, the similarity between the outer shape of the subject of interest 501 and the shape data 401 is the highest, and then the similarity with the shape data 402 is the highest. Therefore, it is identified that the subject of interest 501 is most likely the generic name “lily” and possibly “chrysanthemum”.

  In addition, the shape at the position corresponding to the petal of the subject of interest 501 has the highest similarity with the shape data 403, and then has the highest similarity in the order of the shape data 404 and 405. Therefore, the subject of interest 501 is identified as having a higher possibility of being the generic name “lily” and having a higher possibility of having the individual name “Casablanca”.

  In the example described here, only the shapes are compared, but in actuality, the comparison is also performed using reference information such as color distribution, texture structure, habitat and flowering time.

  The comparison as described above is performed to identify the top 10 reference objects that are considered to be highly similar to the subject of interest. Thereafter, the identified result is presented to the user. Even when the number of reference objects having a high similarity is less than ten, the reference information of the reference object having a high similarity is presented to the user.

  Next, reference information presentation processing of the imaging apparatus according to the present embodiment will be described. This process is performed after the process of FIG.

  First, reference information is registered in the database as OSD (On Screen Display) data. After the processing in FIG. 4, the captured image data stored in the RAM 106 for display on the monitor 115 and reference information related to the subject of interest are output to the display circuit 114.

  The display circuit 114 generates a timing signal for the monitor 115 and outputs the generated timing signal to the monitor 115. The display circuit 114 resizes the captured image data read from the RAM 106 according to the angle of view of the monitor 115. Further, the display circuit 114 superimposes the reference information related to the subject of interest on the captured image data and outputs it to the monitor 115. When the reference information can be acquired, basic information 601 (reference name in FIG. 6A) having a high similarity is displayed on the monitor 115 together with the similarity as shown in FIG. 6A. . The basic information may be displayed as all information corresponding to the top 10 reference objects, or only a part of the basic information may be displayed as shown in FIG. It may be switchable. On the other hand, when the reference information cannot be acquired, information 602 indicating that a reference object having a high degree of similarity has not been registered is displayed on the monitor 115 as shown in FIG.

  Here, when any information in the basic information displayed as shown in FIG. 6A is selected by the user, the selected basic information is changed as shown in FIG. Displayed.

  Next, a method for storing captured image data of the imaging apparatus according to the present embodiment will be described. The captured image data stored in the RAM 106 after shooting is output to the image compression / decompression circuit 111. The image compression / decompression circuit 111 compresses the captured image data according to a predetermined compression format such as a JPEG method. Also, a captured image file is generated by adding predetermined header information such as an Exif (Exchangeable image file format) header to the compressed image data that has been compressed. Thereafter, the captured image file is recorded on the recording medium 113.

  Here, basic information of reference information acquired from the database is written in the header information in addition to the shooting conditions (shooting mode, exposure condition, etc.) at the time of shooting. As the basic information to be written in the header information, all the basic information of the top 10 reference objects having a high degree of similarity may be written, or only the reference information selected by the user may be written. By writing the basic information of the reference information in the header information, it is possible to obtain the reference information related to the subject of interest from the basic information without newly specifying the subject of interest when the captured image data is reproduced. Also, writing only basic information without writing all reference information leads to a reduction in the capacity of the captured image file.

  Note that the feature amount extracted by the feature amount extraction circuit 109 may be written in the captured image file.

  Next, a method for reproducing captured image data of the imaging apparatus according to the present embodiment will be described. When the captured image data is reproduced, the compressed image data of the captured image file recorded in the recording medium 113 is temporarily stored in the RAM 106. The compressed image data stored in the RAM 106 is output to the image compression / decompression circuit 111. The image compression / decompression circuit 111 decompresses the compressed image data and outputs the decompressed image data to the RAM 106.

  In addition, the information acquisition circuit 110 is a database recorded in the flash memory 112 or the recording medium 113 according to the basic information of the reference information related to the captured image data written in the Exif header attached to the expanded captured image data. Get reference information from. Thereafter, the information acquisition circuit 110 temporarily stores the acquired OSD data of the reference information in the RAM 106.

  The expanded captured image data stored in the RAM 106 and the OSD data of the reference information are output to the display circuit 114. The display circuit 114 generates a timing signal for the monitor 115 and outputs the timing signal to the monitor 115. Further, the display circuit 114 resizes the captured image data in accordance with the angle of view of the monitor 115, and superimposes the OSD data of the reference information on the captured image data and outputs it to the monitor 115. Thereby, the reference information is displayed on the monitor 115 together with the captured image.

  As described above, according to the present embodiment, the subject of interest can be specified by the database in the imaging apparatus, so that the subject of interest can be specified even in a place where the radio wave condition is bad. Further, since the target subject can be specified in a relatively short time, the target subject can be specified with low power consumption as a result.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. FIG. 7 is a diagram showing a configuration of an information providing system according to the second embodiment of the present invention. The information providing system illustrated in FIG. 7 includes an imaging device 100 and an external database server 200.

  The basic configuration of the imaging apparatus 100 is the same as that of the first embodiment. However, the second embodiment is different in that the imaging apparatus 100 includes a feature amount transmission circuit 119 and a result reception circuit 120. The feature amount transmission circuit 119 transmits the captured image data or the feature amount of the subject of interest and information for identifying the user of the imaging device 100 to the external database server 200. The result receiving circuit 120 receives reference information related to the subject of interest transmitted from the external database server 200.

  The external database server 200 includes a feature amount receiving unit 201, an authentication unit 202, a server search unit 203, a server database 204, a charging unit 205, and a result transmission unit 206.

  The feature amount receiving unit 201 receives captured image data or feature amounts transmitted from the feature amount transmission circuit 119 of the imaging apparatus 100 and information for identifying the user of the imaging apparatus 100. The authentication unit 202 authenticates whether or not reference information can be provided to the imaging apparatus 100 from information for identifying a user. The server search unit 203 acquires reference information related to the subject of interest by referring to the server database 204 from the captured image data input through the feature amount receiver 201 and the authentication unit 202 or the feature amount of the subject of interest. In the server database 204, reference information related to various reference objects is registered as a database. The billing unit 205 performs billing when transmitting the reference information to the imaging apparatus 100. The result transmission unit 206 transmits the reference information acquisition result by the server search unit 203 and the charging information in the charging unit 205 to the result receiving circuit 120 of the imaging apparatus 100.

  Next, FIG. 8 is a flowchart showing a processing flow until acquisition of reference information in the second embodiment. In FIG. 8, steps similar to those in FIG. 4 are denoted by the same step symbols as in FIG. That is, steps S1 to S5 in FIG. 8 are the same as those in FIG.

  In the determination of step S4 in FIG. 8, when the information acquisition circuit 110 cannot acquire the reference information, the control circuit 117 displays information related to the subject of interest (for example, captured image data or features of the portion of the subject of interest). Amount) is temporarily stored in the RAM 106. Thereafter, the control circuit 117 requests the provision of reference information by transmitting the information related to the subject of interest stored in the RAM 106 to the external database server 200 via the feature amount transmission circuit 119 (step S16). When the communication state is poor, it is desirable to start communication after the communication state is improved.

  The feature amount receiving unit 201 of the external database server 200 outputs the received information (captured image data or feature amount) related to the subject of interest to the server search unit 203. The server search unit 203 matches the input captured image data with the actual image of the reference object registered in the server database 204 (or the feature amount may be extracted and compared) or input. The reference information related to the subject of interest is acquired by comparing the feature amount and the reference information registered in the server database 204 (step S17). After acquiring the reference information, the server search unit 203 outputs the acquired reference information to the charging unit 205. If the reference information cannot be acquired, information indicating that is output to the charging unit 205.

  The feature amount receiving unit 201 outputs information for identifying the user to the authentication unit 202 (step S18). The authentication unit 202 confirms from the information for identifying the input user that the imaging apparatus 100 that has requested the provision of reference information is performing user registration. Then, authentication information indicating the confirmation result is output to billing section 205. When it is confirmed that the imaging apparatus 100 that has requested provision of reference information has performed user registration, the charging unit 205 adds the usage fee of the external database server 200 to the reference information acquired by the server search unit 203. The information shown is added, and these pieces of information are transmitted to the imaging apparatus 100 via the result transmission unit 206 (step S19).

  The reference information and the information indicating the usage fee transmitted from the external database server 200 are received by the result receiving circuit 120. The control circuit 117 temporarily stores in the RAM 106 the reference information received by the result receiving circuit 120 and information indicating the usage fee. Further, reference information transferred from the external database server 200 is newly added to the flash memory 112.

  In general, a server can have a larger database than the imaging apparatus 100 such as a digital camera. For this reason, it is possible to extract feature amounts in more detail than the image capturing apparatus 100 and to search for data corresponding to the extracted features, and it is highly possible that the subject of interest can be identified more correctly than the image capturing apparatus 100.

  Here, when the reference information can be acquired in the determination of step S4, the reference information is displayed on the monitor 115. Information desired by the user is not necessarily included in the reference information displayed in the top 10 items. In such a case, the reference information may be transferred to the external database server 200. In this way, further reference information that cannot be obtained from the external image server 200 from the external database server 200, for example, in the case of flowers, the detailed classification name and subspecies information of the flowers, the place of origin, the growth distribution, the way of raising, etc. It is possible to obtain.

  As described above, according to the present embodiment, the same effect as that of the first embodiment can be obtained, and when the reference information cannot be acquired only by the imaging apparatus 100, the external database server 200 is used. It is possible to acquire reference information.

[Third Embodiment]
Next, a third embodiment of the present invention will be described. FIG. 9 is a diagram showing a configuration of an information providing system in the third embodiment of the present invention. The information providing system illustrated in FIG. 9 includes an imaging device 100 and an external database server 200.

  The imaging apparatus 100 according to the third embodiment includes an audio compression / decompression circuit 121, an audio circuit 122, and various sensors 123. The audio compression / decompression circuit 121 compresses the audio data of the reference information related to the subject of interest using a compression method such as MP3 (MPEG-1 Audio Layer-3) and expands the compressed audio data. The sound circuit 122 includes a microphone for inputting sound from the outside, a sound recognition circuit for recognizing sound data input via the microphone, and a speaker for outputting sound to the outside. Has been. The various sensors 123 are various sensors such as a GPS receiver circuit that acquires positioning radio waves transmitted from GPS (Global Positioning System) satellites, sensors for measuring altitude and temperature, and watches.

  Since the configuration of the external database server 200 is the same as that described in the second embodiment, the description thereof is omitted.

  In the third embodiment, the provision of reference information related to the subject of interest is limited according to the information provision level. This information provision level can be selected from basic information and detailed information. The information provision level is basic information in the initial state, and the user can set the information provision level using the input circuit 116. Further, the information provision level may be set by voice input using the voice circuit 122. Further, the information provision level may be selected from the detailed information.

  FIG. 10 is a flowchart illustrating a flow of processing up to acquisition of reference information according to the third embodiment. In FIG. 10, steps similar to those in FIG. 8 are denoted by the same steps as those in FIG.

  In step S3 of FIG. 10, after obtaining the reference information, the control circuit 117 determines whether or not only the basic information of the reference information is presented to the user (step S100). If it is determined in step S100 that only basic information is presented, that is, if the information provision level is set to basic information, the same processing as in FIG. 8 is performed. That is, the control circuit 117 determines whether or not the reference information can be acquired by the information acquisition circuit 110, and displays the reference information on the monitor 115 when the reference information can be acquired. On the other hand, when the reference information cannot be acquired, the control circuit 117 requests the external database server 200 to provide the reference information.

  In the determination of step S100, when the detailed information other than the basic information is presented, that is, when the information provision level is set to the detailed information, the control circuit 117 determines the captured image data of the subject of interest and the various sensors 123. The detection result (shooting position information acquired via the GPS receiver circuit, altitude measured by the altitude sensor, temperature measured by the temperature sensor, shooting time measured by the clock, etc.), and information provision level, feature quantity The data is transmitted to the external database server 200 via the transmission circuit 119 (step S101). The server search unit 203 of the external database server 200 acquires reference information from the server database 204 based on the information transmitted in step S101. After acquiring the reference information, if the provision level of detailed information is set, the server search unit 203 outputs only the reference information corresponding to the provision level to the accounting unit 205 (step S102). Thereafter, the authentication unit 202 confirms that the imaging device 100 that has requested the provision of reference information is performing user registration. Then, authentication information indicating the confirmation result is output to billing section 205. When it is confirmed that the imaging apparatus 100 that has requested provision of reference information has performed user registration, the charging unit 205 adds the usage fee of the external database server 200 to the reference information acquired by the server search unit 203. The information shown is added, and these pieces of information are transmitted to the imaging apparatus 100 via the result transmission unit 206 (step S19).

  The reference information and the information indicating the usage fee transmitted from the external database server 200 are received by the result receiving circuit 120. The control circuit 117 temporarily stores in the RAM 106 the reference information received by the result receiving circuit 120 and information indicating the usage fee. Further, reference information transferred from the external database server 200 is newly added to the flash memory 112.

  FIG. 11 is a diagram illustrating a display example when the information provision level is detailed information. In the third embodiment, the detailed information of the reference information related to the subject of interest is composed of OSD data and compressed audio data. When the detailed information of the reference information is transmitted from the external database server 200, the transmitted reference information is temporarily stored in the RAM 106. Thereafter, reference information related to the subject of interest is output to the display circuit 114 together with the captured image data. The display circuit 114 generates a timing signal for the monitor 115 and outputs the timing signal to the monitor 115. The display circuit 114 resizes the captured image data in accordance with the angle of view of the monitor 115, superimposes the OSD data of the reference information on the resized captured image data, and outputs the superimposed data to the monitor 115. Thereby, the detailed information of the reference information as shown in FIG.

  On the other hand, the compressed audio data of the reference information related to the subject of interest is output to the audio compression / decompression circuit 121. The audio compression / expansion circuit 121 expands the compressed audio data of the reference information. The expanded audio data is temporarily stored in the RAM 106. The audio data stored in the RAM 106 is output to the audio circuit 122. Based on the audio data, audio related to the subject of interest is output from the speaker of the audio circuit 122.

  Next, a method for storing captured image data of the imaging apparatus according to the present embodiment will be described. The captured image data stored in the RAM 106 after shooting is output to the image compression / decompression circuit 111. The image compression / decompression circuit 111 compresses the captured image data according to a predetermined compression format such as a JPEG method. Also, for example, predetermined header information such as an Exif (Exchangeable image file format) header is added to the compressed image data that has been compressed to generate a captured image file. Thereafter, the captured image file is recorded on the recording medium 113. Further, when the compressed information is included in the reference information, the compressed sound data is also recorded in the captured image file.

  Next, a method for reproducing captured image data of the imaging apparatus according to the present embodiment will be described. When the captured image data is reproduced, the compressed image data and the compressed audio data of the captured image file recorded on the recording medium 113 are temporarily stored in the RAM 106. The stored compressed image data is output to the image compression / decompression circuit 111. The image compression / decompression circuit 111 decompresses the compressed image data and outputs the decompressed image data to the RAM 106. On the other hand, the compressed audio data is output to the audio compression / expansion circuit 121. The voice compression / decompression circuit 121 decompresses the compressed voice data and outputs the decompressed voice data to the RAM 106.

  In addition, the information acquisition circuit 110 is a database recorded in the flash memory 112 or the recording medium 113 according to the basic information of the reference information related to the captured image data written in the Exif header attached to the expanded captured image data. Get reference information from. Thereafter, the information acquisition circuit 110 temporarily stores the acquired OSD data of the reference information in the RAM 106.

  The expanded captured image data stored in the RAM 106 and the OSD data of the reference information are output to the display circuit 114. The display circuit 114 generates a timing signal for the monitor 115 and outputs the timing signal to the monitor 115. Further, the display circuit 114 resizes the captured image data in accordance with the angle of view of the monitor 115, and superimposes the OSD data of the reference information on the captured image data and outputs it to the monitor 115. Thereby, the reference information is displayed on the monitor 115 together with the captured image. The decompressed audio data stored in the RAM 106 is output to the audio circuit 122. The audio circuit 122 reproduces the audio data and outputs audio related to the reference object from the speaker.

  As described above, according to the present embodiment, the same effects as those of the second embodiment can be obtained, and information to be presented to the user can be limited according to the information provision level.

  Although the present invention has been described above based on the embodiments, the present invention is not limited to the above-described embodiments, and various modifications and applications are naturally possible within the scope of the gist of the present invention.

  Further, the above-described embodiments include various stages of the invention, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some configuration requirements are deleted from all the configuration requirements shown in the embodiment, the above-described problem can be solved, and this configuration requirement is deleted when the above-described effects can be obtained. The configuration can also be extracted as an invention.

It is a figure which shows the structure of the imaging device which concerns on the 1st Embodiment of this invention. It is a figure which shows the screen display of a monitor when a user selects an attention object manually. It is the figure which showed an example of shape data. It is a flowchart shown about the flow of a process until acquisition of reference information. It is the figure which showed the process of FIG. 4 typically. It is a figure which shows the example at the time of displaying reference information. It is a figure which shows the structure of the information provision system in the 2nd Embodiment of this invention. It is a flowchart shown about the flow of a process until acquisition of the reference information in the 2nd Embodiment of this invention. It is a figure which shows the structure of the information provision system in the 3rd Embodiment of this invention. Processing flow until acquisition of reference information in the third embodiment of the present invention It is a figure which shows the example of a display when information provision level is made into detailed information.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 ... Imaging device 101 ... Optical system 102 ... Image sensor 103 ... Analog / digital (A / D) conversion circuit 104 ... Pre-processing circuit 105 ... Focus detection circuit 106 ... Random access memory (RAM), 107 DESCRIPTION OF SYMBOLS ... Image processing circuit 108 ... Interesting subject extraction circuit 109 ... Feature amount extraction circuit 110 ... Information acquisition circuit 111 ... Image compression / decompression circuit 112 ... Flash memory 113 113 Recording medium 114 ... Display circuit 115 Monitor 116 ... Input circuit, 116a ... Cross key, 117 ... Control circuit, 118 ... Data bus, 119 ... Feature transmission circuit, 120 ... Result reception circuit, 121 ... Audio compression / decompression circuit, 122 ... Audio circuit, 123 ... Various sensors 200 ... external database server, 201 ... feature receiving unit, 202 ... authentication unit, 203 ... server Bas search section, 204 ... Server Database, 205 ... accounting unit, 206 ... result transmitting unit

Claims (22)

An imaging unit that images a subject and obtains a captured image;
A subject extraction unit that extracts an image of the subject of interest from the captured image;
A feature amount extraction unit that extracts a feature amount related to the subject of interest from the extracted image of the subject of interest;
An information acquisition unit that acquires reference information related to a reference object corresponding to the subject of interest from the extracted feature amount;
An information providing unit for providing the acquired reference information to a user;
An imaging apparatus comprising:   The imaging apparatus according to claim 1, wherein the information acquisition unit includes a storage unit that stores the feature quantity and the reference information in association with each other.   2. The reference information according to claim 1, wherein the reference information includes at least one of a generic name of the reference object, a personal name of the reference object, an image indicating the reference object, and a code for identifying the reference object. Imaging device.   The imaging apparatus according to claim 3, wherein the reference information further includes detailed information regarding the subject of interest.   The feature amount extraction unit extracts at least one of a feature amount relating to the entire subject of interest and a feature amount relating to a part of the subject of interest as a feature amount relating to the subject of interest. The imaging apparatus according to 1.   The feature amount related to the entire subject of interest includes the outer shape of the subject of interest, the overall color distribution of the subject of interest, and the texture structure of the entire subject of interest. The imaging apparatus according to claim 5, comprising a shape of a part of the subject of interest, a color distribution of a part of the subject of interest, and a texture structure of a portion of the subject of interest.   The feature amount extraction unit includes information about a position where the captured image is acquired, information about an altitude of a position where the captured image is acquired, information about a time when the captured image is acquired, and when the captured image is acquired The image pickup apparatus according to claim 5, wherein at least one of the information related to the ambient temperature of the image pickup apparatus is further extracted as a feature amount related to the subject of interest.   The imaging apparatus according to claim 2, wherein the storage unit is configured to be detachable from the imaging apparatus.   The imaging apparatus according to claim 2, wherein the information acquisition unit stores the acquired reference information in the storage unit in association with the extracted subject of interest.   The imaging apparatus according to claim 2, wherein the information acquisition unit stores the acquired reference information in the storage unit in association with the captured image.   The imaging apparatus according to claim 10, wherein the information providing unit provides the user with reference information associated with the captured image when reproducing the captured image stored in the storage unit.   The imaging apparatus according to claim 1, wherein the information acquisition unit limits acquisition of the reference information according to an acquisition level of reference information.   The imaging apparatus according to claim 1, wherein the information providing unit provides the reference information to a user in a form of text display or sound. An optical system that forms an image of the subject on the imaging unit during the imaging;
A focus detection unit for detecting a focus state of the optical system;
Including
The imaging apparatus according to claim 1, wherein the target subject detection unit extracts, as a target subject, a subject that is in focus as a result of detection of a focus state by the focus detection unit. Including an input member for designating an arbitrary subject in the captured image;
The imaging apparatus according to claim 1, wherein the subject-of-interest detection unit extracts a subject designated via the input member as the subject of interest.   The imaging apparatus according to claim 1, wherein the target subject detection unit resizes the captured image before extracting an image of the target subject. A feature amount transmitting unit that transmits the feature amount extracted by the feature amount extraction unit to an external database server that is connected to the outside of the imaging apparatus so as to be communicable and acquires reference information corresponding to the subject of interest from the feature amount. When,
A feature receiving unit for receiving reference information acquired by the external database server;
The imaging apparatus according to claim 1, further comprising: When reference information corresponding to the feature amount extracted by the feature amount extraction unit is not stored in the storage unit or when a user instruction is received, the feature amount extracted by the feature amount extraction unit is A feature amount transmitting unit that is connected to the outside of the imaging device so as to be communicable and transmits reference information corresponding to the subject of interest from the feature amount,
A feature receiving unit for receiving reference information acquired by the external database server;
The imaging apparatus according to any one of claims 2, 8, 9, 10, and 11, further comprising: An information providing system comprising: the imaging device according to claim 17 or 18; and an external database server connected to the imaging device in a communicable manner,
A server search unit for acquiring reference information corresponding to the subject of interest from the feature amount;
A determination unit that determines whether the reference information acquired by the server search unit can be provided to the imaging apparatus;
A charging unit that charges according to provision of the reference information acquired by the server search unit to the imaging device when it is determined by the authentication unit that the reference information acquired by the server search unit can be provided When,
An information providing system comprising: The feature amount transmitting unit transmits information necessary for specifying a user in addition to the feature amount to the external database server,
The information providing system according to claim 19, wherein the external database server further obtains reference information using information necessary for specifying the user. The feature amount transmission unit transmits the captured image to the external database server in addition to the feature amount,
The information providing system according to claim 19, wherein the external database server acquires reference information by extracting a feature amount related to the target subject from the captured image.   The information providing system according to claim 19, wherein, when the reference information is acquired by the server search unit, the charging unit performs charging according to the acquisition of the reference information.

Images