JP4659569B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus for efficiently classifying and searching captured images.

  With the spread of digital cameras and the spread of scanners, digitization of images is progressing. In particular, in the case of news photographs used in newspaper companies, for example, in a baseball game, 1000 or more images may be taken in one game. In order to facilitate later retrieval, tag data such as date, photographer, and the like, which is data in a region other than the image information added to the image (so-called tag data), is added to the image. However, in order to make the search easier, it is desirable that information “what is reflected” is added to the image. Therefore, as a means for adding the information “what is reflected” to the image, a means for the photographer to manually input can be considered. However, as the number of photos increased, it took time and effort to manually enter information.

In order to solve this problem, a technique is known in which a camera automatically adds information “what is captured” to an image (see Patent Document 1). The technique detects a characteristic portion of the subject, for example, a bib number worn on the face / body of the subject of the image. This is a technique for creating a tag indicating information “what is captured” by recognizing who the subject is from information detected by the camera. By using this technique, the photographer can save the trouble of manually adding the tag and can concentrate on the original photographing.
JP-A-5-89244

  However, in the conventional technology, there are only a limited number of parts that can be used by the camera for object recognition alone. For example, even if a camera can detect a face or race bib, it can be recognized that the subject cannot be recognized unless at least the face is detected. The place depends on whether or not there is. Therefore, when the same subject is photographed with various compositions, there is a problem that the recognition accuracy of the subject is greatly influenced by the composition.

In order to solve the above-described problems, the present invention provides an imaging unit that photoelectrically converts a subject image formed by a photographic lens to obtain a captured image, a communication unit that communicates with an external device through a communication line, and a predetermined timing. The subject information receiving means for receiving information on the subject that is updated and periodically transmitted from the external device, and the feature amount of the face area and the character area of the character area from the captured image obtained by the imaging means. extraction means for extracting, on the information received by the object information receiving means, in association with the feature amount of information of the character area, wherein the amount of information of the face region extracted from the photographed image by the extraction unit Storage means for storing in a medium, and addition means for adding information received by the subject information receiving means to the photographed image are provided.

  According to the present invention, the subject can be recognized by the camera automatically extracting the feature of the subject using the detection technique and collating it with the database. This eliminates the need for the photographer to manually input subject information. In addition, since collation is performed from a plurality of features, the recognition accuracy is less affected by the composition.

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. However, the constituent elements described in this embodiment are merely examples, and are not intended to limit the scope of the present invention only to them.

Example 1
The first embodiment will be described below.

  FIG. 1 shows a schematic diagram of a photographing system in which a news photographer photographs a baseball game in the stadium 101.

  Reference numeral 102 denotes a ground where the game is actually played, and a batter box 110 and an electric bulletin board 103 are provided. The electronic bulletin board 103 is provided with a speaker 104, and game information is announced at any time.

  The stadium 101 is provided with a photographer seat 105. The photographer seat 105 is a seat for the photographer 106 to photograph a game. 109 is a batter and stands in the batter box 110. The photographer 106 uses the digital camera 107 to photograph the batter 109.

  In addition, a wireless LAN access point 111 is provided in the stadium 110, and the digital camera 107 can be connected to the Internet 112 through this access point. Reference numeral 113 denotes a broadcast station server. The broadcast station server 113 operates a Web site and distributes game information as needed via the Internet 112. The digital camera 107 can acquire information distributed from the broadcast station server 113 via the Internet 112.

  An image taken by the cameraman 106 is transferred from the digital camera 107 to a personal computer (hereinafter referred to as a PC) 108. The PC 108 is connected to the newspaper company 114 via a network. The image transferred from the digital camera 107 to the PC 108 is transferred to the newspaper company 114 and used as an editing material.

  Various known techniques can be used to transfer an image from the digital camera 107 to the PC 108. For example, the cameraman 106 may connect the digital camera 107 or a recording medium (not shown) in which an image is stored to the PC 108 after the shooting, and the digital camera 107 may transfer the image to the PC 108. Further, the digital camera 107 may transfer an image to the PC 108 by wireless communication according to an instruction from the cameraman 106.

  FIG. 2 is a block diagram of the digital camera 107 shown in FIG. Reference numeral 219 denotes a lens that forms a subject image. Reference numeral 209 denotes an image sensor 209 such as a CCD having a photoelectric conversion function for outputting an object image formed by the lens 219 as an analog image. Reference numeral 211 denotes a display for displaying an image. A socket 208 enables the memory card 207 to be attached and detached. Reference numeral 204 denotes a system storage unit such as a flash memory. The system storage unit 204 stores an operation program of the image capturing apparatus, property data necessary for the image capturing operation, a communication module necessary for communication with an external device, and a web browser for browsing a website. Reference numeral 206 denotes a DRAM which holds work data when a CPU 203 described later executes a program. In addition, an image generated by the photographing operation of the digital camera 107 and an image to be displayed on the display 211 are held.

  The digital camera 107 also includes a mode switch 221 for the user to specify the operation mode of the digital camera 107, that is, the shooting mode or the playback mode. In addition, an operation switch 222 including a plurality of switches such as a cross key, a set button, and a cancel button for a user to input an instruction to the digital camera is provided. Further, a shutter button 223 that instructs to start imaging at the time of imaging is provided.

  The imaging sensor 209 is connected to the imaging circuit 210 via the AD converter 228. The imaging circuit 210 generates a digital image based on the image output from the imaging sensor 209 and records it in the DRAM 106 via the data bus 202.

  The digital camera 107 includes a display circuit 212 that generates a digital image to be displayed on the display 211 from the digital image stored in the DRAM 206. Also, a DA converter 229 that converts the generated digital image into an analog image and outputs the analog image is provided. The imaging circuit 210 and the display circuit 212 are connected to each other via the data bus 202.

  The digital camera 107 also includes a memory card controller 213 and a system storage unit controller 215 connected to the data bus 202, an IO 225, and a CPU 203, which are connected to each other via the data bus 202.

  The digital camera further includes a memory controller 205 connected to the DRAM 206 and connected to the data bus 202.

  The mode switch 221, the operation switch 222, and the shutter button 223 are connected to the IO 225.

  The memory card controller 213 is configured to issue commands and transfer data to the memory card 207.

  The data bus 202 transmits an image at the time of photographing or image reproduction in response to a request from the photographing circuit 210 or the display circuit 212 as a bus master.

  The memory controller 205 receives an access request to the DRAM 206 from the data bus 202, and generates an access signal to the DRAM 206 while arbitrating this request. Then, data writing to the DRAM 206 and data reading from the DRAM 206 are executed.

  The CPU 203 controls the camera sequence according to the program stored in the system storage unit 204. The processes shown in FIGS. 3, 9, and 18 to be described later are executed according to the program stored in the system storage unit 204.

  This apparatus performs a still image shooting operation when the user presses the shutter button 223.

  As a result, the image sensor 209 converts the image into an electrical signal, and outputs this to the AD converter 228. The AD converter 228 converts this into a digital value and outputs it to the imaging circuit 210. The imaging circuit 210 adjusts exposure and white balance, further performs data compression processing, and temporarily records the generated compressed image in the DRAM 206 via the data bus 202 and the memory controller 205. The memory card controller 213 records the compressed image generated in the DRAM 206 on the memory card 207.

  The digital camera 107 further includes a feature detection circuit 200 and a parameter holding unit 201 that holds parameter values necessary for feature detection. The parameter values held by the parameter holding unit 201 are rewritable, and it is possible to record the appearance features of the subject such as a human face, numbers, and scenery.

  In addition, the parameter holding unit can record not only the appearance features but also the features of sounds produced by humans and machines.

  In addition, the imaging apparatus includes a feature value storage circuit 230 that temporarily stores feature values, and a feature value comparison circuit 231 that compares feature values.

  The system storage unit 204 records values indicating the characteristics of the subject. For example, a value obtained by digitizing the positional relationship between the face outline, eyes, nose, and mouth, a value indicating the characteristics of a character pattern, and the characteristics of voices produced by humans and machines are recorded. A character pattern refers to a portion representing a character included in an image. Examples thereof include a back number and a signboard character. When searching for the face portion / number number portion from the subject, this value is loaded into the parameter holding unit 201.

  The feature detection circuit 200 detects a portion showing a desired feature based on input image or sound data and parameter values. Further, a feature value of a portion indicating these desired features is calculated. The digital camera is configured so that the CPU 203 can read these values via the data bus.

  The feature detection circuit 200 changes the parameter value, for example, in the case of an image, the characteristics of a human face, animal face, Arabic numerals, English, hiragana, etc. It is possible to extract features such as electronic sounds.

  In addition, the apparatus can store the feature value generated by past shooting in the memory card 207 or the system storage unit 204. The feature value storage circuit 230 is a circuit that stores the stored feature value, and the CPU 203 can transfer the feature value to the feature value storage circuit 230.

  The feature value comparison circuit 231 compares feature values, compares the feature value output from the feature detection circuit 200 with the feature value output from the feature value storage circuit 230, and calculates the degree of coincidence. The CPU 203 can read the result calculated by the feature value comparison circuit.

  The microphone 233 is a device that reads voice and converts it into an electrical signal as voice data, and is converted into a digital signal by the AD converter 232.

  The voice recognition circuit is a circuit that recognizes voice data sent from the AD converter 232 and converts it into text data.

  The network interface 234 is an interface for connecting to a wireless LAN access point, and can connect to the Internet via the access point to communicate with other information processing apparatuses.

  In this embodiment, it is described that the feature detection circuit 200, the parameter holding unit 201, the feature value storage circuit 230, and the feature value comparison circuit 231 each detect a face, a character pattern, a voice, and the like with one circuit. is doing. However, in the present embodiment, a physically separate circuit may be provided for detecting a face, a character pattern, voice, and the like.

  Next, an operation in which the digital camera 107 receives game information distributed from the broadcast station server 113 will be described with reference to the flowchart shown in FIG.

  Hereinafter, processing executed by the CPU 203 in accordance with a program installed in the digital camera 107 will be described.

  The memory card 207 stores a URL of a site that distributes game information provided by a broadcasting station in advance before a game.

  In step S301, the digital camera 107 is in a connection standby state and waits for a connection instruction from the user.

  The photographer 106 instructs the digital camera 107 to connect to the Internet 112 using the operation switch 222. Upon receiving the connection instruction, the CPU 203 starts connection to the access point 111 via the network interface 234 (steps S302 and S303).

  When the connection to the access point is performed, the connection from the access point 111 to the Internet 112 is performed. When the CPU 203 determines that the connection to the Internet has been completed (step S304), it reads the URL of the site provided by the broadcasting station recorded in the memory card 207 (step S305).

  Then, the CPU 203 accesses the broadcasting station server 113 based on the read URL (S306).

  The CPU 203 sets the digital camera 107 in a reception standby state, and waits for game information to be distributed from the broadcast station server 113 (step S307).

  The broadcast station server 113 transmits the game information to the digital camera 107 that has made access.

  The CPU 203 receives the game information transmitted from the broadcast station server 113 to the digital camera 107 via the network interface 234 (step S308), and records the game information in the DRAM 206 (step S309). Further, the information about the player is written in the database in the digital camera 107, and the process is terminated (step S310). This process will be described later.

  The game information is updated and distributed by the broadcast station server 113 whenever the game is changed. That is, if the digital camera 107 is connected to the Internet during a game, the cameraman 106 can automatically obtain the latest game information.

  Here, the data structure of game information is demonstrated.

  FIG. 4 shows the data structure of game information received by the digital camera 107. The game information includes basic information 401 that is basic information of the game, progress information 402 that indicates the progress of the game, and player information 403 that indicates the information of the participating players. The contents of these data are text data.

  FIG. 5 shows the data structure of the basic information 401. Reference numeral 501 denotes a game name. 502 shows the start time of a game. Reference numeral 503 denotes a stadium where a game is being played.

  FIG. 6 shows the data structure of the progress information 402. Reference numeral 601 denotes the number of times, which indicates the current number of times. Reference numeral 602 denotes the name of the team currently attacking. Reference numeral 603 denotes the weather of the stadium. Reference numeral 604 denotes the temperature of the stadium. Reference numeral 605 indicates the current score. FIG. 6 shows a situation where team A has 4 points and team B has 3 points. Reference numeral 606 denotes a game time, which is an elapsed time from the start of the game until the progress information is updated.

  In addition, progress information changes with progress of a game. Accordingly, the broadcast station server 113 is updated periodically or when information is changed.

  FIG. 7 shows the data structure of the player information 403. The player information is composed of three parts: a batting order, a player name, and a spine number. Reference numeral 701 denotes a batting order, which is from 1 to 9. Reference numeral 702 denotes a player name. Reference numeral 703 denotes the player's number.

  In addition, player information changes with a change of a player. Accordingly, the broadcast station server 113 is updated periodically or when information is changed.

  In the present embodiment, the basic information 401, the progress information 402, and the player information 403 are distributed as a single piece of data called game information. However, these three pieces of information are distributed as separate data. It may be taken.

  For example, the broadcast station server 113 distributes only the basic information 401 and the player information 403 before the start of the game, the progress information 402 is updated and distributed every time the offense and defense changes, and the player information 403 is only when the player is changed. It is good also as a structure which updates and delivers.

  The data structure of the game information has been described above. Hereinafter, the operation of the digital camera 107 after receiving the game information will be described.

  When the game information is received and the recording in the DRAM 206 is completed, the CPU 203 starts writing the player information in the database (step S310 in FIG. 3).

  FIG. 8 shows a database 801 stored in the system storage unit 204. The database 801 is composed of four items. Reference numeral 802 denotes a batting order item, in which the batting order of the players is registered. Reference numeral 803 denotes a player name item in which the name of the player is registered. Reference numeral 804 denotes a back number item, in which a player's back number is registered. Reference numeral 805 denotes a face pattern item, in which a facial feature value of the player is registered. The face feature value will be described later.

  The CPU 203 reads game information from the DRAM 206. The data entered in the batting order 701 in the player information is the batting order item 802 of the database 801, the data of the player name 702 is the player name item 803, the data of the spine number 703 is the spine number item 804, and each item is entered. Register in association.

  By this processing, the batting order, the player name, and the spine number are associated and registered in the database 801 in the camera. A database 801 in which player information is registered is shown in FIG.

  Next, the flow of shooting in the present embodiment will be described using the flowchart shown in FIG.

  Hereinafter, processing executed by the CPU 203 in accordance with a program installed in the digital camera 107 will be described.

  At this point, the database is in the state shown in FIG. Here, a case will be described in which a subject for which information relating to a face has not yet been registered is photographed.

  Here, the object to be detected from the subject is the face / back number, and the parameter holding unit 201 records values indicating the respective characteristics in advance.

  In step S901, the digital camera 107 starts a shooting mode.

  In step S902, the CPU 203 starts standby of the shutter button 223 for starting the control of the display 211 and the photographing operation.

  In step S903, the CPU 203 confirms whether the shutter button 223 has been pressed. If the shutter button 223 has not been pressed, the process returns to step S902 to repeat the process.

  In step S903, when the CPU 203 determines that the cameraman 106 has pressed the shutter button 223, the process proceeds to step S904, and shooting is started.

  In step S904, the image formed on the image sensor 209 is converted into an electrical signal. Then, exposure, white balance adjustment, and the like are performed by image processing, and then stored in the memory. In addition, compressed images are generated in different areas of the DRAM 106. The CPU 203 compresses this image, and the further compressed image is transferred to the memory card 207.

  As a specific example of this embodiment, an image taken in FIG. 11 is shown. The image shown in FIG. 11 has a composition in which the face and the number can be confirmed.

  In step S905, the CPU 203 writes the game information in the captured image. At this time, the CPU 203 reads out the game information shown in FIG. The CPU 203 writes the contents of the basic information shown in FIG. 5 and the contents of the progress information shown in FIG. 6 in the header portion of the photographed image.

  By this processing, various information such as the game name 501, the number of innings 601 at the time of shooting, and the weather 603 at the time of shooting are added to the image. This information is useful information when the photographer 106 retrieves a desired image from a large number of images at a later date.

  Note that if game information is written in all the captured images, the total amount of data in the memory card 207 may increase. Therefore, before writing the game information, the CPU 203 may perform a detection / recognition process (steps S906 to S914), which will be described later, and write the game information on the image only when it is determined that the subject has been recognized.

  In step S906, the CPU 203 starts face detection processing. At this time, the CPU 203 transfers the image stored in the DRAM 206 to the feature detection circuit 200 via the data bus 202. The feature detection circuit 200 detects a face region by using a known face detection technique for the transferred image, and extracts face position information and characteristic part information in the image. The CPU 203 stores this part information as a feature value in the DRAM 206. The characteristic part information of the face is, for example, image information of eyes, nose, mouth, and the like.

  The face portion of the subject is usually near the focusing position of the lens. Utilizing this fact, the CPU 203 can detect the face area of the subject with higher accuracy by performing face detection processing inside a so-called autofocus frame.

  Known face detection techniques are disclosed in, for example, Japanese Patent Application Laid-Open No. 10-232934 and Japanese Patent Application Laid-Open No. 2000-48184, and generally face recognition is performed by combining a plurality of methods disclosed in these documents. Can be considered.

  In step S906, the CPU 203 performs a spine number detection process. Similar to face processing, the feature number detection circuit 200 uses a known pattern detection technique for an image as well as face processing.

  When detecting the turn number, the CPU 203 transfers the detected data to the OCR circuit 236. The transferred data is OCR processed and converted to text data. The CPU 203 stores this data in the DRAM 206 as a feature value of the spine number.

  When detecting the number, it is possible to use the result of face detection described above. Usually, the back number exists on the back of the subject, that is, in the vicinity of the face. By using this fact, it is possible to distinguish the character string pattern in the vicinity of the face from the other character string patterns in the image by determining that the character string pattern is a back number. By using this reference, the accuracy of the identification number detection can be increased.

  In the image shown in FIG. 11, in FIG. 12, the rectangular area 1201 is detected as a face and the rectangular area 1202 is detected as a back number, and the feature value in that area is extracted.

  In the image shown in FIG. 11, after the spine number is detected, the detected spine number is OCR processed by the OCR unit 136 to be converted into text data “55”. This text data is stored in the DRAM 206 as a feature value.

  In the present embodiment, the number to be detected is the back number, but a number plate or the like that the subject has may be the detection target.

  If the CPU 203 determines in step S908 that at least one of the spine number and the face has been detected, in step S909, the CPU 203 reads the records stored in the database 801 in FIG. .

  If neither the identification number nor the face is detected, the CPU 203 returns the process to step S902. The digital camera 107 enters a shutter standby state.

  In step S910, the CPU 203 compares the feature value recorded in the record read from the database 801 in FIG. 10 with the feature value detected in steps S906 and 907.

  If the CPU 203 determines that none of the feature values registered in the record read from the database 801 in FIG. 10 matches the feature values detected in steps S906 and 907, the process proceeds to step S902. Return to. The digital camera 107 enters a shutter standby state.

  A case will be described in which it is determined that some of the feature values read from the database 801 in FIG. 10 match the feature values detected in steps S906 and S907. In this case, the CPU 203 writes the text data input to the player name item 803 in the read record in the header portion of the photographed image, and proceeds to step S913.

  Specifically, a case will be described in which the feature values registered in the database 801 shown in FIG. 10 are compared with the feature values extracted from the image shown in FIG. The CPU 203 searches the character string data “55” obtained by performing OCR processing on the detected spine number and whether the data matching the extracted facial feature value is registered in the spine number item 804 and the face pattern item 805. The CPU 203 reads the records recorded in the database 801 in FIG. 10 in order from the record number 1, and determines whether or not the character string data 55 exists in the spine number item. Also, it is determined whether or not the face pattern item 805 has the same pattern as the detected face pattern.

  In the database 801 in FIG. 10, “55” exists in the uniform number item 804. Therefore, the CPU 103 determines that the spine number 55 exists in the database 801 of FIG. 10 when comparing this record with the feature value, and is registered in the player name item 803 of the read record in the header portion of the photographed image. “SUZUKI”, which is the stored data, is written.

  The facial feature value is not registered in the database 801 of FIG. Therefore, the CPU 203 determines that the same pattern as the detected face pattern does not exist in the face pattern item 805 and writes nothing in the header.

  In step S913, the CPU 203 determines whether another feature value (hereinafter referred to as a new feature value) has been extracted in addition to the feature value (hereinafter referred to as a matched feature value) that matches the feature value read from the database. to decide. If it is determined that a new feature value has been detected, the CPU 203 advances the process to step S914. If the CPU 203 determines that a new feature value has not been detected, the CPU 203 returns the process to step S902 to place the digital camera 107 in a shutter standby state.

  From the image shown in FIG. 11, the feature values of the face and the number are extracted. Of these, facial feature values are not registered in the database 801 of FIG. Accordingly, the feature value of the face is determined as a new feature value.

  In step S914, the CPU 203 registers the detected new feature value in the same record number as the matching feature value, and returns to the shutter standby state.

  That is, among the feature values extracted from the image shown in FIG. 11, the face feature value that is the new feature value is registered in the face pattern item of the record having the same record number as the back number “55” that is the matching feature value. . The database at this time is shown in FIG.

  When the above processing ends, the CPU 203 returns the processing to step S902, and the digital camera 107 enters a shutter standby state.

  Next, a case where a subject whose face has been detected in the past and whose face pattern is registered is photographed will be described.

  The database 801 at this time is as shown in FIG. Originally, the facial feature value is described with various parameters, but here, for convenience, the facial feature value is “XXX”.

  Steps S901 to S905 are the same as those for the first image. Assume that the image shown in FIG. 14 has been taken in step S904.

  In this composition, as a result of the CPU 203 performing the detection processing in steps S906 and S907, only the face shown in the area 1501 in FIG. 15 is detected, and the feature value is extracted. Since the composition of the image shown in FIG. 14 does not include a back number, the back number is not detected. However, since a face is detected, the CPU 203 advances the process from step S908 to step S909. In step S909, the database 801 shown in FIG. 13 is read.

  In step S910, the CPU 203 searches whether the data matching the extracted facial feature value is registered in the face pattern item 805 of the database 801 in FIG.

  Since the facial feature value “XXX” of the player name “SUZUKI” is registered when the first picture is taken, a feature value “XXX” that matches the feature value extracted from the detected face is searched from the database 801 in FIG. Is done. Accordingly, in step S911, the CPU 203 determines that the extracted facial feature value is registered in the database 801 of FIG.

  The CPU 203 determines that the subject whose image has the player name “SUZUKI” has been captured, and writes the value “SUZUKI” of the player name item in the header of the captured image.

  In step S913, the CPU 203 determines that there is no feature value detected other than the face pattern, that is, no new feature value, and sets the camera in the shutter standby mode.

  In this embodiment, various types of detection are performed after shooting, but the detection timing is not limited after shooting. For example, detection may be performed when an electronic viewfinder (hereinafter referred to as EVF) is performed.

  The EVF is a function that indicates a subject to be photographed to the user by repeatedly displaying the image on the display 211 formed on the image sensor 209.

  When detection is performed during EVF, the CPU 203 performs detection processing as needed before shooting. When the shutter button is pressed, the CPU 203 compares the face and back number detected at that time with the database 801 in the camera digital camera 107.

  In this embodiment, by acquiring data related to the subject from the server, it is possible to easily acquire data according to the shooting location, date and time, and event. Moreover, the communication load of a digital camera can be reduced by making the data acquired from a Web server into text data. Further, once the newly extracted feature value of the subject is associated with another feature value, the newly registered feature value can be used independently as an item for identifying the subject. Accordingly, the accuracy of subject recognition is less affected by the composition.

(Example 2)
The second embodiment will be described below. The description of the same parts as those in the first embodiment will be omitted, and the characteristic parts of the second embodiment will be described in detail.

  The configuration of the system and device shown in FIGS.

  FIG. 16 shows a database 1601 in the digital camera 107 in this embodiment. In this embodiment, a batting order item 1602 player name item 1603, a spine number item 1604, a face pattern item 1605, and a voice pattern item 1606 are set in the items of the database 1601.

  The game information shown in FIG. 4 is received from the broadcast station server 113 via the Internet 112 and registered in the database 1601 in the digital camera 107 as in the first embodiment. The contents of the database 1601 at the time of game information reception are shown in FIG.

  Next, the operation of the imaging apparatus in the present embodiment will be described using the flowchart shown in FIG. Here, consider a case where the photographer 106 has photographed a subject that is not registered in the database for both the facial feature value and the voice feature value.

  Since steps S1801 to S1804 are the same as those in the first embodiment, they are omitted here.

  As a specific example of the present embodiment, it is assumed that the image shown in FIG. 19 is taken in step S1804. The image shown in FIG. 19 has a composition in which the back number of the subject can be confirmed but the face cannot be confirmed.

  When shooting is performed in step S1804, the CPU 203 advances the process to step S1805. In step S1805, the CPU 203 sets the camera to the voice recording mode. The voice recording mode is a mode for recording a voice input from the microphone 233 for a predetermined time, for example, 5 seconds after the shutter is pressed.

  The cameraman 106 can input a sound for identifying the subject from the microphone 233 during a certain time in the sound recording mode. For example, it is assumed that the cameraman 106 recognizes that the subject player name is “Tom”. In such a case, voice data “Tom” is input by saying “Tom” to the microphone 233 immediately after the image is captured.

  The input audio data is converted into a digital signal via the AD converter 232, and then associated with the image and stored in the DRAM 206 as a separate file.

  Note that the audio data is not a separate file from the image, and the image and audio data captured by the CPU 203 may be stored together as one image data.

  After a predetermined time has elapsed, the CPU 203 stops the voice recording mode and advances the process to step S1806.

  Writing game information in step S1806, face detection in step S1807, and spine number detection in step S1808 are the same as in the first embodiment.

  In step S1809, the CPU 203 performs voice recognition. The CPU 203 transfers the audio data stored in the DRAM 206 to the feature detection circuit 200 via the data bus 202. The feature detection circuit 200 extracts characteristic information in the voice data by using a known voice recognition technique for the transferred voice data. As an example of a known voice recognition technique, there is a technique disclosed in JP-A-9-135417.

  The CPU 203 stores this characteristic information in the DRAM 206 as a characteristic value. When the voice recognition ends, the CPU 203 advances the process to step S1810.

  In step S1810, it is determined whether any of the number, the face, or the voice has been detected or recognized by the processes in steps S1807 to S1809. If it is determined that it has been detected or recognized, the CPU 203 reads records stored in the database 1601 in FIG. 17 one by one in ascending order of record numbers in step S1811.

  Thereafter, steps S1811 to S1812 perform the same processing as in the first embodiment. Hereinafter, the case where the image shown in FIG. 19 is imaged will be described in detail.

  From the image shown in FIG. 19, the spine number is detected and the voice pattern is recognized. The character pattern extracted from the detected spine number is converted into text data “20” by processing in the OCR unit 136.

  Since the spine number and the voice are detected, the feature value extracted therefrom is compared with the feature value in the database (steps S1810 to S1812).

  In the database 1601 of FIG. 17, the spine number pattern item “20” is registered. Therefore, the value “TOM” of the player name item is written in the header portion of the image (steps S1813 and S1814).

  Further, the detected feature value of the voice is not yet registered in the database 1601 of FIG. 17, that is, a new feature value. Therefore, this new feature value is registered in the database 1601 of FIG. 17 (steps S1815 and S1816). The contents of the database 1601 at this time are shown in FIG. The audio data is described with various parameters. Here, for convenience, the audio feature value is “YYY”.

  Next, a description will be given of processing in the case where a subject whose face feature value is not registered in the database 1601 but whose sound feature value is registered is photographed.

  The database before photographing is in the state shown in FIG. Hereinafter, a description will be given with reference to the flowchart of FIG.

  The image photographed in step S1804 is shown in FIG. Further, it is assumed that the sound “Tom” is input and stored as sound data when this image is taken (step S1805). The image shown in FIG. 21 has a composition in which the face can be confirmed but the back number cannot be seen.

  By the detection process, a face is detected from the image of FIG. 21, and a feature value is extracted. (Step S1807). A feature value “YYY” is extracted from the stored voice data, that is, the voice data “Tom”.

  (Step S1809) Then, the detected feature value is compared with the feature value in the database 1601 of FIG. 20 (step S1810 to step S1812).

  In the audio pattern item 1606 of the database 1601 in FIG. 20, the feature value “ZZZ” has already been registered. Therefore, the value “TOM” of the player name item is written in the header portion of the image (steps S1813 and S1814).

  Further, the feature value extracted from the detected face is a new feature value that is not yet registered in the database 1610. Therefore, the CPU 203 registers the feature value extracted from the detected face in the database (step S1815, step S1816), and returns the process to step S1802.

  The contents of the database at this time are shown in FIG. For the player name “TOM”, the spine number, face, and voice feature values 1604 to 1606 are registered. The feature value of the face is “ZZZ” for convenience. By using this database, the cameraman 106 can recognize who the subject is by photographing the subject with a composition including at least one of a spine number, a face, and voice. That is, the recognition accuracy is less affected by the composition.

  In this embodiment, the camera receives the voice of the cameraman 106 from the microphone 233, but the present invention is not limited to this embodiment. When the microphone 233 receives external sound, it is possible to recognize the player name called by the in-field announcement flowing from the speaker 104.

  For example, when an on-site announcement “No. 6, Tom” is called, the sound may be input to the microphone 233 and the CPU 203 may extract the feature value “YYY” from the sound data of “Tom”. In this case, the CPU 203 writes “TOM” in the header of the image captured after recognition. When the CPU 203 recognizes the announcement for calling the name of the next batter, it finishes writing “TOM” and writes the name of the next batter in the header of the image.

  In the case of such a configuration, the cameraman 106 does not need to input audio data by himself, so that it is possible to concentrate on photographing only.

(Other examples)
It goes without saying that the object of the present invention can also be achieved by using a recording medium on which a program code of software for realizing the functions of the above-described embodiments is recorded. In this case, the object of the present invention is achieved by the computer (or CPU or MPU) of the system or apparatus reading and executing the program code stored in the recording medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.

  As a storage medium for supplying the program code, for example, floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, nonvolatile memory card, ROM, DVD, etc. Can be used.

  Further, an OS (operation system) running on the computer may perform part or all of the actual processing based on the instruction of the program code read by the computer. It goes without saying that the case where the functions of the above-described embodiments are realized by such processing is included in the scope of the present invention.

  Furthermore, the program may be executed via a function expansion device such as a function expansion board inserted into the computer or a function expansion unit connected to the computer. In this case, the program code read from the storage medium is written to the memory provided in the function expansion device. Then, based on the instruction of the program code, the CPU provided in the function expansion board or function expansion unit performs part or all of the actual processing. It goes without saying that the case where the functions of the above-described embodiments are realized by such processing is also included in the scope of the present invention.

Show an overview of the shooting system Shows the configuration of the digital camera The data reception flow from the server is shown. The data structure of game information is shown. The data structure of basic information is shown. The data structure of progress information is shown. The data structure of player information is shown. The database in Example 1 is shown. 5 shows a shooting flow in the first embodiment. The database in Example 1 is shown. The picked-up image in Example 1 is shown. The state of detection in Example 1 is shown. The database in Example 1 is shown. The picked-up image in Example 1 is shown. The state of detection in Example 1 is shown. The database in Example 2 is shown. The database in Example 2 is shown. 5 shows a shooting flow in the second embodiment. The picked-up image in Example 2 is shown. The database in Example 2 is shown. The picked-up image in Example 2 is shown. The database in Example 2 is shown.

Claims (5)

Imaging means for photoelectrically converting a subject image formed by a photographing lens to obtain a photographed image;
A communication means for communicating with an external device through a communication line;
Subject information receiving means for receiving information on a subject that is updated at a predetermined timing and is periodically transmitted from the external device;
Extraction means for extracting the feature amount of the face area and the character area of the subject area from the captured image obtained by the imaging means;
Storage means for associating the information received by the subject information receiving means with the feature amount information of the face area extracted from the photographed image by the extraction means and the feature amount information of the character area in a storage medium When,
Imaging device Ru and a adding means for adding information received in the captured image by the object information reception unit. Character information detecting means for detecting a character pattern indicating information on the subject from the image obtained by the imaging means;
The imaging apparatus according to claim 1, wherein the extraction unit extracts a feature amount of the character region from the character pattern detected by the character information detection unit. Further comprising conversion means for converting the character pattern detected by the character information detection means into text data;
The imaging apparatus according to claim 2 , wherein the extraction unit uses the text data converted by the conversion unit as information on a feature amount of the character region . A voice receiving means for receiving voice;
The extracting means, the image pickup apparatus according to any one of claims 1 to 3, characterized in that extracts a feature quantity from the voice received by the voice receiving unit. Information received by the object information receiving means, the image pickup apparatus according to any one of claims 1 to 4, characterized in that the text data.

Images