JP2008085582A - System for controlling image, image taking apparatus, image control server and method for controlling image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system for controlling an image capable of simply generating information to be added to the image, an image taking apparatus, an image control server and a method for controlling the image. <P>SOLUTION: The system 1000 for controlling the image has a digital camera 1 and a personal computer 200 for communicating with the digital camera 1 through a USB cable 30 and achieving image control server for receiving the image from the digital camera 1 and controlling the image. In the system, the digital camera 1 transmits the image towards the personal computer 200 while transmitting a voice with matching information for the image. In the system, the personal computer 200 stores the received image while recognizing the received voice and converting it into a text, and generates information to be added to the image on the basis of the text. Prior to storage, the added information is mapped to the image associated with the voice used as the basis of generation of additional information. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image management system including an imaging device that forms an image by imaging a subject on an imaging device, and an image management server that receives and manages the image from the imaging device, the imaging device, and the image management server And an image management method in the image management server.

  With the progress of wireless communication technologies such as USB (Universal Serial Bus) standards, wireless communication technologies compliant with Bluetooth standards, radio communication compliant with Bluetooth standards, and infrared communication compliant with IrDA (Infrared Data Association), etc. It has become possible to transmit capacity data at high speed. As a result, for example, a large amount of image data obtained by photographing with a digital camera can be transferred to an image management server that manages images at high speed at an arbitrary timing, thereby limiting the storage capacity of the digital camera. The desired number of photographs can be taken.

  In general, image data obtained by shooting with a digital camera includes additional information such as date information and shooting information, and the image management server saves image data linked to the additional information. Has been done.

  For example, an image storage system that receives and stores image data obtained by photographing with a digital camera via a communication line, creates a directory based on additional information such as audio data converted into text data, and GPS A system for generating a file name based on (Global Positioning System) information and shooting date / time information and storing image data has been proposed (see Patent Document 1).

  In addition, the context of the image data is estimated from attribute information such as date and time, position information, exposure information, etc. attached to the image and user profile information, map information, seasonal event information, event information, etc. registered in advance, There has been proposed an image management system capable of classifying and searching image data based on an estimated result (see Patent Document 2).

  Furthermore, an apparatus and method for creating or updating information about a subject person, which is additional data of a captured image, based on personal information received from outside has been proposed (see Patent Document 3).

There has also been proposed a digital camera and method for identifying a person in an image and adding the person's name to the additional data of the image file (see Patent Document 4).
JP 2005-203994 A JP 2002-10178 A JP 2004-133536 A JP 2004-62868 A

  In recent years, with the progress of communication technology, a large amount of image data obtained by photographing with a photographing device such as a digital camera is transferred to an image management server at an arbitrary timing, and images are collectively managed by the image management server. This tendency is getting stronger. Here, the image management server performs batch management of images based on additional information added to the images. For this reason, it is important to efficiently and accurately generate additional information added to an image.

  However, in the technique proposed in Patent Document 1 described above, in order to generate additional information such as voice data converted into text data, the voice of the speaker needs to be converted on the photographing apparatus side. Here, the load in the speech recognition processing for converting the voice is large, and therefore, if the voice conversion is performed on the side of the photographing apparatus with limited resources (storage capacity, etc.), the burden on the photographing apparatus side is large. There's a problem.

  In the technique proposed in Patent Document 2, it is difficult for a camera user to input a command or additional information for inference to an inference engine for estimating a context, and there are a plurality of context candidates. There is a problem that the inference accuracy of the inference engine may decrease.

  Furthermore, in the technique proposed in Patent Document 3, in order to generate information related to the subject person, it is necessary to create personal information separately and cause the apparatus to receive it.

  In the technique proposed in Patent Document 4, since a person is identified inside the camera and information about the person is generated, it is necessary to use much of the memory capacity provided in the camera. However, it is difficult to sufficiently store image data.

  In view of the circumstances described above, an object of the present invention is to provide an image management system, a photographing apparatus, an image management server, and an image management method that can easily generate additional information to be added to an image.

An image management system of the present invention that achieves the above object includes a photographing device that forms an image by forming a subject on an image sensor, and an image management server that communicates with the photographing device and receives and manages the image from the photographing device. In an image management system comprising
The photographing device is
Audio acquisition means for acquiring audio;
Association means for associating the sound acquired by the sound acquisition means with an image;
The image is transmitted to the image management server, and the sound acquired by the sound acquisition unit is associated with the image associated with the image associated by the association unit. A transmission means for transmitting,
The image management server
Receiving means for receiving the image transmitted from the imaging apparatus and receiving the audio transmitted from the imaging apparatus;
Storage means for storing the image received by the receiving means;
Voice recognition means for recognizing the voice received by the receiving means and converting it into text;
Additional information generating means for generating additional information to be added to an image based on text obtained by recognizing voice by the voice recognition means;
The storage means stores the image and stores the additional information generated by the additional information generation means in association with the image associated with the sound that is the basis of the additional information generation. It is characterized by.

  In the image management system of the present invention, the imaging device transmits an image to the image management server and transmits a sound with information associated with the image, and the image management server receives the image. The image is stored, the received voice is recognized and converted into text, additional information added to the image is generated based on the text, and the additional information is used as the basis for generating the additional information. The image is stored in association with the image associated with. For this reason, the image management server performs voice recognition processing to generate additional information to be added to the image. Therefore, it is not necessary to perform voice recognition processing with the photographing apparatus, and it is possible to easily generate additional information to be added to the image while reducing the processing burden of the photographing apparatus.

Here, the imaging device in the image management system of the present invention includes an audio file creation unit that creates an audio file storing the audio acquired by the audio acquisition unit,
The transmission unit preferably transmits the audio file created by the audio file creation unit when transmitting the audio toward the image management server.

  When the image capturing device creates and transmits a sound file storing sound when transmitting sound to the image management server, the image management server recognizes the sound based on the sound file and adds it to the image. Since the additional information is generated, the additional information to be added to the image can be generated more easily by voice.

Further, the transmission means in the image management system of the present invention transmits the voice acquired by the voice acquisition means as it is when transmitting the voice to the image management server,
It is also a preferable aspect that the image management server includes an audio file creating unit that creates an audio file storing the audio received by the receiving unit.

  In this way, it is possible to further reduce the processing burden on the photographing apparatus side.

Furthermore, the imaging device in the image management system of the present invention,
Position information acquisition means for acquiring geographical position information of the photographing apparatus;
Date and time information acquisition means for acquiring date and time information;
Schedule information acquisition means for acquiring schedule information of the user of the photographing apparatus,
The transmission means further transmits the position information, date information, and schedule information respectively acquired by the position information acquisition means, the date information acquisition means, and the schedule information acquisition means to the image management server. Is,
The receiving means further receives position information, date / time information, and schedule information transmitted from the photographing apparatus,
The additional information generating means generates additional information based on the text obtained by recognizing the voice by the voice recognition means and also based on the position information, date / time information, and schedule information received by the receiving means. It is also preferable.

  In this way, it is possible to provide the image management server with the geographical position information, date / time information, and user schedule information of the photographing apparatus. Therefore, the image management server can easily add additional information added to the image by voice. While being able to generate | occur | produce, the positional information, date information, and schedule information added to the image can also be produced | generated easily.

The image management server in the image management system of the present invention further includes a map database for managing maps,
It is also a preferred aspect that the additional information generating means further generates additional information based on a map in the map database.

  In this way, the image management server can easily generate additional information to be added to the image by voice and can also easily generate map information to be added to the image.

Furthermore, the image management server in the image management system of the present invention comprises face recognition means for recognizing a face reflected on the image from the image received by the receiving means,
The additional information generation means may generate additional information based on the text on which the voice is recognized by the voice recognition means and also based on the face on the image recognized by the face recognition means. preferable.

  As described above, when the additional information is generated based on the text from which the voice is recognized and also based on the face on the image, the subject person information can be used as the additional information by the voice.

Further, the image management server in the image management system of the present invention further comprises a face information database for managing face information,
It is also a preferred aspect that the additional information generating means further generates additional information based on the face information in the face information database.

  As described above, when the additional information is generated based on the face information in the face information database managing the face information, the process for generating the additional information based on the face on the image can be quickly performed.

  Further, the face information database is generated by generating new face information on the basis of the text recognized by the voice recognition means and the face on the image recognized by the face recognition means in the image management system of the present invention. It is also preferable to include a face information adding means for adding to the face.

  As described above, when new face information is generated and added to the face information database based on the recognized text and the face on the recognized image in the image management server, the face of the subject person on the image is displayed. Information can be registered in the face information database.

  The additional information generation means in the image management system of the present invention includes command recognition means for recognizing a command related to additional information generation based on text obtained by recognizing voice by the voice recognition means. It is also a preferable aspect that the additional information is generated according to the recognized command.

  In this way, the user can give an instruction to the additional information generating means.

In addition, the imaging apparatus of the present invention that achieves the above object generates an image by imaging a subject on an imaging device, communicates with an image management server that receives and manages the image, and transmits the image to the image management server In the shooting device
Audio acquisition means for acquiring audio;
Association means for associating the sound acquired by the sound acquisition means with an image;
The image is transmitted to the image management server, and the sound acquired by the sound acquisition unit is associated with the image associated with the image associated by the association unit. Transmission means for transmitting.

  The imaging apparatus of the present invention transmits an image to an image management server and transmits a sound with information associated with the image. For this reason, the image management server performs voice recognition processing to generate additional information to be added to the image. Therefore, it is not necessary to perform voice recognition processing with the photographing apparatus, and the processing burden on the photographing apparatus is reduced. Further, the image management server can easily generate additional information added to the image by voice.

Here, in the photographing apparatus of the present invention, provided with a voice file creation means for creating a voice file storing the voice acquired by the voice acquisition means,
The transmission unit preferably transmits the audio file created by the audio file creation unit when transmitting the audio toward the image management server.

  In this way, since the image management server recognizes the sound based on the sound file and generates additional information added to the image, the additional information added to the image can be more easily generated by sound. it can.

  In addition, in a preferable aspect, the transmission unit in the photographing apparatus of the present invention transmits the voice acquired by the voice acquisition unit as it is when transmitting the voice to the image management server.

  In this way, it is possible to further reduce the processing load of the photographing apparatus.

Furthermore, in the imaging apparatus of the present invention, a position information acquisition unit that acquires geographical position information of the imaging apparatus, a date and time information acquisition unit that acquires date and time information, and a schedule that acquires schedule information of the user of the imaging apparatus Information acquisition means,
The transmission means further transmits the position information, date information, and schedule information respectively acquired by the position information acquisition means, the date information acquisition means, and the schedule information acquisition means to the image management server. It is also preferable.

  In this way, it is possible to provide the image management server with geographical location information, date / time information, and user schedule information of the imaging device, and the image management server includes the location information, date / time information, and schedule information. Additional information can be generated based on the above.

Furthermore, an image management server of the present invention that achieves the above object is an image management server that communicates with an imaging device that forms an image on an image sensor and generates an image, and receives and manages the image from the imaging device.
Receiving means for receiving the image transmitted from the imaging apparatus and receiving the audio transmitted from the imaging apparatus;
Storage means for storing the image received by the receiving means;
Voice recognition means for recognizing the voice received by the receiving means and converting it into text;
Additional information generating means for generating additional information to be added to an image based on text obtained by recognizing voice by the voice recognition means;
The storage means stores the image and stores the additional information generated by the additional information generation means in association with the image associated with the sound that is the basis of the additional information generation. It is characterized by.

  The image management server of the present invention stores the received image, recognizes the received sound and converts it into text, generates additional information to be added to the image based on the text, The image is stored in association with the image associated with the sound that is the basis for generating the additional information. For this reason, the image management server performs voice recognition processing to generate additional information to be added to the image. Therefore, it is not necessary to perform voice recognition processing with the photographing apparatus, and it is possible to easily generate additional information to be added to the image while reducing the processing burden of the photographing apparatus.

  Here, it is preferable that the receiving means in the image management server of the present invention receives an audio file created by the imaging device and transmitted from the imaging device when receiving audio.

  In this way, the image management server can recognize the sound based on the sound file and generate additional information to be added to the image. Therefore, the additional information to be added to the image can be more easily generated by sound. can do.

Further, the receiving means in the image management server of the present invention is for receiving the sound acquired by the photographing apparatus and transmitted as it is when receiving the sound,
It is also a preferable aspect that the image management server includes an audio file creating unit that creates an audio file storing the audio received by the receiving unit.

  In this way, it is possible to further reduce the processing burden on the photographing apparatus side.

Furthermore, the receiving means in the image management server of the present invention further receives position information, date / time information, and schedule information transmitted from the photographing apparatus,
The additional information generating means generates additional information based on the text obtained by recognizing the voice by the voice recognition means and also based on the position information, date / time information, and schedule information received by the receiving means. It is also preferable.

  In this way, it is possible to easily generate additional information to be added to an image by voice, and it is also possible to easily generate position information, date / time information, and schedule information to be added to the image.

The image management server of the present invention further comprises a map database for managing maps,
It is also a preferred aspect that the additional information generating means further generates additional information based on a map in the map database.

  In this way, the image management server can easily generate additional information to be added to the image by voice and can also easily generate map information to be added to the image.

The image management server of the present invention further comprises face recognition means for recognizing a face reflected on the image from the image received by the receiving means,
The additional information generation means may generate additional information based on the text on which the voice is recognized by the voice recognition means and also based on the face on the image recognized by the face recognition means. preferable.

  In this way, the subject person information can be used as additional information by voice.

Furthermore, the image management server of the present invention comprises a face information database for managing face information,
It is also a preferred aspect that the additional information generating means further generates additional information based on the face information in the face information database.

  In this way, it is possible to quickly perform processing for generating additional information based on the face on the image.

  Further, the face information database is generated by generating new face information on the basis of the text recognized by the voice recognition means and the face on the image recognized by the face recognition means in the image management server of the present invention. It is also preferable to include a face information adding means for adding to the face.

  In this way, the face information of the subject person on the image can be registered in the face information database.

  Further, the additional information generation means in the image management server of the present invention includes command recognition means for recognizing a command related to additional information generation based on text obtained by recognizing voice by the voice recognition means. It is also a preferable aspect that the additional information is generated according to the recognized command.

  In this way, the user can give an instruction to the additional information generating means.

Furthermore, the image management method of the present invention that achieves the above-described object provides an image management in an image management server that communicates with an imaging device that forms an image by forming an object on an image sensor and receives and manages the image from the imaging device. In the method
A receiving step of receiving an image transmitted from the imaging device and receiving an audio transmitted from the imaging device;
A storage step for storing the image received in the reception step;
A speech recognition step of recognizing the speech received in the reception step and converting it to text;
An additional information generation step for generating additional information to be added to the image based on the text in which the voice is recognized in the voice recognition step;
The storing step is a step of storing the image and storing the additional information generated in the additional information generating step in association with the image associated with the sound that is the basis of the additional information generation. It is characterized by.

  Since the image management method of the present invention is an image management method in the image management server of the present invention, additional information added to an image can be generated by performing voice recognition processing in the image management server. Therefore, it is not necessary to perform voice recognition processing with the photographing apparatus, and it is possible to easily generate additional information to be added to the image while reducing the processing burden of the photographing apparatus.

  Here, it is preferable that the reception step in the image management method of the present invention is a step of receiving an audio file created by the imaging apparatus and transmitted from the imaging apparatus when receiving audio.

  In such a step, the image management server recognizes the sound based on the sound file and generates additional information to be added to the image. Therefore, the additional information to be added to the image is more easily generated by sound. can do.

  Further, in the image management method of the present invention, the reception step is a step of receiving the voice acquired by the photographing apparatus and transmitted as it is when receiving the voice, and further stores the voice received in the reception step. It is also a preferable aspect to have an audio file creation step for creating an audio file.

  With such a step, the processing burden on the photographing apparatus side can be further reduced.

Furthermore, the receiving step in the image management method of the present invention further receives position information, date / time information, and schedule information transmitted from the photographing apparatus,
The additional information generation step is a step of generating additional information based on the text obtained by recognizing the speech in the speech recognition step and also based on the position information, date / time information, and schedule information received in the reception step. It is also preferable.

  In such a step, the image management server can easily generate additional information to be added to the image by voice, and can also easily generate position information, date / time information, and schedule information to be added to the image. be able to.

In the image management method of the present invention, the image management server includes a map database for managing a map,
It is also a preferred aspect that the additional information generating step is a step of generating additional information based on the map in the map database.

  In such a step, the image management server can easily generate additional information to be added to the image by voice, and can also easily generate map information to be added to the image.

Furthermore, it has a face recognition step for recognizing a face reflected on the image from the image received in the reception step in the image management method of the present invention,
The additional information generation step may be a step of generating additional information based on the text on which the voice is recognized in the voice recognition step and also on the face on the image recognized in the face recognition step. This is a preferred embodiment.

  In such a step, the subject person information can be used as additional information by voice.

In the image management method of the present invention, the image management server further includes a face information database for managing face information,
It is also preferable that the additional information generation step is a step of generating additional information based on face information in the face information database.

  With such a configuration, processing for generating additional information based on the face on the image can be quickly performed.

  Further, the face information database is generated by generating new face information based on the text in which the voice is recognized in the voice recognition step and the face on the image recognized in the face recognition step in the image management method of the present invention. It is also a preferable aspect to have a face information adding step to be added to.

  With such steps, the face information of the subject person on the image can be registered in the face information database.

  Further, the additional information generation step in the image management method of the present invention includes a command recognition step for recognizing a command related to additional information generation based on the text obtained by recognizing the voice in the voice recognition step. It is also preferable that the additional information be generated according to the command recognized in step (b).

  In such a step, the user can give an instruction to the additional information generating means.

  ADVANTAGE OF THE INVENTION According to this invention, the image management system, imaging device, image management server, and image management method which can produce | generate the additional information added to an image simply can be provided.

  Embodiments of the present invention will be described below.

  FIG. 1 is a diagram showing an image management system according to the first embodiment of the present invention.

  An image management system 1000 shown in FIG. 1 includes a digital camera 1 (corresponding to the first embodiment of the photographing apparatus of the present invention) that forms an image by imaging a subject on an image sensor, and communicates with the digital camera 1. A personal computer 200 for realizing the first embodiment of the image management server of the present invention by receiving and managing images from the digital camera 1 is provided.

  Although details of the digital camera 1 will be described later, the digital camera 1 includes a USB connection unit 170 to which the USB connector 31 of the USB cable 30 is connected.

  In addition, FIG. 1 includes a USB connection unit 214 connected to the USB connector 32 of the USB cable 30, receives connection of the digital camera 1 to the connection unit 214 by the USB connector 32, and further performs a predetermined operation. A personal computer 200 that captures image data from the digital camera 1 is shown. The personal computer 200 includes a main body 210, an image display device 220 including a display screen 220 a, a keyboard 230, and a mouse 240. Although the details will be described later, the main body 210 incorporates a CPU (central processing unit), a main memory (random access memory), a hard disk, a communication I / F, and the like. The main body 210 has a CD-ROM loading slot 211 into which a CD-ROM is loaded and a flexible disk loading slot 212 into which a flexible disk is loaded. A CD-ROM drive and a flexible disk drive for accessing a CD-ROM and a flexible disk loaded from 211 and 212 are incorporated. Further, the main body 210 has a power switch 213.

  2 is a top view of the digital camera shown in FIG. 1 as viewed from above (FIG. 2A), a front view as viewed from the front (FIG. 2B), and a rear view as viewed from the back (FIG. 2 (2)). c)).

  As shown in FIGS. 2A and 2B, the digital camera 1 is provided with a lens barrel 10 in the center of the camera body 1a. Further, a power button 11 is provided on the upper surface of the camera body 1a. When the power button 11 is operated, the lens barrel 10 is extended as shown in FIGS. 2 (a) and 2 (b). Ready to shoot.

  Further, on the upper surface of the camera body 1a, in addition to the power button 11 for turning on the power, a release button 12 and a shooting mode dial 12_1 are provided around the release button 12, and the shooting mode dial 12_1 is switched to AUTO. When the release button 12 is pressed while shooting is performed, shooting conditions are automatically set in the digital camera 1 and shooting is performed. When the shooting mode dial 12_1 is switched to the moving image shooting mode (symbol M), moving image shooting is performed. When the shooting mode dial 12_1 is switched to the scene position (symbol SP) side, shooting conditions corresponding to the shooting scene are digital. The camera 1 is automatically set to take a picture.

  As shown in FIG. 2C, an image display device 13 is provided on the back side of the camera body 1a. In the shooting mode, a subject is displayed on the image display device 13 or a menu is displayed. It is displayed. When the playback button 14 on the side of the image display device 13 is pressed once in this shooting mode, the mode is switched to the playback mode and the already shot image is displayed on the image display device 13, and when the playback button 14 is pressed again. Then, the shooting mode is switched and the through image is displayed on the image display device 13. In addition, an F (photo mode) button 15 is provided next to the playback mode button 14, and a frequently used mode such as a pixel setting mode or a sensitivity setting mode can be easily switched by operating the F button 15. You can also do it.

  A cross key 16 and an OK / menu button 17 are provided below the F button 15, and a zoom switch 18 is provided above the F button 15. By operating the cross key 16 or the OK / menu button 17, the setup menu is switched to set the date and time, whether or not to display images, and the shooting menu is switched to select continuous shooting, self-timer, etc. You can do it.

  Also, below the cross key 16, a DSP / BACK button 21 used for switching the display of the image display device 13 or stopping the operation halfway is provided.

  Further, a photometric sensor 190 and a flash light emission window 191 are provided in front of the camera body 1a shown in FIG. 2B. When flash light emission is necessary, the flash light is directed from the flash light emission window 191 toward the subject. Light is emitted. In addition, a microphone 150 for recording sound is disposed at the lower part of the camera body 1a shown in FIG. 2B, and the USB connection unit 170 described above is provided on the side surface of the camera body 1a. A speaker 180 is provided on the bottom surface of the camera body 1a shown in FIG.

  FIG. 3 is a diagram showing an internal configuration of the digital camera shown in FIG.

  The digital camera 1 performs zooming, exposure adjustment, and focusing by driving the zoom lens 10_1a, the iris 10_1b, and the focus lens 10_1c constituting the photographing optical system, and the zoom lens 10_1a, the iris 10_1b, and the focus lens 10_1c. Motor drivers 111, 112, and 113 are provided.

  Further, the digital camera 1 includes a CCD 114, a timing generator 115, and a CPU 116.

  The CCD 114 is a solid-state imaging device that captures subject light that has passed through the zoom lens 10_1a, the iris 10_1b, and the focus lens 10_1c. The CCD 114 is provided with a large number of photoelectric conversion elements such as photodiodes that convert incident subject light into image signals that are electrical signals.

  The timing generator 115 drives the CCD 114 at a predetermined timing according to an instruction from the CPU 116. As a result, the subject light incident on the CCD 114 is photoelectrically converted at a predetermined frame rate and output from the CCD 114 as an analog image signal.

  The CPU 116 controls the entire digital camera 1. Specifically, the CPU 116 has a built-in ROM, and the operation of the entire digital camera 1 is controlled in accordance with the program procedure of the built-in ROM.

  Further, the digital camera 1 includes a CDSAMP 117 that performs processing for reducing noise of the analog image signal output from the CCD 114, and A / A that performs analog / digital conversion of the analog image signal subjected to the processing to a digital image signal. A D conversion unit 118 and an image input controller 119 for transferring image data composed of RGB converted into a digital image signal by the A / D conversion unit 118 to the SDRAM 125 as a memory via a data bus are provided.

  In addition, the digital camera 1 reads the image data of the SDRAM 125 and executes an image processing process 120 for executing an image processing process for converting the data into a YC signal, and executes the JPEG process when the image processing process is completed. A compression processing circuit 121 for compressing the image data, and a video encoder 122 for converting the image data into a video signal and leading it to the image display device 13 described above.

  Further, the digital camera 1 is used as an AF detection circuit 123 that detects image focus information, an AE & AWB detection unit 124 that detects image brightness information and white balance information, and a work area memory. SDRAM 125 in which audio files and schedule information files to be stored are stored, VRAM 126 as a display buffer having an A area and a B area, and a media controller 127 that performs control for storing image data in recording medium 100. It has been.

  3 shows the power button 11, release button 12, shooting mode dial 12_1, playback button 14, F button 15, cross key 16, OK / menu button 17, zoom switch 18, DSP / BACK button 21. An operation unit 10_100 consisting of is shown.

  Furthermore, the digital camera 1 includes a microphone 150 and an A / D conversion circuit 151 that converts an analog audio signal from the microphone 150 into a digital audio signal. The microphone 150 and the A / D conversion circuit 151 correspond to an example of a sound acquisition unit that acquires sound according to the present invention.

  In addition, the digital camera 1 is provided with an association unit 152 that associates sound acquired by the microphone 150 and the A / D conversion circuit 151 with an image.

  Further, the digital camera 1 transmits an image to the personal computer 200 shown in FIG. 1 and associates the sound acquired by the microphone 150 and the A / D conversion circuit 151 with the personal computer 200. A communication I / F 153 (corresponding to an example of a transmission means in the present invention) that transmits the information associated with the image associated by the means 152 is provided. The communication I / F 153 includes the USB connection unit 170 described above.

  In addition, the digital camera 1 is provided with an audio file creation means 160 that creates an audio file storing the audio acquired by the microphone 150 and the A / D conversion circuit 151. Here, the communication I / F 153 transmits the voice file created by the voice file creation unit 160 when sending the voice to the personal computer 200.

  Further, the digital camera 1 is provided with GPS position information acquisition means 154 (corresponding to an example of the position information acquisition means referred to in the present invention) for acquiring geographical position information of the digital camera 1. Specifically, the GPS position information acquisition unit 154 receives a GPS signal from a GPS (Global Positioning System) satellite and acquires GPS position (current position) data.

  Further, the digital camera 1 includes date / time information acquisition means 155 for acquiring date / time information and schedule information acquisition means 156 for acquiring schedule information of the user of the digital camera 1. Here, the communication I / F 153 further sends the position information, date information, and schedule acquired by the GPS position information acquisition unit 154, the date information acquisition unit 155, and the schedule information acquisition unit 156 toward the personal computer 200, respectively. Send information.

  Further, the digital camera 1 is provided with an audio reproduction circuit 157 that generates an audio signal for reproducing audio, and a speaker 180 that receives the audio signal from the audio reproduction circuit 157 and generates audio.

  Here, the operation of the digital camera 1 will be briefly described. The operation of the digital camera 1 is comprehensively controlled by the CPU 116. The CPU 116 has a built-in ROM, and a program is stored in the built-in ROM. The operation of the entire digital camera 1 is controlled by the CPU 116 according to the procedure of this program. Further, the SDRAM 125 temporarily stores image data being processed when the CPU 116 controls the entire digital camera 1 in accordance with the procedure of the program described in the ROM.

  The VRAM 126 serving as a display buffer stores image data representing a through image in order in the A area and B area, and the image data in these areas are alternately supplied to the video encoder 122 to generate an image based on the image data. The images are sequentially switched and displayed on the image display device 13 as a through image.

  Next, the flow of image data of the digital camera 1 will be briefly described. The CPU 116 captures the subject light passing through the photographing optical system by the CCD 114, and outputs the image data generated by the CCD 114 to the CDSAMP 117 side at predetermined intervals, and the A / D conversion unit 118, the image input controller 119, image signal processing. The image data is converted into image data that can be displayed by the circuit 120 and the through image is displayed on the screen of the image display device 13. Framing is arbitrarily performed based on the through image, and the AF detection circuit 123 continuously moves the focus lens 10_1c back and forth to perform in-focus detection to form an image focused on the center of the framing on the CCD 114. At this time, the AE & AWB detection circuit 124 detects the field luminance, and based on the detection result, the motor driver 112 sets a small aperture or an open aperture, and each color signal of R, G, B is used for white balance adjustment. The gain is adjusted. In this way, image data representing a focused image whose exposure is adjusted according to the field luminance is output to the CDSAMP 117 at predetermined intervals according to the timing signal from the timing generator 115, and a through image is displayed at a later stage. It has gained. The photographer performs framing while viewing the through image, and performs a release operation at a photo opportunity.

  When the release operation is performed, the CPU 116 supplies a timing signal from the timing generator 115 to form an image at the time of the release operation on the CCD 114. This timing signal informs the CCD 114 of the start and end of exposure and corresponds to a so-called shutter speed. At the end of the exposure, the CPU 116 outputs image data (image data composed of the three primary colors R, G, and B of RGB light) from the CCD 114 and supplies it to the subsequent CDSAMP 117. Further, the noise of the image data output from the CCD 114 is reduced by the CDSAMP 117, and the image data with the reduced noise is converted into a digital signal by the A / D conversion unit 118. The RGB image data converted into digital signals by the A / D conversion unit 118 is supplied to the SDRAM 125 via the data bus by the subsequent image input controller 119, and the image data of all the pixels of the CCD 114 is stored in the SDRAM 125. Remembered. When the image data corresponding to all the pixel data is stored in the SDRAM 125, the CPU 116 activates the image processing process of the image signal processing circuit 120, reads the image data of the SDRAM 125 by the image processing process, and outputs the image data to the YC signal. Let the conversion take place. Further, when the CPU 116 detects that the image processing process of the image signal processing circuit 120 is completed, the CPU 116 activates the JPEG process of the compression processing circuit 121 and causes the image data to be compressed by the JPEG process. When the CPU 116 detects that the compression processing circuit 121 has completed the compression, the CPU 116 next activates a recording processing process to record the JPEG-compressed image data on the recording medium 100 via the media controller 127. The above is the flow of shooting data.

  Next, the personal computer 200 shown in FIG. 1 will be described.

  FIG. 4 is a diagram showing a schematic circuit block of the personal computer shown in FIG.

  In the personal computer 200 shown in FIG. 4, the CPU 201 that executes various programs and the program stored in the hard disk device 203 constituting the main body 210 shown in FIG. 1 are read and expanded for execution by the CPU 201. A main memory 202, a hard disk device 203 in which various programs, image data, and the like are stored, a flexible disk 204_1 are loaded, a flexible disk drive 204 that accesses the loaded flexible disk 204_1, and a CD-ROM drive that accesses the CD-ROM 205_1 205, a communication I / F 206 connected to the digital camera 1 and having the USB connection unit 214 shown in FIG. 1, and an image display device 220 including a display screen 220a shown in FIG. 1, a keyboard 230, and a mouse 240 It is provided. These are connected to each other via a bus 215.

  FIG. 5 is a diagram showing a configuration of an image management server realized by the personal computer shown in FIG.

  The image management server 300 shown in FIG. 5 receives the image transmitted from the digital camera 1 and also receives the sound transmitted from the digital camera 1. The communication I / F 206 shown in FIG. An example of receiving means) is provided.

  The image management server 300 includes a storage unit 302 that stores an image received by the communication I / F 206 in the image database 301, and a voice recognition unit 303 that recognizes the voice received by the communication I / F 206 and converts it into text. And additional information generating means 304 for generating additional information (metadata) to be added to the image based on the text recognized by the voice recognition means 303. Here, the storage unit 302 stores an image such as an Exif file or a JPEG file in the image database 301, and corresponds to the additional information generated by the additional information generation unit 304 to the sound that is the basis of the additional information generation. The image is stored in the image database 301 in association with the attached image.

  Further, the communication I / F 206 receives position information, date / time information, and schedule information transmitted from the digital camera 1, and the additional information generation unit 304 recognizes the voice by the voice recognition unit 303. The additional information is generated based on the position information, date / time information, and schedule information received by the communication I / F 206.

  Further, the image management server 300 is provided with a map database 305 for managing a map, and the additional information generating unit 304 further generates additional information based on the map in the map database 305. Here, information representing a map managed by the map database 305 includes terrain information, place name, facility name, latitude, longitude, and the like.

  Further, the image management server 300 is provided with a face recognition unit 306 that recognizes a face captured on the image from the image received by the communication I / F 206. Here, the additional information generation unit 304 generates additional information based on the text recognized by the voice recognition unit 303 and also based on the face on the image recognized by the face recognition unit 306.

  Further, the image management server 300 is provided with a face information database 307 for managing face information, and the additional information generating unit 304 further generates additional information based on the face information in the face information database 307.

  The image management server 300 generates new face information based on the text recognized by the voice recognition unit 303 and the face on the image recognized by the face recognition unit 306 to generate a face information database 307. Is provided with face information adding means 308 to be added.

  Here, the additional information generating unit 304 includes a command recognizing unit 304a that recognizes a command related to additional information generation based on text obtained by recognizing the speech by the speech recognizing unit 303, and according to the command recognized by the command recognizing unit 304a. Generate additional information.

  Further, the communication I / F 206 receives an audio file created by the digital camera 1 and transmitted from the digital camera 1 when receiving the audio.

  Furthermore, the image management server 300 also includes an additional information analysis unit 309 that analyzes additional information (including GPS position information) of, for example, an Exif file.

  FIG. 6 is a diagram showing a flow of a context estimation algorithm in the additional information generating means shown in FIG.

  First, in step S1, an additional information analysis result is acquired. Specifically, the shooting date and time (described as date A) and the latitude and longitude information of the shooting location are acquired from the Exif information in the image file. Next, in step S2, it is determined whether there is a schedule information file associated with the image. If it is determined that there is no schedule information file, the process proceeds to step S7 described later. On the other hand, if it is determined that there is a schedule information file, the process proceeds to step S3.

  In step S3, schedule information (date and time, place name, facility name, each list, etc.) is acquired from the schedule information file. Further, in step S4, the schedule corresponding to the date and time A is searched from the schedule information.

  Next, in step S5, it is determined whether there is a schedule corresponding to the date A. If it is determined that there is no schedule corresponding to the date and time A, the process proceeds to step S7. On the other hand, if it is determined that there is a schedule corresponding to the date and time A, the place name and facility name in the corresponding schedule are extracted as the keyword A in step S6.

  In step S7, the place name and facility name recognized by the voice recognition means are acquired as the keyword B.

  In step S8, keywords A and B are searched from the map database as search terms. The search range is near the latitude and longitude of the shooting location, and the specific range is arbitrarily set by the system (for example, 500 m). This is the context estimated by the search result.

  Next, in step S9, the context and keywords A and B are converted into metadata (additional information) and output, and this flow ends.

  FIG. 7 is a diagram showing a storage structure of an image file and an audio file created by the digital camera.

  FIG. 7 shows a main folder 400 (folder name DCIM) representing the entire contents of the recording medium 100 of the digital camera 1. The main folder 400 stores an image data folder 410 (folder name 100_ABC). The image data folder 410 stores a JPEG image file 411 (folder name DSCF0007.JPG), an audio file 412 (folder name DSCF0007.WAV),..., An audio file 41n (folder name DSCF000n.WAV).

  FIG. 8 is a diagram showing an example of a schedule information file stored in the memory (SDRAM) of the digital camera.

  In order to store the schedule information file in the memory (SDRAM) 125 of the digital camera 1, the schedule information file is created by, for example, starting schedule software on the personal computer 200 to create or edit the schedule information. The data is stored in the SDRAM 125 of the digital camera 1 via the communication I / Fs 206 and 153.

  FIG. 8 shows a schedule information file 500 stored in the SDRAM 125 of the digital camera 1. The schedule information file 500 includes a title portion 510 that stores information indicating the title name of the entire schedule, a first schedule portion 520 that stores information indicating information such as the type and schedule of the first schedule, and the like. The second schedule unit 530 stores information indicating information such as the type and schedule of the second schedule.

  FIG. 9 is a diagram showing a flow of an audio acquisition processing routine for acquiring audio with a digital camera.

  The voice acquisition processing routine is started by using the release button 12 of the digital camera 1 as a trigger to complete shooting. Here, an example in which the voice acquisition processing routine is started by using the release button 12 of the digital camera 1 as a trigger and shooting is completed will be described. However, the digital camera 1 includes a recording start button, The user may explicitly press the recording start button, or may select recording start → OK from the menu.

  First, in step S11, recording for acquiring voice is started. Next, in step S12, it is determined whether or not it is an automatic start for automatically starting recording. Here, it is determined that the start is automatic, and the process proceeds to step S14. For example, when the user selects start of recording from the menu, it is determined that the automatic start is not performed, and the process proceeds to step S13. In step S13, a recording target is selected. Here, the operation in step S13 will be described with reference to FIG.

  FIG. 10 is a diagram showing an image file structure for selecting a recording target in step S13 displayed on the image display device of the digital camera.

  On the image display device 13 shown in FIG. 10, a main folder 400 (folder name DCIM) representing the contents of the entire recording medium 100 is shown. The main folder 400 stores an image data folder 410 (folder name 100_ABC) and an image data folder 420 (folder name 101_ABC). The image data folder 410 stores a JPEG image file 410_1 (folder name DSCF0001.JPG), a JPEG image file 410_2 (folder name DSCF0002.JPG),.

  Here, in selecting the recording target, if the main folder 400 is selected by operating the cross key 16 or the like, an audio file is created by associating audio with all images in the camera in common. The created audio file is placed in the main folder 400.

  When the image data folder 410 is selected, an audio file is created by associating audio with all images in the image data folder 410 in common. The created audio file is stored in the image data folder 410, for example, folder name 410_ABC. Arranged as WAV.

  Furthermore, when an individual image is selected, a sound file is created by associating sound with only the image. The created audio file is arranged, for example, as shown in FIG. When selecting individual images, thumbnail images may be selected as shown in FIG.

  FIG. 11 is a diagram showing thumbnail images displayed on the image display device of the digital camera.

  Nine thumbnail images 431,..., 439 are displayed on the image display device 13 shown in FIG. Here, a state in which the thumbnail image 436 among the nine thumbnail images 431,..., 439 is selected is shown. In this manner, a plurality of thumbnail images may be displayed and individual images may be selected from these thumbnail images.

  Returning again to FIG. 9, the description will be continued. In step S14, recording is performed. When recording is automatically started, recording starts immediately. On the other hand, when the recording target is selected in step S13, recording is started by pressing a recording start button, for example.

  Further, in step S15, the recording is finished, and this flow is finished. Note that the end of recording may be ended automatically by a timer or may be ended by pressing the recording start button again.

  FIG. 12 is a diagram showing schematic steps of the image management method in the image management server shown in FIG.

  First, in the reception step S21, the image transmitted from the digital camera 1 is received and the sound transmitted from the digital camera 1 is received. Here, this reception step S21 receives an audio file created by the digital camera 1 and transmitted from the digital camera 1 when receiving the audio. In the reception step S21, the position information, date / time information, and schedule information transmitted from the digital camera 11 are further received.

  Next, in the face recognition step S22, the face reflected on the image is recognized from the image received in the reception step S21.

  Further, in the first storage step S23, the image received in the reception step S21 is stored.

  Next, in the voice recognition step S24, the voice received in the reception step S21 is recognized and converted into text.

  Further, in the additional information generation step S25, additional information to be added to the image is generated based on the text from which the voice is recognized in the voice recognition step S24. Specifically, in this additional information generation step S25, based on the text from which the speech is recognized in the speech recognition step S24, the additional information is also obtained based on the position information, date information, and schedule information received in the reception step S21. Generate. Furthermore, additional information is generated based on the map in the map database.

  Further, in this additional information generation step S25, additional information is generated based on the text recognized in the voice recognition step S24 and also on the face on the image recognized in the face recognition step S22. Also, additional information is generated based on the face information in the face information database. Further, a face information adding step for generating new face information and adding it to the face information database based on the text whose voice is recognized in the voice recognition step S24 and the face on the image recognized in the face recognition step S22. Execute.

  In addition, the additional information generation step S25 includes a command recognition step for recognizing a command related to additional information generation based on the text in which the voice is recognized in the voice recognition step S24, and is added according to the command recognized in the command recognition step. Generate information.

  Next, in the second storage step S26, the image is stored, and the additional information generated in the additional information generation step S25 is associated with the image associated with the sound that is the basis of the additional information generation. Save and end this flow.

  FIG. 13 is a diagram showing a routine flow until the additional information is generated and registered based on the text in which the voice is recognized and also on the face on the image in the image management server shown in FIG.

  For example, when an audio file (WAV) and an image file (JPG) created by the digital camera 1 shown in FIG. 7 are targeted, the processing from step S31 to step S34 is performed on the audio file (WAV). The process from step S35 to step S37 is performed on the image file (JPG).

  First, in step S31, audio information related to an audio file (WAV) is selected. Next, in step S32, the voice is recognized. Further, in step S33, a command related to additional information generation is detected based on the text from which the voice is recognized, and the command is analyzed in step S34. These steps S33 and S34 correspond to the command recognition step in the present invention. In step S38, metadata (additional information) is generated according to this command.

  Furthermore, face analysis is performed in steps S35, S36, and S37. First, in step S35, a face is extracted from the image file (JPG). Next, in step S36, the extracted face is recognized, and the process proceeds to step S37. In step S37, the recognized face data is collated. In step S38, metadata (additional information) according to the face data is generated. Further, in step S39, the metadata generated in step S38 is added. In step S40 (corresponding to the face information adding step in the present invention), new face information is generated and additionally registered in the face information database. End the flow.

  FIG. 14 is a diagram showing a routine flow in the case where the image management server shown in FIG. 5 generates additional information based on text obtained by recognizing voice and according to schedule information.

  Steps S31 to S34 shown in FIG. 14 are the same as steps S31 to S34 shown in FIG. Here, schedule information analysis is performed in steps S51 and S52. First, in step S51, schedule information related to the schedule information file (ICS) is read. Next, in step S52, schedule information is extracted. Further, the process proceeds to step S53.

  In step S53, metadata (additional information) according to the schedule information is generated. Further, in step S54, the metadata generated in step S53 is added to the image, and this flow ends.

  FIG. 15 is a diagram showing an internal configuration of a digital camera constituting the image management system of the second embodiment of the present invention, and FIG. 16 is a configuration of an image management server constituting the image management system of the second embodiment of the present invention. FIG.

  A digital camera 2 shown in FIG. 15 corresponds to the second embodiment of the photographing apparatus of the present invention, and this digital camera 2 is compared with the digital camera 1 shown in FIG. And the point that the communication I / F 153 transmits the sound acquired by the microphone 150 and the A / D conversion circuit 151 as it is when the communication I / F 153 transmits the sound to the image management server 600 shown in FIG. Are different.

  On the other hand, the image management server 600 shown in FIG. 16 corresponds to the second embodiment of the image management server of the present invention, and this image management server 600 is compared with the image management server 300 shown in FIG. In addition, an audio file creation means 601 for creating an audio file storing the audio received by the communication I / F 206 is added, and the communication I / F 206 is acquired by the digital camera 2 and transmitted as it is when receiving the audio. The difference is that the received voice is received.

  FIG. 17 is a diagram showing the steps of the image management method in the image management server shown in FIG.

  The steps of the image management method shown in FIG. 17 are different from the steps of the image management method shown in FIG. 12 described above in that the role of the reception step S21 is different, and the audio file creation step S61 is added. Are different.

  In the reception step S21 shown in FIG. 17, when receiving the voice, the voice acquired by the digital camera 2 and transmitted as it is is received. In the sound file creation step S61, a sound file storing the sound received in the reception step 21 is created. For this reason, the audio processing burden on the digital camera 2 side can be further reduced.

  In the present embodiment, an example of wired communication in which a digital camera and a personal computer are connected by a USB cable has been described. However, the present invention is not limited to this, and for example, a digital camera or a mobile phone having the functions of the digital camera. The photographing device such as a telephone and the image management server such as a personal computer may be performed by wireless communication such as radio wave communication conforming to the Bluetooth standard or infrared communication conforming to IrDA, and is not limited thereto. The invention can be applied to a communication between an imaging device that forms an image by imaging a subject on an image sensor and an image management server that receives and manages the image from the imaging device.

It is a figure which shows the image management system of 1st Embodiment of this invention. It is the top view which looked at the digital camera shown in FIG. 1 from upper direction, the front view seen from the front, and the back view seen from the back. It is a figure which shows the internal structure of the digital camera shown in FIG. It is a figure which shows the schematic circuit block of the personal computer shown in FIG. It is a figure which shows the structure of the image management server implement | achieved by the personal computer shown in FIG. It is a figure which shows the flow of the algorithm of the context estimation in the additional information production | generation means shown in FIG. It is a figure which shows the storage structure of the image file and audio | voice file which were produced with the digital camera. It is a figure which shows an example of the schedule information file stored in the memory (SDRAM) of the digital camera. It is a figure which shows the flow of the audio | voice acquisition process routine which acquires an audio | voice with a digital camera. It is a figure which shows the image file structure for selecting the recording object displayed on the image display apparatus of a digital camera in step S13. It is a figure which shows the thumbnail image displayed on the image display apparatus of a digital camera. It is a figure which shows the general | schematic step of the image management method in the image management server shown in FIG. FIG. 6 is a diagram showing a flow of a routine in the image management server shown in FIG. 5 until the additional information is generated and registered based on the text in which the voice is recognized and also on the face on the image. FIG. 6 is a diagram showing a routine flow in the case where the image management server shown in FIG. 5 generates additional information based on text in which voice is recognized and according to schedule information. It is a figure which shows the internal structure of the digital camera which comprises the image management system of 2nd Embodiment of this invention. It is a figure which shows the structure of the image management server which comprises the image management system of 2nd Embodiment of this invention. It is a figure which shows the step of the image management method in the image management server shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1, 2 Digital camera 1a Camera body 10 Lens barrel 10_1a Zoom lens 10_1b Iris 10_1c Focus lens 10_100 Control group 11 Power button 12 Release button 12_1 Shooting mode dial 13 Image display device 14 Play button 15 F button 16 Cross key 17 OK / Menu button 18 Zoom switch 21 DSP / BACK button 30 USB cable 31, 32 USB connector 100 Recording media 111, 112, 113 Motor driver 114 CCD
115 Timing generator 116, 201 CPU
117 CDSAMP
118 A / D Converter 119 Image Input Controller 120 Image Signal Processing Circuit 121 Compression Processing Circuit 122 Video Encoder 123 AF Detection Circuit 124 AE & AWB Detection Circuit 125 Memory (SDRAM)
126 VRAM
127 Media controller 150 Microphone 151 A / D conversion circuit 152 Corresponding means 153,206 Communication I / F
154 GPS position information acquisition means 155 Date and time information acquisition means 156 Schedule information acquisition means 157 Audio reproduction circuit 160, 601 Audio file creation means 170, 214 USB connection unit 180 Speaker 190 Photometric sensor 191 Flash emission window 200 Personal computer 202 Main memory 203 Hard disk Device 204 Flexible disk drive 204_1 Flexible disk 205 CD-ROM drive 205_1 CD-ROM
210 Main body 211 CD-ROM loading slot 212 Flexible disk loading slot 213 Power switch 215 Bus 220 Image display device 220a Display screen 230 Keyboard 240 Mouse 240 300 Mouse image management server 301 Image database 302 Storage means 303 Voice recognition means 304 Additional information generation Means 304a Command recognition means 305 Map database 306 Face recognition means 307 Face information database 308 Face information addition means 309 Additional information analysis section 400 Main folder 410, 420 Image data folder 410_1, 410_2, 411 Image file 412, ..., 41n Audio file 431, ..., 439 Thumbnail image 500 Schedule information file 510 Title part 520,530 Schedule part 1000 pictures Control system

Claims (31)

In an image management system comprising: an imaging device that forms an image by imaging a subject on an imaging device; and an image management server that communicates with the imaging device and receives and manages images from the imaging device.
The imaging device is
Audio acquisition means for acquiring audio;
Association means for associating the sound acquired by the sound acquisition means with an image;
The image is transmitted to the image management server, and the audio acquired by the audio acquisition unit is associated with the image associated with the image associated by the association unit toward the image management server. A transmission means for transmitting,
The image management server
Receiving means for receiving the image transmitted from the imaging apparatus and receiving the audio transmitted from the imaging apparatus;
Storage means for storing images received by the receiving means;
Voice recognition means for recognizing the voice received by the receiving means and converting it into text;
Additional information generating means for generating additional information to be added to an image based on text obtained by recognizing voice by the voice recognition means;
The storage means stores the image and stores the additional information generated by the additional information generation means in association with the image associated with the sound that is the basis of the additional information generation. An image management system characterized by The photographing apparatus includes a sound file creating unit that creates a sound file storing the sound obtained by the sound obtaining unit,
2. The image management system according to claim 1, wherein the transmission unit transmits the audio file created by the audio file creation unit when transmitting the audio toward the image management server. The transmission unit transmits the audio acquired by the audio acquisition unit as it is when transmitting the audio toward the image management server,
2. The image management system according to claim 1, wherein the image management server includes an audio file creating unit that creates an audio file storing the audio received by the receiving unit. The imaging device is
Position information acquisition means for acquiring geographical position information of the photographing apparatus;
Date and time information acquisition means for acquiring date and time information;
Schedule information acquisition means for acquiring schedule information of the user of the photographing apparatus,
The transmission means further transmits the position information, date / time information, and schedule information respectively acquired by the position information acquisition means, the date / time information acquisition means, and the schedule information acquisition means to the image management server. Is,
The receiving means further receives position information, date / time information, and schedule information transmitted from the photographing apparatus,
The additional information generating means generates additional information based on text obtained by recognizing speech by the speech recognizing means and also based on position information, date / time information, and schedule information received by the receiving means. The image management system according to claim 1. The image management server further comprises a map database for managing maps,
5. The image management system according to claim 4, wherein the additional information generating means further generates additional information based on a map in the map database. The image management server comprises face recognition means for recognizing a face reflected on the image from the image received by the receiving means;
The additional information generation means generates the additional information based on the text on which the voice is recognized by the voice recognition means and also based on the face on the image recognized by the face recognition means. The image management system according to claim 1, wherein: The image management server further comprises a face information database for managing face information,
7. The image management system according to claim 6, wherein the additional information generating means further generates additional information based on face information in the face information database.   Face information adding means for generating new face information based on text recognized by the voice recognition means and a face on the image recognized by the face recognition means and adding the face information to the face information database; The image management system according to claim 7, wherein:   The additional information generation means includes command recognition means for recognizing a command related to additional information generation based on text obtained by recognizing speech by the voice recognition means, and generates additional information according to the command recognized by the command recognition means. The image management system according to claim 1, wherein the image management system is an image management system. In an imaging device that forms an image by imaging a subject on an image sensor, communicates with an image management server that receives and manages the image, and transmits the image to the image management server.
Audio acquisition means for acquiring audio;
Association means for associating the sound acquired by the sound acquisition means with an image;
The image is transmitted to the image management server, and the audio acquired by the audio acquisition unit is associated with the image associated with the image associated by the association unit toward the image management server. An imaging apparatus comprising: a transmission means for transmitting. Voice file creation means for creating a voice file storing the voice acquired by the voice acquisition means;
11. The photographing apparatus according to claim 10, wherein the transmission unit transmits the audio file created by the audio file creation unit when transmitting the audio toward the image management server.   The photographing apparatus according to claim 10, wherein the transmission unit transmits the audio acquired by the audio acquisition unit as it is when transmitting the audio toward the image management server. Position information acquisition means for acquiring geographical position information of the photographing apparatus;
Date and time information acquisition means for acquiring date and time information;
Schedule information acquisition means for acquiring schedule information of the user of the photographing apparatus,
The transmission means further transmits the position information, date / time information, and schedule information respectively acquired by the position information acquisition means, the date / time information acquisition means, and the schedule information acquisition means to the image management server. The photographing apparatus according to claim 10, wherein the photographing apparatus is a thing. In an image management server that communicates with an imaging device that forms an image of a subject on an image sensor and generates an image, and receives and manages the image from the imaging device.
Receiving means for receiving the image transmitted from the imaging apparatus and receiving the audio transmitted from the imaging apparatus;
Storage means for storing images received by the receiving means;
Voice recognition means for recognizing the voice received by the receiving means and converting it into text;
Additional information generating means for generating additional information to be added to an image based on text obtained by recognizing voice by the voice recognition means;
The storage means stores the image and stores the additional information generated by the additional information generation means in association with the image associated with the sound that is the basis of the additional information generation. An image management server characterized by the above.   15. The image management server according to claim 14, wherein the receiving unit is configured to receive an audio file created by the imaging apparatus and transmitted from the imaging apparatus when receiving audio. The receiving means is for receiving sound that is acquired by the photographing apparatus and transmitted as it is when receiving sound;
The image management server according to claim 14, wherein the image management server includes an audio file creating unit that creates an audio file storing the audio received by the receiving unit. The receiving means further receives position information, date / time information, and schedule information transmitted from the photographing apparatus,
The additional information generating means generates additional information based on text obtained by recognizing speech by the speech recognizing means and also based on position information, date / time information, and schedule information received by the receiving means. The image management server according to claim 14. In addition, it has a map database to manage the map,
15. The image management server according to claim 14, wherein the additional information generating means further generates additional information based on a map in the map database. A face recognition means for recognizing a face reflected on the image from the image received by the receiving means;
The additional information generation means generates the additional information based on the text on which the voice is recognized by the voice recognition means and also based on the face on the image recognized by the face recognition means. The image management server according to claim 14, wherein: It has a face information database that manages face information,
20. The image management server according to claim 19, wherein the additional information generating means further generates additional information based on face information in the face information database.   Face information adding means for generating new face information based on text recognized by the voice recognition means and a face on the image recognized by the face recognition means and adding the face information to the face information database; The image management server according to claim 20, wherein the image management server is an image management server.   The additional information generation means includes command recognition means for recognizing a command related to additional information generation based on text obtained by recognizing speech by the voice recognition means, and generates additional information according to the command recognized by the command recognition means. 15. The image management server according to claim 14, wherein the image management server is a server. In an image management method in an image management server that communicates with an imaging device that forms an image by imaging a subject on an image sensor and receives and manages the image from the imaging device.
A receiving step of receiving an image transmitted from the photographing apparatus and receiving a sound transmitted from the photographing apparatus;
A storage step for storing the image received in the reception step;
A voice recognition step of recognizing the voice received in the reception step and converting it into text;
An additional information generation step for generating additional information to be added to the image based on the text in which the voice is recognized in the voice recognition step;
The storing step is a step of storing the image and storing the additional information generated in the additional information generating step in association with the image associated with the sound that is the basis of the additional information generation. An image management method characterized by the above.   24. The image management method according to claim 23, wherein the receiving step is a step of receiving an audio file created by the imaging apparatus and transmitted from the imaging apparatus when receiving audio.   The receiving step is a step of receiving the voice acquired by the photographing apparatus and transmitted as it is when receiving the voice, and further creating an audio file that stores the voice received in the receiving step The image management method according to claim 23, further comprising steps. The reception step further receives position information, date / time information, and schedule information transmitted from the photographing apparatus,
The additional information generation step is a step of generating additional information based on the text obtained by recognizing the voice in the voice recognition step and also based on the position information, date / time information, and schedule information received in the reception step. 24. The image management method according to claim 23. The image management server includes a map database for managing maps,
27. The image management method according to claim 26, wherein the additional information generating step is a step of generating additional information based on a map in the map database. A face recognition step for recognizing a face reflected on the image from the image received in the reception step;
The additional information generation step is a step of generating additional information based on the text on which the voice is recognized in the voice recognition step and also based on the face on the image recognized in the face recognition step. The image management method according to claim 23, characterized in that: The image management server further comprises a face information database for managing face information,
29. The image management method according to claim 28, wherein the additional information generating step is a step of generating additional information based on face information in the face information database.   A face information adding step of generating new face information based on the text in which the voice is recognized in the voice recognition step and the face on the image recognized in the face recognition step and adding the new face information to the face information database. 30. The image management method according to claim 29.   The additional information generation step includes a command recognition step for recognizing a command related to additional information generation based on text obtained by recognizing a voice in the voice recognition step, and generates additional information according to the command recognized in the command recognition step. 24. The image management method according to claim 23, wherein the image management method includes:

Images