JP2009111571A - Imaging apparatus, image index attaching method, image index attaching system, information processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enhance usability of system by effectively reading images stored in an automatic imaging apparatus not based on user manipulations. <P>SOLUTION: Whether a user of the imaging apparatus is in the condition that he is meeting the other person or not can be identified, for example, by analyzing images and voices or by analyzing positional relationship with each imaging apparatus. Based on the result of this determination, an index is attached to the image fetched by the automatic imaging operation. Accordingly, an index can be attached to the image that is considered as the important image among those fetched by the automatic imaging process in view of achieving highly effective reading of images. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an imaging apparatus that automatically captures captured image data without being based on a user's operation as an imaging apparatus, and an image index assigning method for indexing the captured image data. . The present invention also relates to an image indexing system including an imaging device and an information processing device, and the information processing device.

For example, a camera that records a scene that the user sees in daily life as image data by automatically taking a periodic image with a camera worn by the user, such as a life log camera, has been proposed. By using this life log camera, it becomes possible to leave a user's action history, memories, etc. as image data.
In recent years, the publication of diary-like records (for example, blogs) has been increasing on the Internet, but the life log camera as described above is also suitable for use in, for example, procurement of photographic material for such blogs.

In addition, about the related prior art, the following patent documents can be mentioned.
JP 2006-270627 A JP 2007-13843 A

  Here, when a user wears a life log camera that performs periodic automatic imaging as described above and performs daily actions, it is assumed that a large amount of image data is captured and stored. For example, when image data is captured every 10 seconds for automatic imaging, as many as 4320 image data are stored per half day (12 hours) by simple calculation.

  If a large amount of image data is stored in this manner, even if the user looks back on the behavior of the day, it is very burdensome to browse the stored images in order. In other words, in an apparatus that performs automatic imaging, if there is no contrivance for browsing images that are stored in large quantities in this way, the user will not be able to efficiently look back on the action history, and the usefulness of the system Will be seriously harmful.

In the present invention, in view of the above points, for example, when performing automatic imaging for life log use, an image that is important for the user among images that have a large storage amount is efficiently browsed. With the goal.
Therefore, in the present invention, the imaging apparatus is configured as follows.
That is, an imaging unit that captures an image of a subject and obtains captured image data and a control unit are provided.
And the control means
A captured image automatic capture process that automatically captures captured image data obtained by the imaging means without the user's operation;
A determination process for determining whether or not it is in a state where it can be estimated that a person is met based on input information;
Indexing for controlling the captured image data captured by the captured image automatic capturing process to be indexed when it is determined that the person can be estimated to meet the person through the determining process. Process.

Further, in the determination processing by the control means, it is determined whether or not it is possible to estimate that the person is meeting with a person based on the image analysis result of the captured image data obtained by the imaging means. For example, based on the image analysis result regarding whether or not the image that can be estimated to be a human face is included in the captured image data, it is determined whether or not it is possible to estimate that the person is meeting a person.
Alternatively, in the determination process, it is determined whether or not it can be estimated that the user is meeting with a person based on the sound analysis result of the sound collection data obtained by collecting the external sound. For example, it further comprises storage means for storing predetermined voiceprint data, and in the discrimination process, based on a voice analysis result as to whether or not there is a data portion that matches the voiceprint data in the collected sound data. It is determined whether or not it is in a state where it can be estimated that the person is met.
Alternatively, a short-range wireless communication unit that performs short-range wireless communication with an external device is provided, and in the determination process, it is determined whether or not another device exists in a communicable range of the short-range wireless communication unit. By doing so, it is determined whether or not it is in a state where it can be estimated that a person is met.

In the present invention, an image index assigning system including the information processing device and a plurality of imaging devices configured to be capable of data communication with the information processing device is configured as follows. It was.
That is, the information processing apparatus includes information processing apparatus side control means for executing the following processes.
That is, position information acquisition processing for performing data communication to acquire information on the location of the imaging device;
A search process for searching for an imaging apparatus in which another imaging apparatus exists within a predetermined range based on information on the location of the imaging apparatus acquired by the position information acquisition process;
As a result of the search by the search process, there is provided an information processing apparatus side control means for executing a notification process for notifying an imaging apparatus identified as having another imaging apparatus within the predetermined range.
Further, the imaging apparatus includes an imaging unit that captures an image of a subject and obtains captured image data, and an imaging apparatus side control unit.
And the said imaging device side control means is
A captured image automatic capture process that automatically captures captured image data obtained by the imaging means without the user's operation;
Based on the notification by the notification process on the information processing apparatus side, an indexing process is performed to control the captured image data automatically captured by the captured image automatic capture process to be indexed. To do.

In the present invention, the information processing apparatus is configured as follows.
That is, the information processing apparatus of the present invention includes position detection means for detecting the current position, and data between a plurality of imaging apparatuses that are automatically captured image data without being based on user operations. An information processing apparatus configured to be able to communicate, and includes a control unit that executes the following processing.
That is, position information acquisition processing for acquiring current position information detected by the position detection means in the imaging device;
Based on the position information acquired by the position information acquisition process, a search process for searching for an image pickup apparatus in which another image pickup apparatus exists within a predetermined range;
As a result of the search by the search process, there is provided control means for executing a notification process for notifying an image pickup apparatus specified as having another image pickup apparatus within the predetermined range.

According to the present invention, an index can be attached to automatically captured image data in accordance with a determination result as to whether or not the user who carries the imaging device is in a state where he / she is meeting a person. . At this time, it is determined whether or not the person is in a state of meeting with a person. For example, (1) whether or not an image corresponding to a person's face is displayed in the captured image data, and (2) sound pickup sound At least one of the following conditions: whether or not a portion matching the predetermined voiceprint data is detected in the data, and (3) whether or not another device (user) exists within a predetermined range as a positional relationship between the devices. This can be done by determining whether or not it is established.
Here, in the user's daily life, an important event is that the user is meeting other people, such as being facing other people, having a conversation, or acting together. It can be considered that many occur. According to the present invention described above, an index is assigned to image data that is automatically captured in a situation where the user is meeting with another person, which is considered to be an important point in daily life. Can be granted.

According to the present invention, in an imaging apparatus that performs automatic imaging, it is possible to add an index to image data captured in a situation where the user is meeting another person. With this, it is possible to separate a part that is important and a part that is not so from image data stored in large quantities by automatic imaging.
That is, by providing such an index, it is possible to allow a user to efficiently browse a large amount of image data stored by automatic imaging, and it is possible to effectively reduce the user burden when looking back on the action history. . Thus, for example, a system that records and reflects the daily life of the user can be realized as a more useful system.

Hereinafter, the best mode for carrying out the present invention (hereinafter referred to as an embodiment) will be described.

<First Embodiment>
[External appearance of imaging device]

Various forms are assumed as the imaging device 1 of the embodiment, and examples of their appearance are illustrated in FIGS. 1 (a), 1 (b), and 1 (c).
First, FIG. 1A shows an imaging apparatus 1 of a neck type (hanging type). The imaging apparatus 1 has a part to which a strap is attached, for example, and is attached by attaching the strap to this part and hanging it on the user's neck as shown in the figure. The user only needs to wear the imaging lens 3L included in the imaging device 1 so that the imaging can be performed with the front direction of the user as the subject direction.
Although not shown, for example, a display unit used for an imaging monitor or reproduction of a captured image may be provided on the back surface of the imaging device 1 or the like.

FIG. 1B shows an imaging apparatus 1 that is a glasses-type display camera. The imaging device 1 has a mounting unit having a frame structure that makes a half turn, for example, from both sides of the head to the back of the head, and is mounted on the user by being placed on both ear shells as illustrated.
In the imaging device 1, the imaging lens 3 </ b> L is arranged forward so that imaging is performed with the user's visual field direction as the subject direction when the user is wearing the imaging device 1.
Further, in the wearing state as shown in the figure, a pair of display units 5 and 5 for the left eye and the right eye are arranged immediately before both eyes of the user, that is, at a position where a lens in normal glasses is located. Has been. For example, a liquid crystal panel is used for the display unit 5. By controlling the transmittance, a through state as shown in the figure, that is, a transparent or translucent state can be obtained. Since the display unit 5 is in the through state, there is no problem in normal life even if the user always wears it like glasses.
In addition to a pair of display units 5 corresponding to both eyes, a configuration in which one display unit 5 is provided corresponding to one eye is also conceivable.
FIG. 1C shows an example of the appearance in which the display unit 5 is not provided.

In FIG. 1, the neck-mounted type or the eyeglass-type imaging device 1 is exemplified, but various structures for the user to wear the imaging device 1 can be considered. For example, a headphone type, a neckband type, an ear hook type, or any other mounting unit may be worn by the user. Furthermore, for example, a configuration may be adopted in which the user attaches to normal glasses, a visor, headphones, or the like with an attachment such as a clip. Moreover, it does not necessarily have to be worn on the user's head.
In addition, in the case of FIG. 1A, the imaging direction is the front direction of the user, but the imaging device 1 may be worn on the neck so as to capture the back of the user when worn.
In the case of FIGS. 1B and 1C, the imaging direction is the user's field of view, but the imaging lens 3L is attached so as to capture the user's rear, side, upper, and foot directions when worn. A configuration in which a plurality of imaging systems having the same or different imaging directions are provided is also conceivable.
Furthermore, in FIGS. 1A, 1B, and 1C, an imaging direction variable mechanism that can change the subject direction manually or automatically may be provided for one or a plurality of imaging lenses 3L.

Needless to say, forms other than those shown in FIGS. 1A, 1B, and 1C are also conceivable as an imaging apparatus that captures moving images and still images. For example, an imaging apparatus 1 that can be installed in a vehicle and that can image inside or outside the vehicle can be assumed as the present embodiment. For example, it is an imaging device attached to take an image of the interior of a vehicle or an imaging device attached to take an image of a front landscape, a rear landscape, or the like of a car.
Further, an apparatus such as a mobile phone, a PDA (Personal Digital Assistant), and a portable personal computer that has a function as an imaging apparatus can be assumed as the imaging apparatus 1 of the present embodiment.
In these various forms, for example, a microphone that collects external sound may be provided, and an audio signal to be recorded together with image data may be obtained during imaging. Further, a speaker unit and an earphone unit that perform audio output may be formed.
It is also conceivable that a light emitting unit that performs illumination in the direction of the subject is provided in the vicinity of the imaging lens 3L using, for example, an LED (Light Emitting Diode) or a flash light emitting unit for taking a still image.

[Example of internal configuration of imaging device]

Next, an internal configuration example of the imaging device 1 according to the embodiment will be described.
FIG. 2 is a block diagram illustrating an internal configuration of the imaging apparatus 1.
As illustrated, the imaging apparatus 1 includes a system controller 2, an imaging unit 3, an imaging control unit 4, a display unit 5, a display control unit 6, an operation input unit 7, a storage unit 8, a communication unit 9, an image analysis unit 10, A voice analysis unit 11, a bus 12, and a voice input unit 13 are included.

  The system controller 2 includes, for example, a microcomputer including a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), a nonvolatile memory unit, and an interface unit, and controls the entire imaging apparatus 1. The control unit The system controller 2 performs various arithmetic processes and exchanges control signals with each unit via the bus 12 based on a program held in an internal ROM or the like, and causes each unit to execute a required operation.

The imaging unit 3 includes an imaging optical system 3a, an imaging element unit 3b, and an imaging signal processing unit 3c.
The imaging optical system 3a in the imaging unit 3 includes a lens system including the imaging lens 3L illustrated in FIG. 1, a diaphragm, a zoom lens, a focus lens, and the like, and performs a focus operation and a zoom operation on the lens system. A drive system or the like for performing the operation is provided.
The imaging device unit 3b in the imaging unit 3 is provided with a solid-state imaging device array that detects imaging light obtained by the imaging optical system 3a and generates an imaging signal by performing photoelectric conversion. The solid-state imaging device array is, for example, a CCD (Charge Coupled Device) sensor array or a CMOS (Complementary Metal Oxide Semiconductor) sensor array.
The imaging signal processing unit 3c in the imaging unit 3 includes a sample hold / AGC (Automatic Gain Control) circuit that performs gain adjustment and waveform shaping on a signal obtained by the solid-state imaging device, and a video A / D converter, and includes digital data. Obtained image data. Also, white balance processing, luminance processing, color signal processing, and the like are performed on the captured image data.

Imaging is performed by the imaging unit 3 including the imaging optical system 3a, the imaging element unit 3b, and the imaging signal processing unit 3c, and captured image data is obtained.
Image data obtained by the operation of the imaging unit 3 is processed by the imaging control unit 4.
The imaging control unit 4 performs processing such as image compression processing for compressing captured image data at various compression ratios, image size conversion processing, and image format conversion processing according to the control of the system controller 2, and according to the operation situation. The captured image data is transferred to the storage unit 8, the display control unit 6, the communication unit 9, and the like.
Further, the imaging control unit 4 controls on / off of imaging in the imaging unit 3, shutter processing, drive control of the zoom lens and focus lens of the imaging optical system 3a, sensitivity of the imaging element unit 3b, based on an instruction from the system controller 2. Control of the frame rate, parameter control of each process of the imaging signal processing unit 3c, execution process setting, and the like are also performed.

A display unit 5 and a display control unit 6 are provided as a configuration for displaying to the user in the imaging apparatus 1.
The display unit 5 is provided with a display panel unit such as a liquid crystal display and a display driving unit for driving the display panel unit. The display driving unit includes a pixel driving circuit for displaying the image data supplied from the imaging control unit 4 on the display panel unit. The pixel drive circuit applies a drive signal based on the video signal to each pixel arranged in a matrix in the display panel unit at a predetermined horizontal / vertical drive timing, and causes display to be performed.

The display control unit 6 drives the pixel driving circuit in the display unit 5 based on the control of the system controller 2 to execute a predetermined display. For example, monitor display of an image captured by the imaging unit 3 and display of an image reproduced by the storage unit 8 are executed.
For these displays, for example, brightness level adjustment, color correction, contrast adjustment, sharpness (outline emphasis) adjustment, and the like can be performed. Also, generation of an enlarged image obtained by enlarging a part of the image data, generation of a reduced image, soft focus, mosaic, luminance inversion, highlight display (highlight display) of a part of the image, change in atmosphere of the whole color, etc. Image effect processing can also be performed.

The operation input unit 7 has operation elements such as keys, buttons, and dials. For example, an operation element for power on / off operation, an operation related to automatic imaging, a required input operation, and the like. An operator is formed. When not only automatic imaging but also imaging in accordance with the user's shutter operation is possible, for example, an operator used for shutter operation, zoom operation, exposure setting operation, self-timer operation, etc. It may be formed.
The operation input unit 7 supplies information obtained from such an operator to the system controller 2, and the system controller 2 performs necessary arithmetic processing and control corresponding to the information.

The image analysis unit 10 is configured to determine whether or not the user of the imaging apparatus 1 is in a state where it can be estimated that the user of the imaging device 1 is meeting another person based on the captured image data supplied from the imaging control unit 4. Perform image analysis processing.
The image analysis unit 10 performs image analysis processing for detecting a data portion that can be recognized (estimated) as a human face in the captured image data. This analysis process can be realized using, for example, a well-known face recognition technique.
In particular, in this case, in order to prevent the passing person from being detected, the image analysis unit 10 in this case indicates that the time length for detecting the data portion that can be recognized as the same face is a predetermined length or more. It is added as a condition. That is, after a data portion that can be recognized as a human face is detected, the time length for detecting the data portion that is regarded as the same face becomes equal to or longer than the predetermined length, so that a data portion that can be recognized as a human face is detected. The detection information to the effect is obtained.

  When it is determined that a data portion that can be recognized as a human face is detected in the captured image data by the image analysis process, the image analysis unit 10 supplies detection information to that effect to the system controller 2 as human detection information. . On the other hand, if a data portion that can be recognized as a human face has not been detected, information indicating that fact is supplied to the system controller 2 as the human detection information.

  In this case, it is assumed that the image analysis by the image analysis unit 10 is performed on the captured image data of the moving image supplied via the imaging control unit 4, but it is obtained at the capturing timing by automatic imaging, for example. The captured image data as a still image, such as the captured image data, may be performed.

  Based on the person detection information supplied from the image analysis unit 10 as described above, the system controller 2 determines whether or not it is in a state where it is assumed that a person is met. That is, when the person detection information indicates that a data portion that can be recognized as a person's face has been detected in the captured image data, it is determined that the person is in a state of meeting with a person. Get results. On the other hand, if the person detection information indicates that no data portion that can be recognized as a person's face is detected in the captured image data, a determination result indicating that the person is not in a state of meeting with a person is displayed. obtain.

The voice analysis unit 11 performs a voice analysis process for making it possible to determine whether or not it is in a state where it can be estimated that a person is met based on the microphone sound collection data input from the voice input unit 13.
Here, in order to effectively perform the voice analysis processing by the voice analysis unit 11, voiceprint data (voiceprint data 8 b described later) needs to be registered in the imaging device 1 in advance. In the case of this example, as the voiceprint data 8b, data based on the voice of the user wearing (carrying) the imaging device 1 is registered. This is because the possibility that the user of the imaging apparatus 1 is uttering a voice by conversation when meeting with another person is very high.
Note that the voiceprint data 8b may be acquired and stored in advance by another device, or the voice data as the user's voice input via the voice input unit 13 is analyzed. It can also be configured such that the unit 11 analyzes and generates and stores this.

In the voice analysis processing by the voice analysis unit 11, it is analyzed whether or not the collected sound data input from the voice input unit 13 includes a data portion that matches the voice print data 8 b stored as described above. Process. When the voice analysis unit 11 determines that a data portion matching the voice print data 8b is obtained in the collected sound data as a result of the voice analysis processing, the system controller uses the information indicating that as human detection information. 2 is supplied. If no matching data portion is obtained, information indicating that is supplied to the system controller 2 as person detection information.
Based on the person detection information supplied from the voice analysis unit 11 as described above, the system controller 2 determines whether or not it is in a state where it is assumed that a person is met. That is, when the person detection information indicates that a data portion that matches the voiceprint data 8b is obtained in the collected sound data, a determination result is obtained that the person is considered to be in contact with the person, When it is indicated that the data portion that matches the voiceprint data 8b is not obtained in the collected sound data, a determination result indicating that the person is not in a state of meeting with a person is obtained.

The storage unit 8 is used for storing captured image data and other various data.
The storage unit 8 may be configured by a solid-state memory such as a flash memory, or may be configured by an HDD (Hard Disk Drive), for example.
Further, instead of a built-in recording medium, a portable recording medium, for example, a recording / reproducing drive corresponding to a recording medium such as a memory card incorporating a solid-state memory, an optical disk, a magneto-optical disk, or a hologram memory may be used.
Of course, both a built-in type memory such as a solid-state memory and an HDD, and a recording / reproducing drive for a portable recording medium may be mounted.
The storage unit 8 records / reproduces captured image data and other various data based on the control of the system controller 2.

In the storage unit 8 in this case, a captured image data recording area 8a is prepared as a recording area. The captured image data recording area 8a is an area for storing (storing) captured image data obtained by the imaging unit 3. For example, if imaging is performed at regular time intervals as regular automatic imaging, the captured image data is recorded in the captured image data recording area 8a at regular intervals.
The captured image data recording area 8a does not have to be set as a fixed area in the storage unit 8, and in this example, the captured image data may be stored in the storage unit 8. .

  The storage unit 8 also stores the voiceprint data 8b described above. The storage location of the voiceprint data 8b is not limited to the storage unit 8, but may be other storage means (for example, the above-described nonvolatile memory unit included in the system controller 2).

  In the case of this example, the storage unit 8 stores index point management information 8c as information for managing an index added to the captured image data. The index point management information 8c will be described later.

The communication unit 9 is provided as a part that performs data communication with various external devices.
For example, data transmission / reception with a server device (not shown) may be performed. In that case, for example, a configuration may be adopted in which network communication is performed via short-range wireless communication with a network access point by a method such as a wireless LAN, or direct wireless communication is performed with a server device having a corresponding communication function. It may be a thing.
In addition, the communication unit 9 may be connected to a device such as a personal computer using an interface such as a USB (Universal Serial Bus) system to transmit and receive data.
The communication unit 9 can transfer, for example, captured image data captured and stored in the storage unit 8 to a personal computer or other external device. Accordingly, a large number of captured image data recorded and stored in the storage unit 8 by periodic imaging as a life log can be reproduced and displayed on the display unit 5 by the imaging device 1 itself, and an external device such as a personal computer. Can be transferred to and played back on the external device.

In particular, in the case of this example, the communication unit 9 performs wireless data communication directly with an external device by using, for example, Bluetooth or a wireless LAN ad hoc mode (Wi-Fi: Wireless Fidelity ad hoc mode). Is possible.
In order to enable direct wireless data communication with such an external device, the communication unit 9 performs a process of detecting a device capable of external wireless communication. Information representing the detection processing result is supplied to the system controller 2 as needed.
Based on the detection information supplied from the communication unit 9 as described above, the system controller 2 can determine whether there is an external device capable of wireless communication.

Here, in short-range wireless communication such as the above-described Bluetooth or wireless LAN ad hoc mode, when there is another device within the communicable range, the user of the imaging apparatus 1 and the person carrying the other device are It is very close. That is, in such a state, there is a high possibility that the user is meeting a person carrying the other device.
Therefore, in this example, the user of the imaging device 1 is in a state where it is assumed that the user of the imaging apparatus 1 meets another person depending on whether or not there is another device within the wireless communicable range by short-range wireless communication. Whether or not it is determined.

  In this case, the detection information that the communication unit 9 supplies to the system controller 2 as the above detection processing result can be handled as human detection information. That is, when the person detection information from the communication unit 9 indicates that a device capable of wireless communication to the outside is detected, the system controller 2 is in a state where the user is meeting another person. The discrimination result is obtained. On the other hand, if the person detection information from the communication unit 9 indicates that there is no device capable of wireless communication outside, a determination result is obtained that the user is not in a state of meeting with another person. .

Note that the configuration of the imaging apparatus 1 shown in FIG. 1 is merely an example, and various components may be added or deleted according to an operation example or function actually performed.

[Add image index]

The imaging device 1 according to the embodiment basically performs an imaging operation (automatic imaging operation) that is not based on a user's shutter operation when used for a life log application. For example, automatic imaging is periodically performed, and captured image data is stored in the storage unit 8. For example, automatic imaging is performed at predetermined time intervals such as an interval of 5 seconds, an interval of 10 seconds, and an interval of 30 seconds. Of course, instead of periodic imaging, automatic imaging may be performed at irregular timing according to some trigger (trigger other than the user's shutter operation). Furthermore, if the user performs a shutter operation during execution of regular or irregular automatic imaging, the imaging process may be performed normally at that time.

Here, as described above, for example, in an imaging apparatus that performs automatic imaging as a life log camera, the storage amount of captured image data is enormous, and the burden on the user when browsing this is increased.
In order to improve this point, in the present embodiment, it is assumed that the user of the imaging apparatus 1 is in a state where he / she meets another person among the captured image data captured / saved by automatic imaging. An index is attached to image data picked up under such circumstances.

A specific operation example for assigning an image index as such an embodiment will be described with reference to FIGS.
FIG. 3 is a diagram for explaining a determination method for determining whether or not a person is in a state of meeting a person, which is a condition to be indexed in this case. The state where user A and user B face each other is shown.
As shown in this figure, under the situation where the user meets a person, there is a high possibility that each person will be close and have a conversation. In such a state, it is expected that the face of the partner user is projected in the captured image by the imaging apparatus 1 and that the voice uttered by the user is detected along with the conversation. Furthermore, it is assumed that another imaging device 1 is detected within the communicable range by short-range wireless communication. In this figure, as an example, the imaging device 1 is suspended as shown in FIG. Although the case of the type is shown, the same applies to the case where the imaging device 1 is of another type such as the eyeglass type as shown in FIG.

Based on the above points, in this example, when determining whether or not it is in a state of meeting with a person, the determination as shown in (1) to (3) in the figure is performed.
That is, (1) whether a data portion that can be recognized as a face is detected in the captured image data for a predetermined time or more, (2) whether voice that matches the voiceprint data 8b is detected in the collected sound data, or (3) communication is possible. It is assumed that each item is determined as to whether a correct device has been detected.

FIG. 4 is a diagram for explaining a specific example of indexing for captured image data performed based on the determination result of each item described above, and captured image data captured and stored by automatic imaging, The relationship with the index attached is schematically shown.
As shown in this figure, in the case of the present embodiment, as an index, a separate index is attached to each discrimination item as described above. Specifically, the index derived from the image analysis to be attached based on the discrimination result of (1) above, the index derived from the voice analysis based on the discrimination result of (2), and the discrimination of (3) This is an index derived from position analysis attached based on the result.

Here, as described above with reference to FIG. 2, for the system controller 2, the image analysis unit 10, the sound analysis unit 11, and the communication unit 9 respectively perform the above (1), (2), and (3). The person detection information corresponding to the discrimination item is input. The system controller 2 acquires the human detection information input from the image analysis unit 10, the audio analysis unit 11, and the communication unit 9 in this way, for example, at every timing when captured image data is captured by automatic imaging. Then, based on the person detection information of each item acquired in this way, it is determined whether or not the user is in a state where he / she is meeting another person.
In addition, if there is an item that can be determined to be in a state of meeting with a person as a result of these determinations, control is performed so that an index corresponding to the item is attached to the captured image data. I do. That is, based on the person detection information from the image analysis unit 10, when an identification result indicating that the person is in a state of meeting with a person is obtained, an index derived from image analysis is attached to the captured image data. In addition, when it is determined that the person is in a state of meeting with the person based on the person detection information from the voice analysis unit 11, an index derived from voice analysis is attached to the captured image data. To be controlled. Moreover, when the determination result that it is in the state assumed to be meeting the person based on the person detection information from the communication unit 9 is obtained, control is performed so that an index derived from the position analysis is attached.

  As a result, as shown in FIG. 4, for the captured image data that is sequentially captured and stored by automatic imaging, it is determined that the determination result that the person is in a state of meeting with a person is obtained. Indexes corresponding to the items are sequentially attached.

Here, in the present embodiment, the index assignment for the captured image data as described above is realized by the system controller 2 managing each index point using the index point management information 8c shown in FIG. To do.
The index point management information 8c is information for managing the points to be indexed from the image analysis origin, the voice analysis origin, and the position analysis origin described above. Although description by illustration is omitted, as the index point management information 8c in this case, for example, for each ID of captured image data captured and stored by automatic imaging, the image analysis origin, the voice analysis origin, and the position analysis origin Information indicating the presence or absence of each index is associated.
As described above, the system controller 2 is in a state of being in contact with a person as a result of determination for each item executed at the timing at which captured image data is captured and stored by automatic imaging as described above. For an item for which a determination result is obtained, information indicating that there is an index for the item is associated with the ID information of the stored captured image data. Thereby, an index point can be managed for each discrimination item.

By providing the image index as the present embodiment described above, it is possible to estimate that the user of the imaging apparatus 1 is in a state where it can be estimated that the user is meeting another person. Can be indexed.
Here, as mentioned above, important events in the user's daily life are that the user is facing other people, having a conversation, or acting together. It can be thought that it often occurs in the state of meeting with a person. Based on this, according to the present embodiment in which index is applied to image data automatically captured under the situation that the user is meeting with another person as described above, It can be said that an index is attached to an image obtained by photographing a scene that is important in (1).

  By providing an index according to such an embodiment, it is possible to separately present important parts and non-important parts of image data stored in large quantities by automatic imaging. That is, by providing such an index, the user can efficiently browse a large amount of image data stored by automatic imaging, and the burden on the user when looking back on the action history can be effectively reduced. As a result, for example, a system that records and reflects the user's daily life can be realized as a more useful system.

Further, in the present embodiment, separate indexes based on a plurality of determination items for determining whether or not the person is in a state of meeting with a person as described above are provided. According to the above, for example, at the time of browsing, it is possible to search for image data by designating an arbitrary item among the plurality of determination items, so that the degree of freedom in searching for image data can be expanded.

[Processing operation]

Next, a control example for realizing the operation as the first embodiment described above will be described with reference to the flowchart of FIG. FIG. 5 shows control processing executed by the system controller 2 based on a program stored in the internal ROM.
Further, the processing operation shown in this figure is a processing operation for automatic imaging performed by the system controller 2 and captures captured image data obtained by the imaging unit 3 at regular intervals and stores it in the storage unit 8. This is executed at every timing when the processing for the above is performed.

  In FIG. 5, first, in step S101, human detection information is acquired. That is, the human detection information from the image analysis unit 10 described above, the human detection information from the voice analysis unit 11, and the human detection information from the communication unit 9 are acquired.

In the subsequent step S102, first, based on the person detection information from the image analysis unit 10 as the image analysis result, it is determined whether or not the person is in a state of meeting with a person.
In this step S102, the human detection information from the image analysis unit 10 indicates that a data portion that can be recognized as a human face has been detected in the captured image data, and a person is met accordingly. If a positive result is obtained as being in the state, the process for registering the image analysis-derived index points is executed in step S103. That is, the process of updating the information content of the index point management information 8c is performed so that the ID of the captured image data stored by automatic imaging is associated with the information indicating that there is an index derived from image analysis.
When the process of step S103 is executed, the process proceeds to step S104.

  On the other hand, in step S102, the human detection information from the image analysis unit 10 indicates that no data portion that can be recognized as a human face has been detected in the captured image data, and is meeting a person. If a negative result is obtained because it is not in a state to be performed, the process proceeds to step S104 as it is after passing through step S103.

In step S104, based on the person detection information from the voice analysis unit 11 as a voice analysis result, it is determined whether or not it is in a state where a person is met.
In step S104, the person detection information from the voice analysis unit 11 indicates that a data portion that is matched with the voiceprint data 8b is obtained in the collected sound data, and it is assumed that a person is met. In step S105, a process for registering speech analysis-derived index points is executed. That is, the information content of the index point management information 8c is updated so that the ID of the captured image data saved by automatic imaging is associated with the information indicating that there is an index derived from speech analysis.
When the process of step S105 is executed, the process proceeds to step S106.

  On the other hand, in step S104, the person detection information from the voice analysis unit 11 indicates that a data portion that matches the voiceprint data 8b is not obtained in the collected sound data. If a negative result is obtained that the current state is not present, step S105 is passed, and the process proceeds directly to step S106.

In step S106, it is determined based on the person detection information from the communication unit 9 as the communicable device analysis result whether or not the person is in a state of meeting a person. When the person detection information from the communication unit 9 indicates that an externally communicable device has been detected, and a positive result is obtained that the person is said to be meeting a person accordingly In step S107, processing for registering the index point derived from speech analysis is executed. That is, the information content of the index point management information 8c is updated so that the ID of the captured image data stored by automatic imaging is associated with the information indicating the presence of an index derived from position analysis.
When the process of step S107 is executed, the processing operation shown in FIG.

On the other hand, in step S106, the person detection information from the communication unit 9 indicates that no communicable device has been detected, and a negative result has been obtained that the person is not in a state of meeting with a person. In this case, the processing operation is terminated after passing through step S107.

<Second Embodiment>

Next, a second embodiment will be described.
In the second embodiment, the position analysis technique is changed for the image index assigning operation as the first embodiment described above. Specifically, in the second embodiment, it is determined whether or not the distance between the devices is close by using the area information where the wireless repeater is installed in the cellular phone network (WAN) or the Internet network. Thus, it is determined whether or not the person is in a state of meeting with a person.

In order to realize the operation as the second embodiment, a system as shown in FIG. 6 is constructed.
In FIG. 6, the system of the second embodiment includes a plurality of imaging devices 1, a plurality of base stations 30, and a server device 20 as shown in the figure.
First, as the imaging apparatus 1 in this case, the communication unit 9 is configured to be able to communicate with a base station 30 as a radio base station in a mobile phone network as shown in the figure, for example. Note that, when performing communication using the Internet network, the base station 30 serves as a wireless LAN access point.
In this case, the base station 30 is fixedly installed for each area (for example, area A and area B in the figure), and each base station 30 and its installation area correspond one-to-one.
Each imaging device 1 in this case is configured to communicate with a required device on the network (in particular, the server device 20 in this case) via the nearest base station 30. Thus, since each imaging device 1 communicates via the nearest base station 30, each base station 30 grasps the imaging device 1 existing in its own area.

  The server device 20 can identify each base station 30 (the nearest base station 30) that can communicate with each imaging device 1 by making an inquiry to each base station 30. That is, the location area of each imaging device 1 can be specified by this information.

  In the second embodiment, after the server device 20 obtains information on the location area of each imaging device 1 in this way, it is determined whether or not there are a plurality of imaging devices 1 in the same area. To do. Then, when there are a plurality of imaging devices 1 in the same area as described above, a notification that there is another device in the same area is sent to each corresponding imaging device 1.

  Each imaging device 1 determines whether or not it is in a state of meeting a person based on the presence or absence of such a notification from the server device 20. In other words, if there is another imaging device 1 in the same area, the user of the imaging device 1 is quite close, so that it is said that it is in a state where it is said that it is meeting a person. A discrimination result is obtained.

FIG. 7 shows an internal configuration example of the server device 20.
The server device 20 includes a control unit 21, a time measuring unit 22, a storage unit 23, and a communication unit 24.
In FIG. 7, the control unit 21 performs overall control of the server device 20. The control unit 21 includes a CPU, a ROM, a RAM, a nonvolatile memory unit, an interface unit, and the like. For example, based on an operation program stored in a storage unit such as the ROM, each control unit 21 includes Perform control and perform necessary actions.

  The time counting unit 22 measures the current time in, for example, year: month: day: hour: minute: second, and outputs the current time information obtained thereby to the control unit 21. Thus, based on the information of the current time measured by the time counting unit 22, the control unit 21 can perform processing requiring time management such as generation of various history information.

The storage unit 23 records and reproduces various data based on the control of the control unit 21.
The storage unit 23 may be configured by a solid-state memory such as a RAM or a flash memory, or may be configured by an HDD, for example. Further, instead of a built-in recording medium, a portable recording medium, for example, a recording / reproducing drive corresponding to a recording medium such as a memory card incorporating a solid-state memory, an optical disk, a magneto-optical disk, or a hologram memory may be used. Further, both a built-in type memory such as a solid-state memory and an HDD, and a recording / reproducing drive for a portable recording medium may be mounted.

  The communication unit 24 is configured to be able to perform data communication with an external device (particularly the imaging apparatus 1) via the base station 30 shown in FIG. The communication unit 24 in the server device 20 may be basically configured to perform communication by wired network connection, but may be configured to be wirelessly connected to the network.

FIG. 8 is a flowchart showing an example of control in the server device 20 for realizing the operation as the second embodiment described above.
FIG. 8 shows a control process executed by the control unit 21 based on the program stored in the ROM in the control unit 21 described with reference to FIG.
Further, the processing operation shown in this figure is repeatedly executed at a predetermined cycle.

In FIG. 8, first, in step S201, processing for confirming the location area for each device is executed. That is, for example, based on the result of inquiring each base station 30 as described above, information on the location area of each imaging device 1 is obtained.
In addition, the information on the location area of each imaging device 1 can also be confirmed by detecting the information of the base station 30 via the result of communication with each imaging device 1.

In a succeeding step S202, it is determined whether or not there are a plurality of devices for each area. Then, in the next step S203, it is determined whether or not there is a corresponding area. That is, based on the result of the determination process in step S202, it is determined whether there is an area where a plurality of imaging devices 1 are present.
If an affirmative result is obtained in step S203 that the corresponding area exists, the process proceeds to step S204 to notify each device in the area that there is another device in the area. That is, the communication unit 24 is controlled so that the above notification is performed for each imaging device 1 in the corresponding area. When the process of step S204 is executed, the processing operation shown in FIG.
On the other hand, if a negative result is obtained in step S203 that there is no corresponding area, the process operation shown in FIG.

In addition, although description by illustration is abbreviate | omitted, in each imaging device 1 in the case of 2nd Embodiment, the notification which the server apparatus 20 side performs as mentioned above was demonstrated in step S106 in previous FIG. It is acquired as information that replaces the human detection information as the result of analyzing the communicable device. That is, in this case, the imaging device 1 (system controller 2) obtains the determination result that it is in a state that it is assumed to meet a person when the notification from the server device 20 is received, A process for registering the index point derived from the analysis is executed. If there is no notification, a determination result indicating that the person is not in a state of meeting with a person is obtained, and index points derived from position analysis are not registered.

<Third Embodiment>

In the third embodiment, as in the second embodiment, the position analysis technique is changed with respect to the operation as the first embodiment. Specifically, in the third embodiment, the imaging apparatus 31 provided with position detection means for detecting the current position is used, and each imaging is performed in the server apparatus 40 configured to be capable of data communication with the imaging apparatus 31. By determining whether or not there is another imaging device 50 at a close position from the information on the current position acquired from the device 31, it is estimated whether or not it is in a state where it is assumed that a person is met. is there.

FIG. 9 illustrates an internal configuration example of the imaging device 31 according to the third embodiment.
In FIG. 9, parts already described in FIG. 2 are given the same reference numerals and description thereof is omitted.
As can be seen by comparing FIG. 9 with the previous FIG. 2, the imaging device 31 of the third embodiment is different from the configuration of the imaging device 1 shown in FIG. (Global Positioning System) unit 14 is added. The GPS unit 14 receives radio waves from a GPS satellite (not shown), and obtains latitude / longitude information as information representing the current position. Then, the current position information based on the latitude / longitude is output to the system controller 2.

FIG. 10 is a diagram schematically illustrating the configuration and specific operation contents of the system according to the third embodiment.
As shown in FIG. 10, the system according to the third embodiment includes a plurality of imaging devices 31, a network 32, and a server device 40.
In the system according to the third embodiment, each imaging device 31 transmits current position information detected by the GPS unit 14 to the server device 40 via the network 32. In this case, transmission of the current position information from each imaging device 31 is repeatedly performed at a predetermined cycle, for example.

  The server device 40 has the same internal configuration as the server device 20 shown in FIG. In this case, the server device 40 (control unit 21) acquires the current position information transmitted from each imaging device 31, and for each imaging device 31, determines whether there is another imaging device 31 within a predetermined distance. Determine. If there is another imaging device 31 within a predetermined distance, the corresponding imaging device 31 is notified to that effect.

Here, when determining whether or not there is another imaging device 31 within a predetermined distance for each imaging device 31, the other imaging device 31 is brute-forced for each imaging device 31 and the predetermined distance is reached. It is determined whether or not it is within.
At this time, the distance between any two imaging devices 31 is Distance, the current position (latitude, longitude) of one imaging device 31 is LatA, LonA, and the current position of the other imaging device 31 is LatB, LonB. Toki,
Distance ² = {(LatA−LatB) ² + (LonA−LonB) ² }
Obtained by performing an operation based on. The actual distance is proportional to the value obtained by the above equation. As the value of the “predetermined distance” serving as a discrimination criterion, a value corresponding to the value obtained by this equation is set in advance. Alternatively, as the value of the “predetermined distance”, a value corresponding to the actual distance is set, and a predetermined coefficient is given to the value obtained by the above equation so that appropriate discrimination can be performed.

  The imaging device 31 in this case also acquires the notification from the server device 40 as described above as information that replaces the human detection information as the position analysis result. That is, when there is a notification from the server device 40, the imaging device 31 (system controller 2) obtains a determination result indicating that it is in a state of meeting a person, and an index derived from position analysis. Execute the process for registering points. If there is no notification, a determination result indicating that the person is not in a state of meeting with a person is obtained, and index points derived from position analysis are not registered.

FIG. 11 is a flowchart showing a control example to be performed by the server device 40 in order to realize the operation as the third embodiment described above.
Note that FIG. 11 also shows a control process executed by the control unit (control unit 21) of the server device 40 based on a program stored in an internal ROM or the like.
Also, the processing operation shown in this figure is repeatedly executed at a predetermined cycle.

  In FIG. 11, in step S301, current position information from each device is acquired. In step S302, it is determined for each device whether there is another device within a predetermined distance. That is, for each imaging device 31, for example, the distance between the imaging device 31 and another individual imaging device 31 is calculated by the method exemplified above, and based on the result, is there another imaging device 31 within a predetermined distance? Determine whether or not.

In the next step S303, it is determined whether or not there is a device with another device within a predetermined distance. As a result of the determination for each imaging device 31 in step S302, if an affirmative result is obtained assuming that there is another imaging device 31 within a predetermined distance, the process proceeds to step S304. Notification to the device to be performed. That is, the control of the communication unit 24 is performed such that a notification indicating that is given to each of the imaging devices 31 for which there is another imaging device 31 within a predetermined distance. When the process of step S304 is executed, the processing operation shown in FIG.
On the other hand, if a negative result is obtained in step S303 that there is no image pickup apparatus 31 that is assumed to have another image pickup apparatus 31 within a predetermined distance, the process operation shown in FIG. Exit.

[Modification of Third Embodiment]
Here, in the above description, it is assumed that the determination processing by the server device 40 is performed in real time in parallel with the automatic imaging in each imaging device 31, but the determination by the server device 40 is performed afterwards. It can also be.
Specifically, in each imaging device 31 in that case, the process of periodically acquiring the current position information is performed. However, instead of immediately transmitting the acquired current position information to the server device 40, each The current position information obtained at the time is accumulated in predetermined storage means such as the storage unit 8. Then, the information stored in this way is transmitted to the server device 40 based on an instruction from the server device 40 side.

In the server device 40 in this case, based on the result of acquiring the information accumulated in each imaging device 31 as described above, is there another imaging device 31 within a predetermined distance for each imaging device 31 at each time point? Determine whether or not.
Here, based on the accumulated information acquired from each imaging device 31 as described above, the server device 40 in this case generates an accumulated information database as shown in FIG.
In FIG. 12, the accumulated information database is information in which each device is associated with the information on the current position acquired from the device with each time (time) tm as a reference. In addition, what is necessary is just to produce | generate this storage information database in the memory | storage part 23, for example.

  In this case, the server device 40 determines for each imaging device 31 whether or not there is another imaging device 31 within a predetermined distance based on the accumulated information database as shown in FIG. To do. That is, as a result of performing this determination, the server device 40 obtains information indicating a determination result as to whether or not the other imaging device 31 for each imaging device 31 is within a predetermined distance at each time point tm.

At this time, if it is determined that there is another imaging device 31 within a predetermined distance as a result of the determination made at a certain time tmn, the time tmn is a point to be indexed for the imaging device 31. become.
In the server device 40 in this case, as a result of determining whether or not there is another device within a predetermined distance for each imaging device 31 at a certain time tmn as described above, the corresponding imaging at that time tmn is performed. When the apparatus 31 is present, the time point tmn is registered as an index point to be notified to the corresponding imaging apparatus 31.
As a result of repeatedly performing such processing at each time point tm, the server device 40 generates registration information as shown in FIG. That is, for each device (imaging device 31), information in which information on a time point tm representing a point to be indexed is registered (associated) is generated.

  Based on the index point registration information for each imaging device 31 generated in this way, the server device 40 performs the corresponding imaging device 31 (that is, the imaging device 31 associated with the time point tm as an index point). Information on the time point tm indicating the index point is notified.

  The imaging device 31 in this case attaches an index derived from position analysis to the captured image data stored by automatic imaging based on the information on the time point tm notified from the server device 40 in this way. Specifically, the system controller 2 in this case collates the storage time (shooting time) of each stored captured image data with the time point tm from the server device 40, and the shooting time information that is supposed to match the time point tm. The information content of the index point management information 8b is updated so that information indicating that there is an index derived from position analysis is associated with the ID of the captured image data associated with.

  FIG. 14 is a flowchart showing an example of control to be performed by the server device 40 in order to realize the operation as a modification of the third embodiment. 14 also shows a control process executed by the control unit (control unit 21) of the server device 40 based on a program stored in an internal ROM or the like.

First, in step S401, a process start trigger is awaited. In other words, for example, a predetermined condition determined to start the process shown in FIG.
If a start trigger is generated, a process for acquiring accumulated information of each device is executed in step S402. That is, the storage information (each current position information and its acquisition time) is instructed to be transmitted to each imaging device 31, and the storage information transmitted in response to the transmission instruction is transmitted to a required storage unit such as the storage unit 23, for example. Control processing to be stored.

  In a succeeding step S403, a storage information database generation process is executed. That is, as shown in FIG. 12, information is generated by associating each device with the information on the current position acquired from the device on the basis of each time (time) tm.

  In the next step S404, first, processing for setting the time point identification value n to 1 is executed. The time point identification value n is a value that the control unit 21 counts to identify each time point tm, as will be understood from the following description.

In a succeeding step S405, a process of acquiring data at the time point tmn is executed. That is, a process of acquiring data (current position information and device ID information) associated with the time point tmn in the stored information database is executed.
Then, in the next step S406, for each device, it is determined whether or not there is another device within a predetermined distance. In the next step S407, whether or not there is a device with another device within the predetermined distance. Is determined.

If it is determined in step S407 that there is another device within a predetermined distance from the result of determination for each device at the time point tmn in step S406, an affirmative result is obtained in step S408. Then, the process of registering the time point tmn as the index point of the corresponding device is executed. That is, in the registration information shown in FIG. 13, the process of updating the content of the registration information is performed so that the information of the time point tmn is associated with the ID of the corresponding device as an index point.
When the process of step S408 is executed, the process proceeds to step S409.
On the other hand, if it is determined in step S407 that there is no other device within a predetermined distance from the result of determination for each device at the time tmn, a negative result is obtained. The process proceeds to step S409 after passing S408.

  In step S409, it is determined whether n = N. Here, the value of “N” represents the number of times tm (number of current position information) included in the accumulated information acquired from each imaging device 31. The numerical value “N” can be acquired, for example, in the process of generating the stored information database in the previous step S403. Alternatively, if the acquisition cycle of the current position information in each imaging device 31 is fixed, the calculation can be performed based on the information on the cycle and the previous time information when the accumulated information is acquired from each imaging device 31. .

In step S409, if n = N is not satisfied, that is, if a negative result is obtained that all the time points tm are not yet determined, the process proceeds to step S410, and after incrementing the value of n (n = n + 1), The process returns to the previous step S405.
On the other hand, in step S409, if n = N, that is, if a positive result is obtained as a result of discrimination for all time points tm, the process proceeds to step S411.

In step S411, processing for notifying the corresponding device of an index point is executed according to the registration information. That is, in the registration information generated with the registration processing in the previous step S408, the imaging device 31 associated with the time point tm as the index point is notified of the information of the time point tm as the index point. The communication unit 24 is controlled.
When the process of step S411 is executed, the processing operation shown in FIG.

  In the third embodiment, in the above description, the case where the position detection unit included in each imaging device 31 is a GPS unit is exemplified. However, instead of this, provided by Wi-Fi or a mobile phone company A position detecting means using a position information service may be provided.

Further, in the third embodiment, when an actual system is assumed, the server device 40 may cause a huge processing burden when performing a brute force calculation for each imaging device 31. For example, when there are n imaging devices 31, the distance calculation is performed 1 / 2n (n-1). Therefore, in practice, a plurality of server devices 40 may be provided, and at least the distance calculation may be shared. In that case, it is effective to group the imaging devices 31 that each server device 40 handles in advance. For example, when the method as the third embodiment as described with reference to FIG. 10 and FIG. 11 is adopted, a plurality of server devices 40 to which the acquired current position information is to be transmitted for each imaging device 31. It may be set in advance from among the server devices 40.
Or when the method as a modification of 3rd Embodiment is taken, what is necessary is just to set the imaging device 31 which should acquire storage information for every server apparatus 40 previously.

In particular, when the technique as a modified example of the third embodiment is adopted, the operation as the server apparatus 40 can be performed by the imaging apparatus 31 selected in advance by some technique. In this case, the selected imaging device 31 acquires accumulated information from the other imaging devices 31 in a state where the imaging device 31 is not performing automatic imaging, such as during a time when the user does not go out, such as at night. Then, the same processing as that of the server device 40 described above may be performed to notify each imaging device 31 of the index point. For the sake of confirmation, the selected imaging device 31 generates a stored information database including its own stored information. In addition, the selected imaging device 31 performs indexing for the image data based on the index points registered for itself.

<Modification>

Although the embodiments of the present invention have been described above, the present invention is not limited to the specific examples described so far, and various modifications can be considered.
For example, in the description so far, the case where the determination as to whether or not the person is in a state of meeting with a person is performed is based on the results of image analysis, sound analysis, and analysis of the positional relationship between the imaging devices. In addition, for example, the detection can be performed using a detection signal from an infrared sensor such as a pyroelectric type. In that case, it may be determined whether or not the person is in a state of meeting with a person based on the result of determining whether or not the presence of a person is detected from the detection signal of the infrared sensor.

In the image analysis, the condition that the detection time length of the data portion as a human face is equal to or longer than a predetermined length is given, but other conditions can of course be given.
For example, a data portion that can be recognized as a human face in order to prevent detection of a small face in the image (that is, a person who is far from the user and cannot be estimated to meet the user) It is also possible to impose conditions such as that the image size is equal to or larger than a predetermined size. Specifically, in this case, the image analysis unit 10 detects a data portion that can be recognized as a human face, and the data portion that can be recognized as a human face when the image size of the data portion is equal to or larger than the predetermined size. What is necessary is just to supply the person detection information to the effect of having detected to the system controller 2. FIG. Of course, the condition of the detection time length can also be added in this case.

In particular, in the first embodiment, communication can be performed when determining whether or not a device is in a state of meeting a person by determining whether or not there is a device capable of wireless communication. It can also be added on condition that the device is the imaging device 1. That is, when a communicable device is detected and the device can be identified as the imaging device 1, a determination result is obtained that it is in a state that it is assumed that a person is met.
According to this, for example, when it is assumed that there are many devices capable of wireless communication around the imaging apparatus, an index is added unnecessarily due to the presence of external devices that can communicate with the outside. Prevention can be achieved.

Further, in the description so far, the determination result as to whether or not the person is in a state of being in contact with a person is within a capturing period of captured image data by automatic imaging (that is, if the automatic imaging period is 5 seconds). The explanation was made on the premise that it can be obtained within 5 seconds). However, for example, in the case of adding to the condition that the face detection time length is not less than a predetermined length as exemplified in the embodiment as an image analysis. In other words, there may be a case where the timing at which the determination result is obtained is not in time within one capture period.
For example, in response to such a case, it is determined in which capture period the determination as to whether or not the person is said to be meeting a person is performed. You should keep it. Thereby, even if it is a case where a discrimination | determination result is not in time for the corresponding capture period, the discrimination | determination result for every captured image can be managed appropriately.

Further, in the description so far, as an index, an example in which a plurality of types are assigned separately for each discrimination item has been illustrated, but for example, an index may be provided with only one discrimination item with only one discrimination item. it can.
Alternatively, as in the case of the specific examples described so far, after making a determination based on a plurality of determination items, as a result, a positive result is obtained for a plurality of predetermined determination items. It is also possible to obtain a final (final) discrimination result and finally attach only one type of index. Specifically, for example, when a positive result is obtained in both the discrimination processing based on image analysis and the discrimination processing based on position analysis, or both of the discrimination processing based on speech analysis and the discrimination processing based on position analysis If a positive result is obtained, a final discrimination result that it is assumed that the person is in contact with a person is obtained, and when the discrimination result is obtained, only one kind of index is attached. Is.
Here, if a comprehensive determination result is obtained from the determination results of a plurality of items as described above, it is possible to more accurately determine whether or not the person is in a state of meeting with a person. it can.

In addition, the index manages the point to which the index is attached by using separate management information (index point management information 8c). However, the index is stored in the header of the image data (eg, in the file). It is also possible to attach it by embedding it in ().
However, if the management is performed using separate management information as exemplified in the embodiment, the process of scanning the image data is not necessary for searching the index, and accordingly, a high-speed search can be performed.

In the embodiments, the description has been made on the assumption that the image data to be captured by automatic imaging is still image imaging. However, the present invention can also be applied to an imaging apparatus that performs moving image imaging.
For example, an operation method may be considered in which moving image capturing is not performed continuously at all times, but is performed intermittently for a predetermined time. In such a case, it is only necessary to determine whether or not each captured image data to be captured is in a state where it is considered to meet a person, and to assign an index based on the determination result.

  Here, when moving image data is assumed as captured image data, it is conceivable to add a continuous index. That is, in this case, it is continuously determined whether or not it is in a state where it is assumed that a person is met, and when a positive result is obtained by the determination, an ON point of the marker as an index is set. Then, as long as a state in which an affirmative result is obtained by such continuous determination continues, the marker is kept on, and an OFF point of the marker is set in response to obtaining a negative result. Thereby, a continuous marker (index) can be attached in the time series direction corresponding to the moving image.

In the description so far, the specific use of the index has not been mentioned, but for example, the following use can be considered.
For example, it may be used for searching for images during browsing. Alternatively, when displaying a thumbnail image in the case of displaying a list of representative images (representative scenes) of a stored image that are picked up at the time of browsing, it is conceivable to select an image on which the thumbnail image should be displayed using an index. It is done.
In addition, particularly when a moving image is assumed, there may be a use such as placing a chapter at an index point.

It is explanatory drawing of the example of an external appearance of the imaging device as embodiment of this invention. It is the block diagram which showed the example of an internal structure of the imaging device of embodiment. It is a figure for demonstrating the discrimination | determination method about whether it is in the state assumed to meet the person as 1st Embodiment. As a diagram for explaining a specific example of index assignment for captured image data, it is a diagram schematically showing the relationship between captured image data captured and stored by automatic imaging and an index attached thereto is there. It is a flowchart for demonstrating the example of control for implement | achieving the operation | movement as 1st Embodiment. It is the figure which showed typically about the system configuration | structure as 2nd Embodiment, and the operation | movement performed by this system. It is the block diagram which showed the example of the internal structure of the server apparatus as 2nd Embodiment. It is a flowchart for demonstrating the example of control for implement | achieving the operation | movement as 2nd Embodiment. It is the block diagram which showed the internal structural example of the imaging device as 3rd Embodiment. It is the figure which showed typically the system configuration | structure as 3rd Embodiment, and the operation | movement performed by this system. It is a flowchart for demonstrating the example of control for implement | achieving the operation | movement as 3rd Embodiment. It is the figure which showed the structural example of the accumulation | storage information database produced | generated with the system as a modification of 3rd Embodiment. It is the figure which showed the structural example of the registration information about the index point produced | generated with the system as a modification of 3rd Embodiment. It is a flowchart for demonstrating the example of control for implement | achieving the operation | movement as a modification of 3rd Embodiment.

Explanation of symbols

    1,31 imaging device, 2 system controller, 3 imaging unit, 3a imaging optical system, 3b imaging element unit, 3c imaging signal processing unit, 4 imaging control unit, 5 display unit, 6 display control unit, 7 operation input unit, 8 Storage unit, 9 communication unit, 10 image analysis unit, 11 audio analysis unit, 12 bus, 13 audio input unit, 14 GPS unit, 20, 40 server device, 21 control unit, 22 timekeeping unit, 23 storage unit, 24 communication Department, 30 base stations, 32 networks

Claims (12)

Imaging means for capturing an image of a subject and obtaining captured image data;
With control means,
The control means includes
A captured image automatic capture process that automatically captures captured image data obtained by the imaging means without the user's operation;
A determination process for determining whether or not it is in a state where it can be estimated that a person is met based on input information;
Indexing for controlling the captured image data captured by the captured image automatic capturing process to be indexed when it is determined that the person can be estimated to meet the person through the determining process. Process and execute,
An imaging apparatus characterized by that. In the determination process by the control means,
Based on the image analysis result of the captured image data obtained by the imaging means, it is determined whether or not it is in a state that can be estimated to meet with a person,
The imaging apparatus according to claim 1. In the above discrimination process,
Based on the image analysis result about whether or not an image that can be estimated to be a human face is included in the captured image data, it is determined whether or not it is a state that can be estimated to meet a person,
The imaging apparatus according to claim 2. In the determination process by the control means,
It is determined whether it is in a state where it can be estimated that the person is meeting with a person based on the sound analysis result of the sound collection data obtained by collecting the external sound,
The imaging apparatus according to claim 1. A storage means for storing predetermined voiceprint data;
In the above discrimination process,
It is determined whether or not it is in a state where it can be estimated that a person is met, based on a voice analysis result as to whether or not there is a data portion that matches the voiceprint data in the collected sound data.
The imaging apparatus according to claim 4. With short-range wireless communication means for performing short-range wireless communication with an external device,
In the determination process by the control means,
Based on the determination result as to whether there is another device in the communicable range by the short-range wireless communication means, it is determined whether it is in a state that can be estimated to meet a person,
The imaging apparatus according to claim 1. In the determination process by the control means,
Whether or not it is possible to presume meeting with a person by determining whether or not a plurality of predetermined conditions that are presumed to be meeting with a person are met To obtain the final discrimination result of
The imaging apparatus according to claim 1. An image index assigning method in an imaging apparatus that automatically captures captured image data obtained by capturing an image of an object without the user's operation,
A determination procedure for determining whether or not it is in a state where it can be estimated that a person is met based on input information,
When it is determined that it is in a state where it can be estimated that the person meets the person according to the determination procedure, an indexing procedure for controlling the captured image data that is automatically captured to be indexed;
An image indexing method characterized by comprising: An image indexing system including an information processing device and a plurality of imaging devices configured to be capable of data communication between the information processing device,
The information processing apparatus
Position information acquisition processing for performing data communication to acquire information on the location of the imaging device;
A search process for searching for an imaging apparatus in which another imaging apparatus exists within a predetermined range based on information on the location of the imaging apparatus acquired by the position information acquisition process;
As a result of the search by the search process, the information processing apparatus-side control means for executing a notification process for notifying an imaging apparatus specified as having another imaging apparatus within the predetermined range,
The imaging apparatus is
Imaging means for capturing an image of a subject and obtaining captured image data;
With the imaging device side control means,
The imaging device side control means includes:
A captured image automatic capture process that automatically captures captured image data obtained by the imaging means without the user's operation;
Based on the notification by the notification process on the information processing apparatus side, an indexing process is performed to control the captured image data automatically captured by the captured image automatic capture process to be indexed. To
An image indexing system characterized by that. The imaging device is configured to perform data communication with the information processing device via a wireless communication repeater,
In the position information acquisition process by the information processing apparatus side control means,
Obtaining information on the location of the imaging device based on area information identified from the wireless communication repeater that relays communication data about the imaging device;
The image indexing system according to claim 9. The imaging apparatus is
In addition to providing position detection means for detecting the current position,
In the position information acquisition process by the information processing apparatus side control means,
Obtaining information on the current position detected by the imaging device as information on the location;
The image indexing system according to claim 9. An information processing apparatus comprising position detection means for detecting a current position and capable of data communication with a plurality of image pickup apparatuses that are automatically captured image data not based on a user's operation Because
Position information acquisition processing for acquiring current position information detected by the position detection means in the imaging device;
Based on the position information acquired by the position information acquisition process, a search process for searching for an image pickup apparatus in which another image pickup apparatus exists within a predetermined range;
As a result of the search by the search process, a control unit is provided that performs a notification process for notifying an imaging device that is identified as having another imaging device within the predetermined range.
An information processing apparatus characterized by that.

Images