JP2009055080A - Imaging apparatus, and imaging method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform automatic imaging in appropriate operation setting in accordance with whether a person is around. <P>SOLUTION: Whether a person is around is discriminated, for example, by a pyroelectric infrared sensor, a picked-up image analysis or the like. Image quality (compression rate and picture size) and the setting of an imaging interval are changed in accordance with the result of the discrimination. For example, when a person is around, high definition picked-up image data are stored, so that an image valuable to a user may be recorded with high definition, and when a person is not around, low definition (small capacity) picked-up image data are stored, so that a recording capacity can be saved about an image being not so much valuable to the user. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and an imaging method.

JP 2006-270627 A JP 2007-13843 A

  For example, a camera that records a scene that the user sees in daily life as image data by automatically taking a periodic image with a camera worn by the user, such as a life log camera, has been proposed. By using this life log camera, it becomes possible to leave a user's action history, memories, etc. as image data.

  As a conventional technique, the above-described patent document discloses a technique for starting a photographing operation based on the detected amount of infrared rays and the sound analysis result of a collected sound signal.

As a life log camera as described above, for example, a small imaging device is attached to a spectacle-type or head-mounted mounting unit, and automatic imaging is performed at regular intervals, for example, and the captured image data is stored automatically However, from this point of view, if a user wears a life log camera and performs daily actions, images may be captured and stored even in the absence of nearby people. It is done.
Here, in general, it is predicted that events that are important to people and memorable events will occur more frequently in the presence of people nearby. Therefore, when a captured image is saved under the situation where there is no person nearby as described above, an image that is not so valuable to the user is saved. Furthermore, an unfavorable situation in which the data recording capacity of the recording medium storing the captured image is wasted correspondingly.

  On the other hand, when an apparatus that performs automatic imaging as described above is assumed, it may be considered that protection of the privacy of the person being imaged is important. In other words, especially when automatic imaging is performed, there is a possibility that a scene that significantly impairs the privacy of the person to be imaged may be captured and stored due to imaging at a timing not intended by the user, and such a situation is to be prevented. You can also think about it.

The present invention has been made in view of the above points, and an object of the present invention is to perform more appropriate image capturing when performing automatic image capturing for life log use, for example.
Therefore, in the present invention, the imaging apparatus is configured as follows.
That is, the image processing apparatus includes an imaging unit that captures an image of a subject, obtains captured image data, and stores the captured image data. The determination unit determines whether there is a person nearby based on input information. On the basis of the result, a control unit configured to perform an imaging operation setting related to an imaging operation by the imaging unit is provided.

Further, the control unit performs image quality setting, image size setting, imaging interval time setting, or frame rate setting as the imaging operation setting.
In addition, when the determination unit obtains a determination result that there is a person nearby, the control unit performs control so that the superior setting is performed as the imaging operation setting. Is obtained, the non-dominant setting is performed as the imaging operation setting.
The superior setting is an image quality setting that provides higher image quality than the image setting of the non-dominant setting, an image size setting that is larger than the image size setting of the non-dominant setting, or imaging of the non-dominant setting. For example, an imaging interval time setting in which the imaging interval time is shorter than the interval setting, or a frame rate setting in which the frame rate is set higher than the frame rate setting of the non-dominant setting.

  According to the present invention, it is possible to perform the imaging operation setting for the imaging operation for storing the captured image data based on the determination result of whether or not there is a person near the user who carries the imaging device. In other words, for example, the image quality, image size, imaging interval, and the like of the captured image data can be set based on the determination result of whether or not there is a person nearby.

As described above, according to the present invention, the imaging operation settings such as the image quality, the image size, and the imaging interval of the captured image data are determined based on the determination result of whether or not there is a person near the user carrying the imaging device. Can be done.
For example, as the imaging operation setting, the setting of the imaging interval is an element that affects the number of stored captured images. For example, when it is determined that there is no person nearby, it is determined that there is a person nearby. If the setting is made so that the imaging interval is longer than that, it is possible to effectively avoid a situation where a large number of images that are not so important to the user are stored.
In addition, as the imaging operation setting, the imaging interval, the image quality of the captured image data, the image size, and the like are elements (items) that affect the data recording capacity of the captured image data. For example, it is determined that there is no person nearby. If the setting of these items is set so that the data recording capacity decreases in comparison with the case where it is determined that there is a person nearby, it is recorded with a captured image that is not so important to the user. It is possible to effectively avoid a situation where the data recording capacity of the medium is wasted.

  Alternatively, as the imaging operation setting, in particular, the image quality setting, the image size setting, and the like are items that affect the particular ease / difficulty of the person projected in the image. Therefore, for example, when it is determined that there is a person nearby, if the imaging operation setting is performed in a direction that makes it difficult to specify the person in the image, the privacy of the person to be imaged can be protected. it can.

  In this way, according to the present invention, it is possible to perform an appropriate imaging operation according to the presence or absence of a nearby person.

Embodiments of the present invention will be described below. The description will be made in the following order.
[1. Appearance example of imaging device]
[2. Configuration example of imaging apparatus]
[3. Imaging operation setting example I]
[4. Imaging operation setting example II]
[5. Imaging operation setting example III]
[6. Imaging operation setting example IV]
[7. Effects and modifications of the embodiment]

[1. Appearance example of imaging device]

Various forms are assumed as the imaging apparatus 1 of the embodiment, and examples of their appearance are illustrated in FIGS.
FIG. 1A shows a neck-mounted type imaging apparatus 1. The imaging apparatus 1 has a part to which a strap is attached, for example, and is attached by attaching the strap to this part and hanging it on the user's neck as shown in the figure. The user only needs to wear the imaging lens 3L included in the imaging device 1 so that the imaging can be performed with the front direction of the user as the subject direction.
Although not shown, for example, a display unit used for an imaging monitor or reproduction of a captured image may be provided on the back surface of the imaging device 1 or the like.

FIG. 1B shows an imaging apparatus 1 that is a glasses-type display camera. The imaging device 1 has a mounting unit having a frame structure that makes a half turn, for example, from both sides of the head to the back of the head, and is mounted on the user by being placed on both ear shells as illustrated.
In the imaging device 1, the imaging lens 3 </ b> L is arranged forward so that imaging is performed with the user's visual field direction as the subject direction when the user is wearing the imaging device 1.
Further, in the wearing state as shown in the figure, a pair of display units 5 and 5 for the left eye and the right eye are arranged immediately before both eyes of the user, that is, at a position where a lens in normal glasses is located. Has been. For example, a liquid crystal panel is used for the display unit 5. By controlling the transmittance, a through state as shown in the figure, that is, a transparent or translucent state can be obtained. Since the display unit 5 is in the through state, there is no problem in normal life even if the user always wears it like glasses.
In addition to a pair of display units 5 corresponding to both eyes, a configuration in which one display unit 5 is provided corresponding to one eye is also conceivable. A configuration in which the display unit 5 is not provided is also conceivable.

In FIGS. 1A and 1B, the neck-mounted type or the eyeglass-type imaging device 1 is exemplified, but various structures for the user to wear the imaging device 1 can be considered. For example, a headphone type, a neckband type, an ear hook type, or any other mounting unit may be worn by the user. Furthermore, for example, a configuration may be adopted in which the user attaches to normal glasses, a visor, headphones, or the like with an attachment such as a clip. Moreover, it does not necessarily have to be worn on the user's head.
In addition, in the case of FIG. 1A, the imaging direction is the front direction of the user, but the imaging device 1 may be worn on the neck so as to capture the back of the user when worn.
In the case of FIG. 1B, the imaging direction is the user's field of view, but the imaging lens 3 </ b> L is attached so as to capture the user's rear, side, upper, foot direction, and the like when worn. Alternatively, a configuration in which a plurality of imaging systems having the same or different imaging directions is provided is also conceivable.
Further, in FIGS. 1A and 1B, an imaging direction variable mechanism that can change the subject direction manually or automatically may be provided for one or a plurality of imaging lenses 3L.

Needless to say, forms other than those shown in FIGS. 1A and 1B are also conceivable as an imaging apparatus that captures moving images and still images. For example, an imaging apparatus 1 that can be installed in a vehicle and that can image inside or outside the vehicle can be assumed as the present embodiment. For example, it is an imaging device attached to take an image of the interior of a vehicle or an imaging device attached to take an image of a front landscape, a rear landscape, or the like of a car.
Further, an apparatus such as a mobile phone, a PDA (Personal Digital Assistant), and a portable personal computer that has a function as an imaging apparatus can be assumed as the imaging apparatus 1 of the present embodiment.
In these various forms, for example, a microphone that collects external sound may be provided, and an audio signal to be recorded together with image data may be obtained during imaging. Further, a speaker unit and an earphone unit that perform audio output may be formed.
It is also conceivable that a light emitting unit that performs illumination in the direction of the subject is provided in the vicinity of the imaging lens 3L using, for example, an LED (Light Emitting Diode) or a flash light emitting unit for taking a still image.

[2. Configuration example of imaging apparatus]

Next, a configuration example of the imaging device 1 according to the embodiment will be described.
FIG. 2 is a block diagram illustrating an internal configuration of the imaging apparatus 1.
As illustrated, the imaging apparatus 1 includes a system controller 2, an imaging unit 3, an imaging control unit 4, a display unit 5, a display control unit 6, an operation input unit 7, a storage unit 8, a communication unit 9, an image analysis unit 10, The voice analysis unit 11, the bus 12, the voice input unit 13, and the human sensor unit 14 are included.

  The system controller 2 includes, for example, a microcomputer including a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), a nonvolatile memory unit, and an interface unit, and controls the entire imaging apparatus 1. The control unit The system controller 2 performs various arithmetic processes and exchanges control signals with each unit via the bus 12 based on a program held in an internal ROM or the like, and causes each unit to execute a required operation.

The imaging unit 3 includes an imaging optical system 3a, an imaging element unit 3b, and an imaging signal processing unit 3c.
The imaging optical system 3a in the imaging unit 3 includes a lens system including the imaging lens 3L illustrated in FIG. 1, a diaphragm, a zoom lens, a focus lens, and the like, and performs a focus operation and a zoom operation on the lens system. A drive system or the like for performing the operation is provided.
The imaging device unit 3b in the imaging unit 3 is provided with a solid-state imaging device array that detects imaging light obtained by the imaging optical system 3a and generates an imaging signal by performing photoelectric conversion. The solid-state imaging device array is, for example, a CCD (Charge Coupled Device) sensor array or a CMOS (Complementary Metal Oxide Semiconductor) sensor array.
The imaging signal processing unit 3c in the imaging unit 3 includes a sample hold / AGC (Automatic Gain Control) circuit that performs gain adjustment and waveform shaping on a signal obtained by the solid-state imaging device, and a video A / D converter, and includes digital data. Obtained image data. Also, white balance processing, luminance processing, color signal processing, and the like are performed on the captured image data.

Imaging is performed by the imaging unit 3 including the imaging optical system 3a, the imaging element unit 3b, and the imaging signal processing unit 3c, and captured image data is obtained.
Image data obtained by the operation of the imaging unit 3 is processed by the imaging control unit 4.
The imaging control unit 4 performs processing such as image compression processing for compressing captured image data at various compression ratios, image size conversion processing, and image format conversion processing according to the control of the system controller 2, and according to the operation situation. The captured image data is transferred to the storage unit 8, the display control unit 6, the communication unit 9, and the like.
Further, the imaging control unit 4 controls on / off of imaging in the imaging unit 3, shutter processing, drive control of the zoom lens and focus lens of the imaging optical system 3a, sensitivity of the imaging element unit 3b, based on an instruction from the system controller 2. Control of the frame rate, parameter control of each process of the imaging signal processing unit 3c, execution process setting, and the like are also performed.

A display unit 5 and a display control unit 6 are provided as a configuration for displaying to the user in the imaging apparatus 1.
The display unit 5 is provided with a display panel unit such as a liquid crystal display and a display driving unit for driving the display panel unit. The display driving unit includes a pixel driving circuit for displaying the image data supplied from the imaging control unit 4 on the display panel unit. The pixel drive circuit applies a drive signal based on the video signal to each pixel arranged in a matrix in the display panel unit at a predetermined horizontal / vertical drive timing, and causes display to be performed.

The display control unit 6 drives the pixel driving circuit in the display unit 5 based on the control of the system controller 2 to execute a predetermined display. For example, monitor display of an image captured by the imaging unit 3 and display of an image reproduced by the storage unit 8 are executed.
For these displays, for example, brightness level adjustment, color correction, contrast adjustment, sharpness (outline emphasis) adjustment, and the like can be performed. Also, generation of an enlarged image obtained by enlarging a part of the image data, generation of a reduced image, soft focus, mosaic, luminance inversion, highlight display (highlight display) of a part of the image, change in atmosphere of the whole color, etc. Image effect processing can also be performed.

The operation input unit 7 has operation elements such as keys, buttons, and dials. For example, an operation element for power on / off operation, an operation related to automatic imaging, a required input operation, and the like. An operator is formed. When not only automatic imaging but also imaging in accordance with the user's shutter operation is possible, for example, an operator used for shutter operation, zoom operation, exposure setting operation, self-timer operation, etc. It may be formed.
The operation input unit 7 supplies information obtained from such an operator to the system controller 2, and the system controller 2 performs necessary arithmetic processing and control corresponding to the information.

The human sensor unit 14 obtains a detection signal for determining whether or not there is a person nearby. For example, in the case of the embodiment, an infrared sensor such as a pyroelectric type is used as the human sensor unit 14.
Here, as an imaging operation setting as an embodiment, a case where it is simply determined whether there is a person (unspecified person) nearby, as in setting example I and setting example II described later, As in setting example IV, it can be roughly divided into a case where it is determined whether or not there is a specific person nearby.
In the case of this example using an infrared sensor as the human sensor unit 14, based on the detection signal (human detection information) from the human sensor unit 14, it is merely determined whether or not there is a person nearby. Specifically, the infrared detection information from the human sensor unit 14 in this case is supplied to the system controller 2, and the system controller 2 determines whether there is a person nearby based on the infrared detection information.

  Here, when an infrared sensor is used as the human sensor unit 14, the degree of “near” in determining whether or not a person is near is determined by setting the threshold for the infrared detection amount set in the system controller 2 or the sensitivity of the infrared sensor. Can be adjusted.

The image analysis unit 10 performs image analysis processing for determining whether or not there is a person nearby based on the captured image data supplied from the imaging control unit 4.
Image analysis processing by the image analysis unit 10 includes both image analysis processing for simply determining whether or not a person is nearby and image analysis processing for determining whether or not a specific person is nearby. Is possible.
For example, when it is determined whether or not there is a person nearby, the image analysis unit 10 performs image analysis to detect a data portion that can be recognized as a human face in the captured image data. As a result of this analysis, when a data portion that can be recognized as a human face is detected in the captured image data, detection information (person detection information) to that effect is supplied to the system controller 2. On the other hand, when a data portion that can be recognized as a human face is not detected in the captured image data, human detection information indicating that is supplied to the system controller 2. The system controller 2 determines whether or not a person is nearby based on the person detection information supplied from the image analysis unit 10 in this way.

When the image analysis unit 10 performs image analysis processing for determining whether or not a specific person is nearby, a face database storing face data for each individual is stored, for example, as described later. It is made to store in the part 8. FIG.
Here, as already known, the face of a person can be converted into feature data as relative position information of components of the face. For example, the ratio of the distance EN between the center of the eye and the nose to the distance Ed between the eyes (Ed / EN), or the ratio between the distance EM between the center of the eye and the mouth and the distance Ed between the eyes (Ed / EM). Is information that is unique to each individual and that is not affected by changes in appearance due to a hairstyle, glasses, or the like. Furthermore, it is known that it does not change with aging.
In view of this point, the face database is information that stores facial feature data (also referred to as face data) such as Ed / EN and Ed / EM as described above for each individual. Then, the image analysis unit 10 first detects a data portion that can be recognized as a human face from the captured image data by the above-described image analysis, and if there is a data portion that can be recognized as a face, Image analysis is performed to obtain the face data. Then, the acquired face data is compared with each face data in the face database, and it is determined whether there is face data matching the acquired face data in the face database. The image analysis unit 10 supplies information representing the determination result to the system controller 2 as specific person analysis information.
The system controller 2 determines whether or not there is a specific person nearby based on the specific person analysis information. That is, when the specific person analysis information indicates that there is face data that matches the acquired face data in the face database, a determination result that there is a specific person nearby is obtained, and the acquired in the face database When it is indicated that there is no face data matching the face data, a determination result that there is no specific person nearby is obtained.

  Here, when performing an analysis using such an image analysis unit 10, the degree of “near” whether or not there is a person (including a specific person) in the vicinity is detected as a data portion that can be recognized as a human face. It can be adjusted by setting the threshold value of the detection range at the time. That is, it is necessary to set a larger detection range threshold when detecting a data portion that is a human face and to have a larger face in the image in order to be detected as a human face data portion. By doing so, the degree of “near” can be increased. On the other hand, if the threshold value of the detection range when detecting the data portion that is a human face is set small so that even a small face in the image is detected as a human face data portion, The degree of “near” can be lowered.

The voice analysis unit 11 performs voice analysis processing on whether or not there is a person nearby based on the microphone sound collection data input from the voice input unit 13.
The speech analysis processing by the speech analysis unit 11 includes both speech analysis processing for simply determining whether or not there is a person nearby and speech analysis processing for determining whether or not a specific person is nearby. Is possible.
For example, when it is simply determined whether or not there is a person nearby, feature data common to the voice of the person is set in advance for the voice analysis unit 11. That is, the voice analysis unit 11 in this case performs a voice analysis process for detecting a data portion that matches the characteristic data about the human voice set in this manner from the collected sound data input from the voice input unit 13. As a result, when a data portion that matches the feature data is detected in the collected sound data, detection information (person detection information) indicating that fact is supplied to the system controller 2, and the feature data is included in the collected sound data. If the data portion that matches is not detected, human detection information indicating that fact is supplied to the system controller 2.
The system controller 2 determines whether or not there is a person nearby based on the person detection information from the voice analysis unit 11. That is, when the person detection information indicates that a data portion matching the feature data is detected in the sound collection data, a determination result that there is a person nearby is obtained, and the person detection information is included in the sound collection data. If it is shown that no data portion matching the feature data is detected, a determination result that there is no person nearby is obtained.

If the voice analysis unit 11 performs voice analysis processing for determining whether or not a specific person is nearby, for example, a voice print database storing voice print data for each individual is stored as described later. It is made to store in the part 8. FIG.
In this case, the voice analysis unit 11 sequentially performs a process of comparing the collected sound data input from the voice input unit 13 with each voice print data in such a voice print database, It is determined whether or not a data portion that matches the voiceprint data is obtained. The voice analysis unit 11 supplies information representing the determination result to the system controller 2 as specific person analysis information.
The system controller 2 determines whether there is a specific person nearby based on the specific person analysis information. That is, when the specific person analysis information indicates that a data portion matching the voice print data in the voice print database is obtained in the sound pickup data, a determination result that there is a specific person nearby is obtained, and the sound pickup data If it indicates that no data portion matching the voiceprint data in the voiceprint database is obtained, a determination result that there is no specific person nearby is obtained.

  Here, when performing an analysis using such a voice analysis unit 11, the degree of “near” whether or not a person (including a specific person) is nearby is a data portion that matches the feature data and voiceprint data. When it is determined that the threshold value is set, a threshold value for the sound volume is also provided, and can be adjusted by setting the sound volume threshold value. That is, the degree of “near” can be increased by increasing the volume threshold, and the degree of “near” can be decreased by decreasing the volume threshold.

The storage unit 8 is used for storing captured image data and other various data.
The storage unit 8 may be configured by a solid-state memory such as a flash memory, or may be configured by an HDD (Hard Disk Drive), for example.
Further, instead of a built-in recording medium, a portable recording medium, for example, a recording / reproducing drive corresponding to a recording medium such as a memory card incorporating a solid-state memory, an optical disk, a magneto-optical disk, or a hologram memory may be used.
Of course, both a built-in type memory such as a solid-state memory and an HDD, and a recording / reproducing drive for a portable recording medium may be mounted.
The storage unit 8 records / reproduces captured image data and other various data based on the control of the system controller 2.

In the storage unit 8 in this case, a captured image data recording area 8a and a personal identification database recording area 8b are prepared as recording areas.
The captured image data recording area 8a is an area for storing captured image data obtained by the imaging unit 3. For example, if imaging is performed at regular time intervals as regular automatic imaging, the captured image data is recorded in the captured image data recording area 8a at regular intervals.

A database for personal identification is recorded in the database recording area 8b for personal identification. This personal identification database is used, for example, when determining whether or not a specific person is nearby in setting examples III and IV described later.
Here, examples of the personal identification database include the face database and voiceprint database described above. 3 and 4 show an example of the data structure of the face database and an example of the data structure of the voiceprint database, respectively. As can be seen from these figures, the face database is a database storing individual face data, and the voice print database is a database storing individual voice print data.

  The captured image data recording area 8a and the personal identification database storage area 8b described above do not need to be set as fixed areas in the storage unit 8. In the present example, the captured image data and the personal identification database need only be stored in the storage unit 8.

The communication unit 9 is provided as a part that performs data communication with various external devices.
For example, data transmission / reception with a server device (not shown) may be performed. In that case, for example, a configuration may be adopted in which network communication is performed via short-range wireless communication with a network access point using a method such as wireless LAN or Bluetooth, or direct wireless communication with a server device having a corresponding communication function. It may be what performs.
In addition, the communication unit 9 may be connected to a device such as a personal computer using an interface such as a USB (Universal Serial Bus) system to transmit and receive data.
The communication unit 9 can transfer, for example, captured image data captured and stored in the storage unit 8 to a personal computer or other external device. Accordingly, a large number of captured image data recorded and stored in the storage unit 8 by periodic imaging as a life log can be reproduced and displayed on the display unit 5 by the imaging device 1 itself, and an external device such as a personal computer. Can be transferred to and played back on the external device.

Here, as understood from the above description, the imaging apparatus 1 according to the embodiment has a function of determining whether there is a person nearby or whether there is a specific person nearby. However, for confirmation, the configuration for realizing the function of determining whether or not there is a person nearby, and determining whether or not a specific person is nearby, as exemplified in the embodiment below, is organized. Keep it.

~ Determination of whether there are people nearby ~
A method of providing an infrared sensor as the human sensor unit 14 and using the detected information.
A method using image analysis by the image analysis unit 10.
A method using voice analysis by the voice analysis unit 11.

~ Determination of whether there is a specific person nearby ~
A method of using image analysis using a face database by the image analysis unit 10.
A method of using voice analysis using a voice print database by the voice analysis unit 11.

Although the configuration of the imaging device 1 has been described above, this is only an example. Addition and deletion of various components are naturally conceivable depending on the operation examples and functions actually implemented.

[3. Imaging operation setting example I]

A characteristic operation example in the imaging apparatus 1 of the above embodiment will be described below.
The imaging device 1 according to the embodiment basically performs an imaging operation that is not based on a user's shutter operation when used for a life log application. For example, automatic imaging is periodically performed, and captured image data is stored in the storage unit 8. For example, automatic imaging is performed at predetermined time intervals such as an interval of 5 seconds, an interval of 10 seconds, and an interval of 30 seconds. Of course, instead of periodic imaging, automatic imaging may be performed at irregular timing according to some trigger (trigger other than the user's shutter operation). Furthermore, if the user performs a shutter operation during execution of regular or irregular automatic imaging, the imaging process may be performed normally at that time.

  When the user wears the imaging device 1 in daily life and thus automatically performs periodic imaging in this manner, for example, a scene corresponding to the user's action is captured. However, when performing such automatic imaging, it is conceivable that the captured image is stored under the condition that there is no person nearby as described above. Generally speaking, events that are important to people or memorable events are expected to occur more frequently in situations where there are people nearby, so imaging in situations where there are no people nearby If the image is stored, an image that is not so valuable to the user is stored. If an image having no value is stored in this manner, the data recording capacity of the recording medium is wasted.

  Therefore, in this example, it is determined whether or not there is a person nearby, and in a situation where there is a person nearby, imaging is performed with a relatively superior setting as an imaging operation setting, while on the other hand, there is no person nearby. In such a situation, imaging is performed with a non-dominant setting.

Here, as the imaging operation setting, an image quality setting, an image size setting, or an imaging interval setting can be considered.
The image quality setting is the image quality of the captured image data, which can be set to a high or low compression rate, for example. That is, the captured image data obtained by the imaging unit 3 is compressed by the imaging control unit 4 by a predetermined compression method, transferred to the storage unit 8, and recorded in the captured image data recording area 8a. In this case, the higher the compression ratio, the smaller the data capacity, but the quality of the stored captured image data decreases. On the other hand, if the compression rate is low (or if it is not compressed), the data capacity increases, but the quality of the stored captured image data increases.
Therefore, it is possible to change the quality of image data to be stored as a setting of the compression rate, and to effectively use the storage capacity.

The image size setting can be considered as, for example, the number of pixels of the captured image data. For example, the image size corresponding to the number of pixels in the image sensor unit 3b is maximized, and the number of pixels is decreased by, for example, pixel thinning processing. The image size becomes smaller.
Again, the larger the image size, the higher the quality but the larger the capacity. On the other hand, the smaller the image size, the lower the image quality but the smaller the capacity.

  The imaging interval setting is a time setting as an imaging interval when periodic automatic imaging is performed as described above. If the imaging interval is shortened, the number of stored captured image data per unit time increases, which is suitable as a life log, but requires a large amount of recording capacity as a whole. On the other hand, if the imaging interval is increased, the number of captured image data is reduced, but the overall recording capacity is saved. For example, a setting in which the imaging interval is switched between an interval of 10 seconds and an interval of 30 seconds can be considered.

In these imaging operation settings, the above-described superior setting means a setting for reducing the compression rate, a setting for increasing the image size, or a setting for shortening the imaging interval in this example.
On the other hand, the non-dominant setting is a setting for increasing the compression rate, a setting for reducing the image size, or a setting for increasing the imaging interval.

Based on this, as described above, when there is a person nearby, imaging is performed with a superior setting, and when there is no person nearby, imaging is performed with a non-dominant setting. According to the operation of the example, many captured images are stored when a person is not nearby, and the recording capacity of the recording medium is wasted due to the storage of captured images when a person is not nearby. It is possible to prevent such a situation.
On the other hand, when people are nearby, imaging is performed with superior settings, so images that are valuable to the user are superior in terms of higher image quality, larger image size, more images, etc. You can save effectively by setting.

At this time, when using human detection information by the image analysis unit 10 to determine whether or not a person is nearby, imaging is performed in a scene where the user wearing the imaging device 1 is facing the person. Images can be saved effectively.
Further, in the case of determining whether or not a person is nearby based on the person detection information by the voice analysis unit 11, a captured image in a scene where the user wearing the imaging device 1 is interacting with the person is effectively used. Can be saved.

  Here, a control example for realizing the imaging operation setting as described above will be described with reference to the following flowchart of FIG. Here, a case where the imaging operation setting is an image size, a compression rate, and an imaging interval is illustrated. FIG. 5 shows control processing executed by the system controller 2 based on a program stored in the internal ROM.

First, in step S101, human detection information is acquired. That is, the human detection information from the infrared sensor as the human sensor unit 14 described above, the human detection information from the image analysis unit 10, or the human detection information from the voice analysis unit 11 is acquired.
In subsequent step S102, it is determined whether or not there is a person nearby. That is, based on the human detection information from the infrared sensor as the human sensor unit 14, the human detection information from the image analysis unit 10, or the human detection information acquired from the voice analysis unit 11, it is close as described above. Determine if there are people.

  If an affirmative result is obtained in step S102 that there is a person in the vicinity, the process proceeds to step S103, and the imaging control unit 4 is set to [Image size: large, compression rate: low, imaging interval: short] as the imaging operation setting. Instruct. As a result, the imaging control unit 4 performs the imaging operation setting according to the above-described content instructed from the system controller 2.

  On the other hand, if a negative result is obtained in step S102 that there is no person nearby, the process proceeds to step S104, and the imaging control unit sets [image size: small, compression rate: high, imaging interval: long] as the imaging operation setting. 4 is instructed. As a result, the imaging control unit 4 performs non-dominant imaging operation settings as compared with the case where a person is near.

When the processes in steps S103 and S104 are executed, it is determined in step S105 whether an end trigger has occurred. That is, for example, whether a predetermined trigger that has been determined in advance to end automatic imaging has occurred, such as an operation input for turning off the power of the imaging apparatus 1 from the operation input unit 7 or an operation input that instructs to stop automatic imaging. The process which discriminates about is performed.
If a negative result is obtained in step S105 that the end trigger has not occurred, the process returns to the previous step S101, and according to the determination result of whether there is a person nearby until the end trigger is generated. The processing for performing the imaging operation setting is repeatedly performed.
If a positive result is obtained in step S105 that an end trigger has occurred, the processing operation shown in FIG.

In the embodiment, as a method for determining whether or not there is a person nearby, only one of the methods listed above may be used, or a comprehensive result from the determination results by a plurality of methods. It is also possible to obtain a discrimination result. For example, when obtaining a comprehensive discrimination result from discrimination results obtained by a plurality of methods, if a positive result is obtained by discrimination processing by a plurality of methods, a comprehensive discrimination result that a person is near is obtained. do it.
As described above, if a comprehensive discrimination result is obtained from the discrimination results obtained by a plurality of methods, it is possible to more accurately determine whether there is a person nearby.

[4. Imaging operation setting example II]

In the above-described imaging operation setting example I, the imaging operation setting is advantageous when there is a person nearby, and the imaging operation setting is advantageous when the person is not nearby, so there is no value for the user. However, as described above, the problem with automatic imaging is that the subject Can also mention privacy issues.
Therefore, as the imaging operation setting example II, the imaging operation setting based on the determination result of whether or not there is a person nearby is performed as follows.

FIG. 6 is a flowchart showing the processing operation of the system controller 2 to be performed in order to realize the operation as the imaging operation setting example II. In FIG. 6, parts having the same processing contents as those already described in FIG. 5 are given the same step numbers and description thereof is omitted.
As shown in the figure, the processing operation in this case is obtained by replacing the processing operation executed in accordance with the result of the determination processing in step S102 with respect to the processing operation in FIG. That is, if a positive result is obtained in step S102 that there is a person nearby, the process proceeds to step S104, and [image size: small, compression rate: high, imaging interval: long] is instructed as the imaging operation setting. On the other hand, if an affirmative result is obtained in step S102 that there is no person nearby, the process proceeds to step S103 to instruct [image size: large, compression rate: low, imaging interval: short] as the imaging operation setting.
In other words, in contrast to the case of setting example I above, when there is a person nearby, a non-dominant imaging operation setting is performed compared to when there is no person nearby. .

According to such setting example II, when it is determined that there is a person nearby, the setting of image size: small and compression ratio: high is performed as the imaging operation setting, so that the person in the image is specified. The imaging operation setting can be performed in a direction that makes it difficult. Thereby, the privacy of the person to be imaged can be protected.

[5. Imaging operation setting example III]

In the imaging operation setting example III, imaging operation setting is performed based on a determination result of whether or not a specific person is nearby.
FIG. 7 shows the processing operation of the system controller 2 to be executed to realize the operation as the setting example III. In this figure as well, parts that are the same as those already described in FIG. 5 are given the same reference numerals and explanations thereof are omitted.
As shown in the figure, the processing operation in this case is such that the determination processing of step S202 is executed instead of the determination processing of step S201 and step S102 instead of step S101 in the processing operation of the previous FIG. .

First, in step S201, specific person analysis information is acquired. That is, the specific person analysis information from the image analysis unit 10 described above or the specific person analysis information from the voice analysis unit 11 is acquired.
Then, in the subsequent step S202, it is determined whether or not there is a specific person nearby. That is, based on the specific person analysis information from the image analysis unit 10 or the voice analysis unit 11, it is determined whether or not there is a specific person nearby as described above.

If an affirmative result is obtained in step S202 that there is a person nearby, the process proceeds to step S103 to instruct [image size: large, compression rate: low, imaging interval: short] as the imaging operation setting.
On the other hand, if a negative result is obtained in step S202 that there is no person nearby, the process proceeds to step S104 to instruct [image size: small, compression rate: high, imaging interval: long] as the imaging operation setting.

With this setting example III, captured images in a scene where a specific person is nearby can be stored with superior settings such as a larger image size, higher image quality, and a larger number of images. That is, this makes it possible to more effectively store a captured image in a scene where a specific person is nearby.
At this time, when determining whether or not a specific person is nearby based on the analysis information of the image analysis unit 10, the user wearing the imaging device 1 and the specific person face each other. The captured image can be effectively saved.
Further, when determining whether or not a specific person is nearby based on the analysis information of the voice analysis unit 11, a captured image in a scene where a user wearing the imaging apparatus 1 and a specific person are interacting with each other. Can be saved effectively.

  As setting example III, as in the case of setting example I, an advantageous setting is set when there is a person nearby, and a non-dominating setting is set when there is no person. As in setting example II, when a specific person is nearby, non-dominant settings (for example, image size: small, compression rate: high, imaging interval: long) are performed, and when a specific person is not nearby It is also possible to perform various settings (for example, image size: large, compression rate: low, imaging interval: short). By doing so, it is possible to protect the privacy of a specific person.

In the embodiment, as a method for determining whether or not there is a specific person nearby, only one of the above-described methods may be used, or the determination results obtained by a plurality of methods may be used. It is also possible to obtain a comprehensive discrimination result. For example, in this case as well, if it is determined that a comprehensive discrimination result is obtained from the discrimination results obtained by a plurality of methods, if a positive result is obtained by the discrimination processing by a plurality of methods, it is assumed that a specific person is nearby. What is necessary is just to obtain a typical discrimination result.
As described above, if a comprehensive determination result is obtained from the determination results obtained by a plurality of methods, it is possible to more accurately determine whether there is a specific person nearby.

[6. Imaging operation setting example IV]

Imaging operation setting example IV is a combination of determining whether there is a person nearby and determining whether there is a specific person nearby. If there is no nearby person, there is a person nearby but a specific person In the case where the specific person is nearby, the case classification is performed, and the imaging operation setting corresponding to each case is performed.

  FIG. 8 is a flowchart showing the processing operation of the system controller 2 to be executed in order to realize the operation as the setting example IV. In this figure, parts that are the same as the processing contents already described so far are assigned the same step numbers and description thereof is omitted.

  In FIG. 8, in this case, first, the processes of steps S101 and S102 are executed to determine whether or not there is a person nearby based on the person detection information. If a negative result is obtained because there is no person nearby, the process proceeds to step S104, and [image size: small, compression rate: high, imaging interval: long] is instructed as an imaging operation setting.

  On the other hand, if a positive result is obtained in step S102 that there is a person nearby, the process proceeds to step S201 to acquire specific person analysis information, and then in step S202, it is determined whether or not the person is a specific person. . For confirmation, the processing content of the system controller 2 in step S202 itself is the discrimination processing based on the specific person analysis information from the image analysis unit 10 or the voice analysis unit 11, so that The processing content is the same as the processing in step S202 described.

If a negative result is obtained in step S202 that the person is not a specific person, the process proceeds to step S301, and the image capturing control unit 4 sets [image size: medium, compression ratio: medium, image capturing interval: medium] as the image capturing operation setting. To instruct.
On the other hand, if an affirmative result is obtained in step S202 that the person is a specific person, the process proceeds to step S103, and the imaging control unit 4 sets [image size: large, compression rate: low, imaging interval: short] as the imaging operation setting. To instruct.

  Also in this case, after executing the process for instructing the imaging operation setting, the process proceeds to step S105 to determine whether or not an end trigger has occurred. If a negative result is obtained, the process returns to step S101 to obtain an affirmative result. If it does, the processing operation ends.

With this setting example IV, when there is no person nearby, there are people nearby but not a specific person, or when there is a specific person nearby, the imaging operation settings Can be changed step by step. Specifically, according to the above example, when a nearby person is a specific person, a larger image size / higher image quality, a larger number of sheets, etc. than when a nearby person is not a specific person, The captured image can be effectively saved by a more advantageous setting.
Also in this case, when there is no person nearby, a non-dominant setting is performed, so that a large amount of images that are not very valuable for the user are stored, and the recording capacity of the recording medium is wasted It is possible to prevent such a situation that it is consumed.

As setting example IV, the most advantageous setting is made when a specific person is nearby, but conversely, the most advantageous setting is made when no person is nearby. You can also. Specifically, step S103 and step S104 in FIG. 8 are interchanged. By doing in this way, the privacy of a specific person can be protected more firmly.

[7. Effects and modifications of the embodiment]

According to the present embodiment described above, the imaging operation setting can be adaptively switched according to the presence or absence of a nearby person. As a result, for example, a problem that a large number of images that are not very valuable to the user are recorded, a problem that the recording capacity of the recording medium is wasted with an image that is not very valuable to the user, or an imaging is performed. It is possible to obtain an effect such as protecting the privacy of the person shown in the image.
In this way, according to the present embodiment, for example, an operation for performing automatic imaging as a life log can be an appropriate operation according to the presence or absence of a nearby person.

Here, for confirmation, the essence of the present invention is to perform an imaging operation setting according to whether or not a person is nearby. Thus, the imaging operation can be made an appropriate operation according to the presence or absence of a nearby person.
Accordingly, as specific items of the imaging operation setting and the setting contents, it is sufficient to select an appropriate one according to the purpose to be achieved depending on whether a person is nearby or not. It should not be limited to the specific items and setting contents exemplified in.

Although the embodiments of the present invention have been described above, the present invention is not limited to the embodiments described so far, and various modifications can be considered.
For example, although the embodiments have been described from the viewpoint of still image capturing, the present invention can also be applied to image capturing apparatuses that perform moving image capturing.
For example, when considering an imaging device that continuously captures moving images, the frame rate, image size, and compression rate are sequentially changed when there is a person nearby or not, when there is a specific person nearby, or near When there is no person / a person nearby but a specific person / a specific person nearby, the setting can be changed according to each.

  In addition to continuously capturing moving images, for example, an operation method in which moving images are captured intermittently for a certain period of time can be considered. In such a case, the frame rate, image size, compression rate, imaging interval, etc. Can be changed according to each case described above.

Furthermore, it is also conceivable to switch between still image capturing / moving image capturing with moving image capturing being a superior setting over still image capturing.
For example, in the operation example, when there is no person nearby, still image capturing is performed, and when there is a person nearby, continuous or intermittent moving image capturing is performed as a superior setting.

Further, in the embodiment, only the case where an infrared sensor is used as the human sensor unit 14 is illustrated, but other than this, for example, an odor sensor can be used. When the odor sensor is used, it is possible to both determine whether or not there is a person nearby and whether or not there is a specific person nearby.
Here, examples of the odor sensor include a quartz vibration gas sensor having a sensor element in which a sensitive film such as lipid is coated on the surface of a quartz diaphragm. Such an odor sensor is provided with a plurality of sensor elements each having sensitivity to different odors, and the odor type is determined based on the odor characteristic characterized by the detected amount of each odor obtained from each sensor element. Can be used. By adopting such a method, it is possible to perform more accurate determination than when performing determination based on the detection amount of one kind of odor by one odor sensor element.

  When simply determining whether a person is nearby using an odor sensor, odor characteristic data (hereinafter referred to as odor information) common to the person is stored in the system controller 2 in advance. The system controller 2 determines whether or not the odor detection information obtained as the person detection information from the odor sensor matches the odor information stored in advance as described above. If they match, a determination result that there is a person nearby is obtained, and if they do not match, a determination result that a person is not nearby is obtained.

Further, when it is determined whether there is a specific person nearby using an odor sensor, an odor database storing odor information for each individual is used. For example, such an odor database can be recorded in the storage unit 8 or the like.
In this case, the system controller 2 collates the odor detection information from the odor sensor with each odor information in the odor database, and determines whether there is odor information that matches the odor detection information in the odor database. As a result, if there is odor information that matches the odor detection information in the odor database, a discrimination result that there is a specific person nearby is obtained, and conversely, odor information that matches the odor detection information in the odor database If there is no, a determination result is obtained that there is no specific person nearby.

When performing odor determination using an odor sensor, the stability of odor determination may be reduced due to various odors mixed into the sensor depending on the environment in which the device is used. Conceivable. For example, examples of the odor discrimination method taking this point into account include a multivariate analysis method and a method using a neural network. For example, in multivariate analysis, by conducting experiments in advance and measuring what kind of odor the sensor element's detected quantity characteristics can be obtained in various cases (environments), the detection target For each odor, a characteristic of the odor detection amount of each sensor element that is characteristic in each case is calculated. In the determination, a determination is made as to the coincidence with the characteristics thus determined.
Also, in the case of discrimination using a neural network, the neural network is made to learn the characteristics of the detection amount of each sensor element when various odors to be detected are input to the odor sensor in advance. The actual neural network is mounted on the device to make an actual discrimination.

  Here, when such an odor sensor is used as the human sensor unit 14, the degree of “near” in determining whether there is a person (including a specific person) nearby is the amount of odor detection set in the system controller 2. Can be adjusted by setting the threshold value and the sensitivity of the odor sensor.

Further, when determining whether or not a specific person is nearby, in addition to performing analysis by the image analysis unit 10 and the sound analysis unit 11 exemplified in the embodiment, other imaging devices 1 capable of wireless communication can be used. A technique using the assigned personal ID can also be adopted. Specifically, this method is based on the premise that each individual is carrying and using the imaging device 1 assigned with an ID for each individual. When another imaging device 1 capable of data communication is detected, the personal ID of the other imaging device 1 capable of communication is acquired, and is the acquired personal ID registered in the personal ID database in advance? By determining whether or not there is a specific person in the vicinity, it is determined.
When this method is realized, a personal ID database storing a personal ID assigned in advance for each imaging device 1 (that is, for each individual) is recorded in advance in, for example, the storage unit 8 or the like.

In this case, the communication unit 9 performs a process of detecting another imaging device 1 capable of wireless communication. When another imaging device 1 capable of wireless communication is detected, the communication unit 9 notifies the system controller 2 to that effect. Do.
The system controller 2 obtains a personal ID from the other imaging device 1 in response to the notification that the other imaging device 1 capable of wireless communication is detected from the communication unit 9 as described above. I do. Then, the personal ID acquired in this way is compared with each personal ID in the personal ID database to determine whether there is a personal ID in the personal ID database that matches the acquired personal ID. If there is a personal ID that matches the personal ID acquired from the other imaging device 1 in the personal ID database, a determination result that there is a specific person nearby is obtained, and the acquisition is performed in the personal ID database. If there is no personal ID that matches the personal ID, a determination result is obtained that there is no specific person nearby.

  When the method using the personal ID of another imaging device 1 capable of wireless communication in this way is used, the adjustment of the degree of “near” is, for example, a threshold for the intensity of communication radio waves from the other imaging device 1. It can be provided by setting the threshold value. That is, the degree of “near” can be increased by setting the threshold value large so that it is not determined that there is a specific person nearby unless the field intensity is stronger. Further, the degree of “near” can be reduced by setting the threshold value small and determining that there is a specific person nearby even when the radio field intensity is weak.

Alternatively, whether or not there is a specific person in the vicinity is determined based on the result of collating the ID of the wireless tag, for example, on the premise that a wireless tag assigned with unique ID information in advance is attached to the individual. You can also. That is, in this case, the imaging apparatus 1 is provided with a reading unit for reading the wireless tag, and a database storing the ID for each wireless tag is prepared. When the ID from the wireless tag is read by the reading unit, the ID is compared with the ID in the database to determine whether there is a matching ID. This makes it possible to determine whether or not a specific person is nearby.
Note that the adjustment of the degree of “near” in this case can also be performed by providing a threshold for the radio wave intensity, for example. In other words, the degree of “near” can be increased by setting a large threshold for the radio wave intensity so that it is not determined that there is a specific person nearby unless the radio wave intensity is stronger. Further, the degree of “near” can be reduced by setting the threshold value small and determining that there is a specific person nearby even when the field intensity is weak.

  Further, in the description so far, an example in which a database for personal identification to be referred to when determining whether or not a specific person is nearby (whether or not a specific person) is recorded in the imaging apparatus 1 is illustrated. However, the personal identification database may be stored in an external device such as a server device capable of data communication via the communication unit 9. In this case, the collation process may be performed by the external device. In that case, the imaging device 1 side uses the face data acquired from the captured image data, or the sound collection data, the odor information, the personal ID, and the wireless tag ID. The data is transferred to the external device and collated with the corresponding personal identification database. Then, the collation result may be acquired from the external device, and it may be determined whether there is a specific person nearby based on the collation result.

It is explanatory drawing of the example of an external appearance of the imaging device of this invention. It is a block diagram of the imaging device of an embodiment. It is explanatory drawing of the data structure example of the face database of embodiment. It is explanatory drawing of the example of a data structure of the voiceprint database of embodiment. 10 is a flowchart of an imaging operation setting example I according to the embodiment. It is a flowchart of imaging operation setting example II of an embodiment. It is a flowchart of imaging operation setting example III of an embodiment. It is a flowchart of imaging operation setting example IV of an embodiment.

Explanation of symbols

    DESCRIPTION OF SYMBOLS 1 Imaging device, 2 System controller, 3 Imaging part, 3a Imaging optical system, 3b Imaging element part, 3c Imaging signal processing part, 4 Imaging control part, 5 Display part, 6 Display control part, 7 Operation input part, 8 Storage part , 9 communication unit, 10 image analysis unit, 11 voice analysis unit, 12 bus, 13 voice input unit, 14 human sensor unit

Claims (12)

Imaging means for imaging a subject to obtain captured image data, and storing the captured image data;
A discriminating means for discriminating whether or not there is a person nearby based on the input information;
A control unit configured to perform an imaging operation setting related to an imaging operation by the imaging unit based on a determination result by the determination unit;
An imaging apparatus comprising:   The imaging apparatus according to claim 1, wherein the control unit performs image quality setting, image size setting, imaging interval time setting, or frame rate setting as the imaging operation setting. When the determination means obtains a determination result that there is a person nearby, the control means performs control so that the superior setting is performed as the imaging operation setting, and on the other hand, there is no person nearby. When the determination result is obtained, non-dominant setting is performed as the imaging operation setting.
The imaging apparatus according to claim 1. The superior setting is an image quality setting that provides a higher image quality than the image setting of the non-dominant setting, an image size setting that is larger than the image size setting of the non-dominant setting, or an imaging interval setting of the non-dominant setting. An imaging interval time setting in which the imaging interval time is shorter than the above, or a frame rate setting that is higher than the frame rate setting of the non-dominant setting,
The imaging apparatus according to claim 3. The determination means determines whether or not a person is nearby based on the detection result of the input infrared information.
The imaging apparatus according to claim 1. The determining means determines whether or not a person is nearby based on the result of image analysis of captured image data obtained by capturing an image of a subject.
The imaging apparatus according to claim 1. The determination means determines whether or not a person is nearby based on the result of performing sound analysis on sound collection data obtained by collecting external sound,
The imaging apparatus according to claim 1. The determination means determines whether there is a specific person nearby based on the input information and pre-stored person specifying information.
The imaging apparatus according to claim 1. The determination means determines whether or not there is a specific person nearby based on the result of image analysis based on the captured image data obtained by imaging the subject and the face specifying data for each person.
The imaging apparatus according to claim 8. The determination means determines whether or not there is a specific person nearby based on a result of voice analysis based on sound collection data obtained by collecting external sound and voice print data for each person,
The imaging apparatus according to claim 8. The determination means is configured to be able to perform both a person presence / absence determination process as to whether or not a person is nearby and a specific person presence / absence determination process as to whether or not a specific person is nearby,
The control means includes
Based on the determination result of the person presence / absence determination process and the specific person presence / absence determination process, when there is no person nearby, when there is a person nearby but is not a specific person, and when there is a specific person nearby Perform the above imaging operation settings according to
The imaging apparatus according to claim 1. As an imaging operation, as an imaging method of an imaging apparatus that obtains captured image data of a subject and performs storage processing of the captured image data,
A determination procedure for determining whether there is a person nearby based on the input information,
A control procedure for performing an imaging operation setting related to the imaging operation based on a determination result by the determination procedure;
An imaging method comprising:

Images