JP2012028836A - Image processing apparatus and imaging apparatus using the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique for supporting the photographing behavior of a photographing person who photographs a moving image.SOLUTION: A first acquisition unit 11 acquires an image from a first imaging unit 210 for imaging a moving image for recording. A second acquisition unit 12 acquires an image from a second imaging unit 220 for imaging the photographing person. On the basis of the image acquired by the second acquisition unit 12, the determination unit 14 determines whether the photographing person is viewing a monitor 230 displaying a moving image for recording. A control unit 20 varies the status of the imaging apparatus 200 in accordance with a determination result obtained by the determination unit 14.

Description

  The present invention relates to an image processing apparatus that encodes a moving image to be captured and an imaging apparatus equipped with the image processing apparatus.

  In recent years, digital movie cameras that allow general users to easily shoot moving images have become widespread, and among them, it is possible to shoot moving images with full HD (high definition) (1920 × 1080 pixels) image quality. There are also things. In addition, digital movie cameras in recent years have become multi-functional, and the user is not readily using these functions.

JP 2009-153085 A

  Especially for users who are unfamiliar with the handling of digital movie cameras and users who are not good at handling electronic devices such as highly electronic cameras in the first place, it is difficult to use the functions installed in digital movie cameras. . For example, even with one camera shake correction function, those users take pictures with the function turned off, and conversely, by always using the function at the maximum intensity, the power consumption is wasted more than necessary.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide a technique for supporting a photographing action of a photographer who is photographing a moving image.

  An image processing apparatus according to an aspect of the present invention is an image processing apparatus to be mounted on an imaging apparatus for capturing a moving image, and obtains an image from a first imaging unit for capturing a moving image for recording. A moving image for recording is displayed based on the image acquired by the first acquisition unit, the second acquisition unit for acquiring an image from the second imaging unit for imaging the photographer, and the second acquisition unit. A determination unit that determines whether or not the photographer is looking at the monitor that is being operated, and a control unit that changes the status of the imaging device according to a determination result by the determination unit.

  Another aspect of the present invention is an imaging apparatus. This apparatus includes a first imaging unit for imaging a recording moving image, a second imaging unit for imaging a photographer, and the above-described image processing apparatus.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, and the like are also effective as an aspect of the present invention.

  According to the present invention, it is possible to support the photographing action of a photographer who photographs a moving image.

1 is a schematic external view of an imaging apparatus according to an embodiment of the present invention. 1 is a configuration diagram of an imaging apparatus equipped with an image processing apparatus according to an embodiment of the present invention. It is a figure which shows the relationship between the frame image input into a branch part, the frame image encoded by the 1st image encoding part, and the frame image encoded by the 2nd image encoding part. It is a figure which shows the example of a switching timing between the single codec mode by HD image quality, and the dual encoding mode by HD image quality and SD image quality. It is a figure which shows the example of a switching timing between the single codec mode by SD image quality, and the dual encoding mode by HD image quality and SD image quality. 6A to 6C are diagrams illustrating an example of camera shake correction by the camera shake correction unit. FIGS. 7A to 7C are diagrams illustrating an example of auto framing processing by the auto framing processing unit.

  FIG. 1 is a schematic external view of an imaging apparatus 200 according to an embodiment of the present invention. In the imaging apparatus 200, a first imaging unit 210, a second imaging unit 220, and a monitor 230 are installed. The first imaging unit 210 is a main camera for capturing a moving image of a subject such as a person, a landscape, a structure (such as a railway vehicle). The second imaging unit 220 is a sub camera for imaging the photographer, more specifically, the eyes of the photographer. A monitor (also referred to as a viewer or a viewfinder) 230 mainly displays a moving image captured by the first imaging unit 210.

  The second imaging unit 220 is not a so-called camera for taking a selfie and recording, but a camera for determining whether or not the photographer is looking at or looking at the monitor 230. Therefore, the image captured by the second imaging unit 220 is basically not recorded and discarded. On the other hand, the first imaging unit 210 is a camera for capturing a moving image for recording.

  The second imaging unit 220 is installed at a position where an image for determining whether or not the photographer is looking at the monitor 230 can be taken. Specifically, it is preferably installed at a position where the photographer's face can be photographed at a normal average holding position by the photographer of the imaging apparatus 200. For example, the second imaging unit 220 may be installed at any position on the periphery of the monitor 230.

  By determining whether or not the photographer is looking at the monitor 230, the photographer's behavior and psychological state can be estimated as follows.

First, when the photographer is looking at the monitor 230, the following estimation is valid.
1. The photographer thinks that he does not want to miss the scene he is currently shooting (does not want to make a mistake).
2. The subject or composition to be targeted is being changed.
3. The zoom magnification is being changed.
4). Moving the camera slightly moves the target off the frame, so it is more focused on shooting than usual.

Next, when the photographer does not look at the monitor 230, the following estimation is valid.
1. I do not intend to shoot.
2. For reasons such as changing the composition, we try to grasp the entire shooting space without going through the camera.
3. Losing the target and looking for the target without looking through the camera.

The embodiments of the present invention will be specifically described below based on these findings.
FIG. 2 is a configuration diagram of an imaging apparatus 200 equipped with the image processing apparatus 100 according to the embodiment of the present invention. The imaging device 200 includes a first imaging unit 210, a second imaging unit 220, a monitor 230, a sound acquisition unit 240, an operation unit 250, and the image processing device 100.

  The first imaging unit 210 will be described more specifically. The first imaging unit 210 continuously acquires frame images and supplies them to the image processing apparatus 100 as moving images. The first imaging unit 210 includes a solid-state image sensor (not shown) such as a charge coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) image sensor, and a signal processing circuit (not shown) that processes a signal output from the solid-state image sensor. Is provided. The signal processing circuit can convert the analog three primary color signals R, G, and B output from the solid-state imaging device into digital luminance signals Y and color difference signals Cr and Cb.

  The second imaging unit 220 is basically the same as the first imaging unit 210, but a lower specification than the first imaging unit 210 can be used. When the above-described determination as to whether or not the photographer is looking at the monitor 230 is executed at regular time intervals (for example, at intervals of 5 seconds), the second imaging unit 220 is not necessarily equipped with a function for capturing a moving image. It is not necessary to have the ability to capture still images at regular intervals. Note that the second imaging unit 220 may have a lower specification than the first imaging unit 210 in terms of resolution and angle of view.

  The sound acquisition unit 240 converts sound acquired from the outside into an electric signal and outputs the sound signal to the image processing apparatus 100 as a sound signal. The operation unit 250 receives a user instruction, generates a control signal based on the instruction, and outputs the control signal to the image processing apparatus 100.

  The image processing apparatus 100 includes a first acquisition unit 11, a second acquisition unit 12, a face detection unit 13, a determination unit 14, a control unit 20, a camera shake correction unit 21, an auto framing processing unit 22, a branching unit 31, a resolution / angle of view. A conversion unit 32, an image encoding unit 40, an audio encoding unit 45, a multiplexing unit 50, a recording unit 51, and an input / output unit 52 are provided. The image encoding unit 40 includes a first image encoding unit 41 and a second image encoding unit 42.

  The configuration of the image processing apparatus 100 can be realized by an arbitrary processor, memory, or other LSI in terms of hardware, and is realized by a program loaded in the memory in terms of software. It depicts the functional blocks that are realized. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The first acquisition unit 11 acquires an image from the first imaging unit 210 and supplies the acquired image to the branch unit 31. The second acquisition unit 12 acquires an image from the second imaging unit 220 and supplies the acquired image to the face detection unit 13.

  The face detection unit 13 detects a human face from the frame image captured by the second imaging unit 220. The face detection unit 13 can use an existing general face detection technique. For example, the face image can be detected from the frame image by using a discriminator for identifying the face image. In the present embodiment, since it is necessary to determine whether or not the photographer is looking at the monitor 230, the face detection unit 13 scans whether or not there is a face image facing the monitor 230. The scanning result is supplied to the determination unit 14.

  Based on the image acquired by the second acquisition unit 12, the determination unit 14 determines whether or not the photographer is looking at the monitor 230 displaying the recording moving image acquired by the first acquisition unit. judge. Specifically, the scan result of the face image is acquired from the face detection unit 13, and when a face image facing the monitor 230 is detected, it is determined that the photographer is looking at the monitor 230, and the monitor 230 If the face image facing the camera is not detected, it is determined that the photographer is not looking at the monitor 230.

  Since the face detection unit 13 repeatedly performs scanning while enlarging and reducing the frame image, the size of the face image in the frame image is also detected. Therefore, the distance between the photographer and the monitor 230 can be estimated based on the size of the face image. Further, if a discriminator capable of discriminating the seriousness of a facial expression is used, it is possible to discriminate whether a photographer looking at the monitor 230 is serious or relaxed. In addition, if a discriminator generated by photographing a specific person is used, it can be determined whether or not the person is looking at the monitor 230. In this case, even if another person is looking at the monitor 230, it is determined that the person is not looking.

  The control unit 20 changes the status of the imaging apparatus 200 according to the determination result by the determination unit 14. A specific example of this status change will be described later.

  The branching unit 31 outputs the moving image supplied from the first acquisition unit 11 to the first image encoding unit 41 or outputs it to the resolution / view angle conversion unit 32 according to the instruction by the control signal from the control unit 20. Or output to both.

  The resolution / view angle conversion unit 32 can convert the resolution and / or the view angle of the frame image constituting the moving image input from the branch unit 31. For example, the resolution of the frame image can be reduced, or the central area of the frame image can be cut out and the peripheral area can be deleted to narrow the angle of view. Also, the pixels in the frame image may be thinned to reduce the resolution and narrow the angle of view. The resolution / view angle conversion unit 32 outputs a moving image composed of frame images after the resolution and / or the view angle are converted to the second image encoding unit 42.

  Note that a super-resolution processor (not shown) may be provided between the resolution / view angle converter 32 and the second image encoder 42. The super-resolution processing unit may improve the resolution of the frame image whose angle of view is adjusted narrowly by the resolution / view angle conversion unit 32 by super-resolution processing, and restore the original frame image size. As a result, the size of the frame image whose angle of view is adjusted narrowly to enlarge the region of interest can be restored to the original size at the original resolution.

  The image encoding unit 40 can encode the moving image captured by the first imaging unit 210 in parallel or simultaneously with both the first image quality and the second image quality different from the first image quality. That is, one type of moving image can be dual-encoded. In FIG. 2, the first image encoding unit 41 receives the moving image input from the branching unit 31, and the second image encoding unit 42 converts the moving image input from the resolution / view angle converting unit 32 in parallel or simultaneously. Can be encoded.

  The first image quality moving image and the second image quality moving image are encoded with different resolutions and / or angles of view. For example, various combinations are possible for the combination of the resolution of the first image quality moving image and the resolution of the second image quality moving image. For example, any two combinations of a 1920 × 1080 pixel size, a 1280 × 720 pixel size, a 640 × 480 pixel size, a 448 × 336 pixel size, and a 192 × 108 pixel size may be used.

  The first image quality moving image and the second image quality moving image may be encoded at different resolutions and / or angles of view and at different frame rates. For example, any two combinations of 60 fps, 30 fps, and 15 fps may be used. In the case of a low resolution such as a 448 × 336 pixel size or a 192 × 108 pixel size, a high frame rate such as 240 fps or 600 fps can be assigned.

  The image encoding unit 40 compression-encodes the first image quality moving image and the second image quality moving image according to a predetermined standard. For example, H.M. H.264 / AVC, H.H. H.264 / SVC, MPEG-2, or MPEG-4 can be used for compression encoding.

  The image encoding unit 40 may encode the first image quality moving image and the second image quality moving image in a time-division manner by software processing by one hardware encoder or general-purpose processor, or two hardware components. The first image quality moving image and the second image quality moving image may be encoded in parallel by an encoder. The image encoding unit 40 outputs the encoded data of the first image quality moving image (also referred to as an encoded stream) and the encoded image data of the second image quality moving image to the multiplexing unit 50.

  The voice encoding unit 45 encodes the voice signal supplied from the voice acquisition unit 240. For example, compression encoding is performed according to standards such as AAC and MP3. The audio encoding unit 45 outputs the encoded audio encoded data to the multiplexing unit 50.

  The multiplexing unit 50 encodes the encoded data of the first image quality moving image input from the first image encoding unit 41 and the second image quality moving image input from the second image encoding unit 42. The data and the encoded audio data input from the audio encoding unit 45 are multiplexed to generate one moving image file. For example, a container file can be generated according to the MP4 file format. The container file can include a container describing header information, metadata, time information, and the like of each encoded data. By referring to the container file on the decoding side, synchronization between the first image quality moving image, the second image quality moving image and the sound can be easily performed, and random access can be easily performed.

  The recording unit 51 records the moving image file multiplexed by the multiplexing unit 50 on a predetermined recording medium. As the recording medium, at least one of a built-in memory and a detachable removable memory can be employed. For example, a semiconductor memory or a hard disk can be adopted as the built-in memory. In addition, a memory card, a removable hard disk, or an optical disk can be adopted as the removable memory.

  The input / output unit 52 communicates with an external device via a predetermined interface. For example, a moving image file recorded on the recording medium can be transferred to a PC or an external hard disk by connecting to a PC or an external hard disk with a USB cable. In addition, the first image quality moving image and the second image quality moving image can be displayed on a television screen by connecting to a television with a D terminal, an S terminal, or an HDMI terminal.

  FIG. 3 shows the relationship between the frame image F1 input to the branching unit 31, the frame image F2 encoded by the first image encoding unit 41, and the frame image F3 encoded by the second image encoding unit 42. FIG. In FIG. 3, the first image quality moving image is a HD (1280 × 720 pixel) size frame image, and the second image quality moving image is an SD (640 × 480 pixel) size frame image. An example of a moving image composed of In this example, an HD size frame image F1 is input to the branching unit 31.

  The branching unit 31 outputs the HD size frame image F1 to the first image encoding unit 41 and the resolution / view angle conversion unit 32. The resolution / view angle conversion unit 32 converts the HD size frame image F1 into an SD size frame image F3. The first image encoding unit 41 encodes the HD size frame image F1 input from the branching unit 31 as it is. The second image encoding unit 42 encodes the SD-size frame image F3 input from the resolution / view angle conversion unit 32.

  The aspect ratio of the HD size frame image F2 encoded by the first image encoding unit 41 is 16: 9, and the aspect ratio of the SD size frame image F3 encoded by the second image encoding unit 42 is. The ratio is 4: 3. The SD size frame image F3 is generated by leaving the central area of the HD size frame image F2 and deleting the peripheral area.

  Hereinafter, a specific example of the status change by the control unit 20 will be described. In the first specific example, on / off of the dual encoding function is controlled. That is, as a result of the determination by the determination unit 14, the control unit 20 uses a plurality of moving image encoded data with different image quality from the moving image acquired by the first acquisition unit 11 when the photographer is looking at the monitor 230. When the photographer is not looking at the monitor 230, control is performed to generate one type of moving image encoded data from the moving image acquired by the first acquisition unit 11.

  As described above, the image encoding unit 40 may encode the moving image acquired by the first acquisition unit 11 with both the first image quality and the second image quality lower than the first image quality. It is also possible to encode one of the second image quality. Using this function, when the photographer is looking at the monitor 230 as a result of determination by the determination unit 14, the control unit 20 uses the first image quality and the second image quality from the moving image acquired by the first acquisition unit 11. Two types of moving image encoded data are generated with the image quality, and one type of moving image encoded data with the second image quality is obtained from the moving image acquired by the first acquisition unit 11 when the photographer is not looking at the monitor 230. Can be controlled to generate. Further, when the photographer is not looking at the monitor 230, it may be controlled to generate one type of moving image encoded data with the first image quality from the moving image acquired by the first acquisition unit 11.

  In the same manner as in FIG. 3, the first image quality moving image is composed of a frame image of HD (1280 × 720 pixels) size, and the second image quality moving image is SD (640 × 480 pixels) size. A case where a moving image is composed of the frame images will be described as an example.

  FIG. 4 is a diagram illustrating an example of switching timing between a single codec mode based on HD image quality and a dual encoding mode based on HD image quality and SD image quality. In this example, a moving image of HD image quality is encoded during the entire shooting period, and the period during which the determination unit 14 determines that the photographer is looking at the monitor 230 (hereinafter referred to as an attention period) out of the entire shooting period. Both HD quality video and SD quality video are encoded. That is, the image encoding unit 40 continuously encodes a captured moving image with HD image quality and intermittently encodes with SD image quality.

  The main purpose of this example is to save high-quality moving images for viewing on a PC or TV, and send some of the attention scenes attached to e-mails or post them to sites on the Internet. It is suitable for the purpose of following.

  In FIG. 4, two attention periods are set during the entire photographing period. Shooting is started at the shooting start timing Ts0, and single codec with HD image quality is started. Subsequently, dual encoding with HD image quality and SD image quality is started at the first attention period start timing Ts1. Subsequently, at the first attention period end timing Te1, dual encoding with HD image quality and SD image quality ends, and single codec with HD image quality is started. Subsequently, dual encoding with HD image quality and SD image quality is started at the second attention period start timing Ts2. Subsequently, at the second attention period end timing Te2, dual encoding by HD image quality and SD image quality ends, and single codec by HD image quality is started. Finally, shooting is ended at the shooting end timing Te0, and the single codec with HD image quality is ended.

  FIG. 5 is a diagram illustrating an example of switching timing between a single codec mode based on SD image quality and a dual encoding mode based on HD image quality and SD image quality. In this example, an SD image quality moving image is encoded in the entire shooting period, and both the HD image quality moving image and the SD image quality moving image are encoded in the attention period in the entire shooting period. That is, the image encoding unit 40 continuously encodes the captured moving image with SD image quality and intermittently encodes with HD image quality.

  The main purpose of this example is to send a full-length captured video as an e-mail attachment or post it to a site on the Internet, and save some of the scenes of interest for viewing on a PC or TV It is suitable for the purpose of obeying.

  The switching example shown in FIG. 5 is the same as the switching example shown in FIG. 4 in which the HD image quality and the SD image quality are interchanged, and thus the description thereof is omitted.

  Next, specific example 2 of the status change by the control unit 20 will be described. In the second specific example, the on / off of the camera shake correction function or the intensity thereof is controlled. That is, as a result of the determination by the determination unit 14, the control unit 20 turns off the camera shake correction function when the photographer is looking at the monitor 230 or weakens its strength, and corrects the camera shake when the photographer is not looking at the monitor 230. Turn the function on or increase its strength.

  Returning to FIG. 2, the camera shake correction unit 21 executes existing general electronic camera shake correction in response to an instruction from the control unit 20. That is, a frame image of the entire imaging range acquired from the first acquisition unit 11 is temporarily stored, and an effective area to be output to the branch unit 31 is selected from the frame image. More specifically, the shift amount between frames that are temporally forward and backward is calculated, and the effective area is moved so as to cancel the shift amount.

  6A to 6C are diagrams illustrating an example of camera shake correction performed by the camera shake correction unit 21. FIG. FIG. 6A shows a frame image one frame before the current frame image. FIG. 6B shows a current frame image before correction. FIG. 6C shows the current frame image after correction. The imaging range P is constant over the entire imaging period.

  A person image is shown as a subject in the effective area E1 shown in FIG. Here, the left eye of the person is set at the feature point FP1 of the frame image. The feature point FP2 of the frame image in the effective area E2b shown in FIG. 6B has moved a predetermined distance in the right direction compared to the frame image of the previous frame. Therefore, as shown in FIG. 6C, in order to cancel the movement, the effective area E2a of the current frame is moved to the left by the distance d. Thereby, camera shake is corrected.

  It should be noted that the compensation amount for the deviation amount does not have to be completely coincident, and the intensity of the correction may be adjusted to 50%, 80%, or the like. For example, as a result of determination by the determination unit 14, the control unit 20 sets the camera shake correction function to ON with an intensity of 50% or more when a specific photographer registered in advance who is unaccustomed to shooting is looking at the monitor 230. May be.

  Next, specific example 3 of the status change by the control unit 20 will be described. In specific example 3, on / off of the auto framing function is controlled. That is, as a result of the determination by the determination unit 14, the control unit 20 turns off the auto framing function when the photographer is looking at the monitor 230, and turns on the auto framing function when the photographer is not looking at the monitor 230. .

  Returning to FIG. 2, the auto framing processing unit 22 executes an existing general auto framing process in response to an instruction from the control unit 20. That is, the position of the subject to be targeted in the region to be trimmed from the frame image is fixed. More specifically, even if the subject moves in the frame screen or the direction of the camera changes, it is first detected in the frame image by adaptively moving the area to be trimmed from the frame image. Control is performed so that the position of the subject is maintained in the recording image. In the present embodiment, a frame image acquired from the first acquisition unit 11 is temporarily stored, and a trimming region to be output to the branch unit 31 is selected from the frame image.

  FIGS. 7A to 7C are diagrams illustrating an example of the auto framing processing by the auto framing processing unit 22. FIG. 7A shows a frame image F11 that is two frames before the current frame image F13. FIG. 7B shows a frame image F12 that is one frame before the current frame image F13. FIG. 7C shows the current frame image F13.

  FIGS. 7A to 7C show an example in which a person to be a subject is moving in the right direction. The trimming areas T1 to T3 move in the right direction in accordance with the movement of the person so that the person to be the subject in the area is located at the center. Thereby, a moving image in which the person is photographed in the center can be recorded.

  Next, specific example 4 of the status change by the control unit 20 will be described. In the fourth specific example, it is controlled whether or not the voice to be recorded has directivity. That is, when the photographer is looking at the monitor 230 as a result of the determination by the determination unit 14, the control unit 20 records sound data with directivity in the sound, and the photographer is not looking at the monitor 230. The voice data is recorded without giving directivity to the voice.

  More specifically, when the photographer is looking at the monitor 230, the control unit 20 makes the sound level from the direction in which the first image capturing unit 210 is facing higher than the sound level from the other direction. The voice encoding unit 45 encodes the collected voice. For example, the level of sound collected from directions other than the direction in which the first imaging unit 210 is facing is attenuated. On the other hand, when the photographer is not looking at the monitor 230, the control unit 20 equally processes the voices collected from all directions and causes the voice encoding unit 45 to encode them.

  Next, a specific example 5 of the status change by the control unit 20 will be described. In the specific example 5, it is controlled whether or not the captured moving image is encoded and / or recorded. That is, the control unit 20 stops encoding and / or recording of the captured moving image when the photographer does not look at the monitor 230 as a result of the determination by the determination unit 14. Alternatively, the entire imaging apparatus 200 is turned off or shifted to the power saving mode. On the other hand, when the photographer is watching the monitor 230, the control unit 20 resumes encoding and / or recording of a moving image. Alternatively, the power supply of the entire imaging apparatus 200 is turned on from off or returned from the power saving mode to the normal mode.

  Next, a specific example 6 of the status change by the control unit 20 will be described. In the specific example 6, the bit rate at the time of encoding the moving image is controlled. That is, as a result of determination by the determination unit 14, the control unit 20 maintains or increases the encoding bit rate of the captured moving image when the photographer is looking at the monitor 230, and the photographer is not looking at the monitor 230. At this time, the coding bit rate of the moving image is lowered.

  In addition, when controlling a bit rate, the seriousness degree of the facial expression detected by the face detection part 13 may be considered. When the photographer is looking at the monitor 230, the control unit 20 may control the bit rate to increase as the seriousness increases. For example, a proportional relationship may be provided between the seriousness and the bit rate.

  Next, a specific example 7 of the status change by the control unit 20 will be described. In the seventh specific example, the monitor 230 is turned on / off. That is, as a result of determination by the determination unit 14, the control unit 20 controls the monitor 230 to turn on when the photographer is looking at the monitor 230, and when the photographer is not looking at the monitor 230, Control power off. For example, when the monitor 230 is a liquid crystal monitor, the backlight is on / off controlled.

  As described above, according to the present embodiment, the photographing action of the photographer can be supported by changing the status of the imaging apparatus 200 according to whether or not the photographer is looking at the monitor 230. . For example, in the first specific example, at the time of shooting, it is possible to generate moving image encoded data of two types of image quality by dual-encoding the moving image. At that time, a necessary scene is recorded with two types of image quality. And a request for suppressing the increase in data capacity can be automatically satisfied without any user operation. Also in the above specific examples 2 to 7, it is possible to automatically set to a mode that matches the situation at the time of shooting without user operation.

  The present invention has been described based on the embodiments. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are also within the scope of the present invention. is there.

  In the above-described embodiment, the example in which the image encoding unit 40 includes two encoding units has been described. In this regard, the image coding unit 40 may include a single coding unit. In this case, Specific Example 1 is not applicable, but Specific Examples 2 to 7 are applicable.

  DESCRIPTION OF SYMBOLS 11 1st acquisition part, 12 2nd acquisition part, 13 Face detection part, 14 Judgment part, 20 Control part, 21 Camera shake correction part, 22 Auto framing process part, 31 Branch part, 32 Resolution / view angle conversion part, 40 images Encoding unit, 41 First image encoding unit, 42 Second image encoding unit, 45 Audio encoding unit, 50 Multiplexing unit, 51 Recording unit, 52 Input / output unit, 100 Image processing device, 200 Imaging device, 210 1st imaging part, 220 2nd imaging part, 230 monitor, 240 audio | voice acquisition part, 250 operation part.

Claims (5)

An image processing device to be mounted on an imaging device for capturing a moving image,
A first acquisition unit that acquires an image from a first imaging unit for capturing a moving image for recording;
A second acquisition unit that acquires an image from a second imaging unit for imaging the photographer;
A determination unit that determines whether or not the photographer is viewing a monitor displaying the recording moving image based on the image acquired by the second acquisition unit;
A control unit that changes a status of the imaging device in accordance with a determination result by the determination unit;
An image processing apparatus comprising:   When the photographer is watching the monitor as a result of the determination by the determination unit, the control unit obtains a plurality of pieces of encoded video data with different image quality from the moving images acquired by the first acquisition unit. 2. The method according to claim 1, wherein when the photographer is not looking at the monitor, control is performed so as to generate one type of moving image encoded data from the moving image acquired by the first acquisition unit. An image processing apparatus according to 1. The moving image acquired by the first acquisition unit may be encoded with both the first image quality and the second image quality that is lower than the first image quality. An encoding unit that can also
As a result of determination by the determination unit, when the photographer is watching the monitor, two types of moving image encoding are performed with the first image quality and the second image quality from the moving image acquired by the first acquisition unit. Generate data, and generate one type of moving image encoded data with the first image quality or the second image quality from the moving image acquired by the first acquisition unit when the photographer is not looking at the monitor The image processing apparatus according to claim 2, wherein the image processing apparatus is controlled to perform.   When the photographer is looking at the monitor as a result of determination by the determination unit, the control unit turns off or weakens the camera shake correction function, and when the photographer is not looking at the monitor, the camera shake correction The image processing apparatus according to claim 1, wherein the function is turned on or the strength thereof is increased. A first imaging unit for capturing a moving image for recording;
A second imaging unit for imaging the photographer;
An image processing device according to any one of claims 1 to 4,
An imaging apparatus comprising:

Images