JP2020109512A - Image processing system, control method of image processing system - Google Patents

Abstract

To provide an image processing system capable of outputting images on a monitor which allows a user to use as if using an optical mirror, an imaging apparatus, an image processing method, and an imaging method.SOLUTION: The image processing system outputs images on a display unit that displays an image including the user captured by image capturing means to the user. The image processing system includes: image acquisition means that acquires an image of the user viewed from a direction different from the display unit from the image capturing means; distance acquisition means that acquires distance information between the user and the display unit; and control means that enlarges or reduces the image acquired from the acquisition means so as to prevent the display size of the user in the image from changing due to the change in the distance information, and displays the image on the display unit.SELECTED DRAWING: Figure 4

Description

The present invention relates to an image processing device, an imaging device, an image processing method, and an imaging method for displaying an image captured by a camera on a monitor.

Conventionally, a digital mirror technology that enhances convenience by generating an image from a direction that cannot be seen by an optical mirror in front of the camera as an image obtained by capturing with multiple cameras and displaying it on a monitor installed in front of the user. Is disclosed (Patent Document 1).

JP, 2002-290964, A

However, in the conventional technique disclosed in Patent Document 1, since the position of the monitor and the camera are different, when the user moves back and forth and adjusts the size of the image displayed on the monitor, an image (subject) is obtained as with an optical mirror. It doesn't change, so it feels strange.

Therefore, an object of the present invention is to provide an image processing device, an imaging device, an image processing method, and an imaging method that output an image to a monitor that can be used with a feeling similar to that of using an optical mirror. Is.

In order to achieve the above object, an image processing device of the present invention is an image processing device that outputs the image to a display unit that displays an image including the user captured by an image capturing unit toward the user. An image acquisition unit that acquires, from the imaging unit, an image of the user viewed from a direction different from the display unit; and a distance acquisition unit that acquires distance information between the user and the display unit. A control unit for enlarging or reducing the image acquired from the acquisition unit and displaying the image on the display unit so as to suppress a change in the display size of the user in the image due to a change in the distance information. And are included. An image processing apparatus control method according to the present invention is a method for controlling an image processing apparatus, which outputs the image to a display unit that displays an image including the user captured by an image capturing unit toward the user, An image acquisition step of acquiring an image of the user viewed from a different direction from the display unit from the imaging unit; a distance acquisition step of acquiring distance information between the user and the display unit; And a control step of displaying the image on the display unit by enlarging or reducing the image acquired from the acquisition unit so as to suppress a change in the display size of the user in the image due to the change. Is characterized by.

According to the present invention, the monitor can be used with a feeling close to that of using an optical mirror.

The bird's-eye view for explaining the composition of the digital mirror concerning the present invention. FIG. 3 is a block diagram for explaining the configuration of a digital camera according to the present invention. FIG. 3 is a block diagram for explaining the configuration of a digital monitor according to the present invention. 3 is a flowchart for explaining the operation of the digital monitor according to the first embodiment. 3 is a flowchart for explaining the operation of the digital camera according to the first embodiment. 9 is a flowchart for explaining the operation of the digital monitor according to the second embodiment. 9 is a flowchart for explaining the operation of the digital monitor according to the second embodiment. 9 is a flowchart for explaining the operation of the digital camera according to the second embodiment.

[First Embodiment]
Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the drawings. Note that one embodiment described below is an image processing system including an imaging device and an image processing device. Hereinafter, an embodiment in which the present invention is applied to a digital camera as an example of an imaging device, a digital monitor as an example of an image processing device, and a digital mirror as an example of an image processing system using them will be described.

FIG. 1 is a bird's-eye view for explaining the configuration of the digital mirror according to the first embodiment.

The digital monitor 105 has, for example, a monitor (display device) such as a liquid crystal display and an image processing device that outputs an image to the display device.

The digital cameras 101, 102, 103, 104 are digital cameras arranged so as to capture different angles of view. The digital cameras 101 to 104 are arranged so that the angle of view can include at least part of a predetermined three-dimensional space including a space existing when the monitor target 106, which is a user, uses the monitor target 106. In the present embodiment, the optical axes of the digital camera 101 and the digital camera 104 are arranged at initial positions such that the optical axes are substantially orthogonal on the xy plane shown in FIG. Further, the digital cameras 102 and 103 are arranged at angles of the angular ranges 107 and 108 from the side closer to the digital camera 101 side than the digital camera 104.

The number and the positional relationship of the digital cameras are not limited to the above-mentioned form, and the installation angle of the camera is set so that the monitor target 106 can be captured in a wider angle range (for example, the viewpoint image can be generated in the entire 360 degree angle range). The range may be expanded or the number may be increased. On the other hand, the number of cameras to be arranged may be reduced by providing a camera capable of shooting at a wider angle.

The digital monitor 105 can display an image captured by each digital camera on the monitor by communicating with the digital cameras 101 to 104 by wire or wirelessly. The user 106 confirms the viewpoint image generated by using a part or all of the images of the digital cameras 101 to 104 on the digital monitor 105, so that the user's appearance can be changed without changing his or her face direction or line of sight. , You can check the posture from multiple directions. For example, the user can check the form of sports such as baseball and archery on the digital monitor 105 from any direction.

Further, in the present embodiment, the monitoring target 106 is a user, but it is also conceivable that the product to be inspected is a monitoring target in a factory or the like. In this case, the present digital mirror is installed in the place where the process of inspecting is performed, and the inspector performs inspection while moving the monitor target 106, which is a product, with respect to the monitor of the digital monitor 105. An inspector can confirm an image from an arbitrary direction on the digital monitor 105 without changing the direction of the monitor target 105 that is the inspection target or by changing the minimum direction.

FIG. 2 is a block diagram showing a functional configuration of the digital camera 200 (digital cameras 101 to 104 in FIG. 1) according to the embodiment of the present invention. The control unit 201 is, for example, a CPU, and controls the operation of each block included in the digital camera 200 by reading the operation program of each block included in the digital camera 200 from the ROM 202, expanding the program in the RAM 203, and executing the program. The ROM 202 is a rewritable non-volatile memory and stores, in addition to the operation program of each block included in the digital camera 200, parameters necessary for the operation of each block. The RAM 203 is a rewritable volatile memory and is used as a temporary storage area for data output in the operation of each block included in the digital camera 200.

The optical system 205 includes a zoom lens, a focus lens, and a lens driving unit that drives these lenses, and forms a subject image on the imaging unit 206. The image pickup unit 206 is, for example, an image pickup device such as a CCD or CMOS sensor, photoelectrically converts an optical image formed on the image pickup device by the optical system 205, and outputs the obtained analog image signal to the A/D conversion unit 207. To do. The A/D conversion unit 207 applies A/D conversion processing to the input analog image signal and outputs the obtained digital image data to the RAM 203 for storage. The control unit 201 controls the optical system 205 based on the image data stored in the RAM 203, and performs imaging control such as AF (automatic focus adjustment) and AE (automatic exposure control).

The image data stored in the RAM 203 is input to the image processing unit 208, subjected to processing such as color conversion, white balance, and gamma correction, converted into YCrCb luminance and color difference signals, and output to the communication unit 204 or to the ROM 202. Will be recorded.

The communication unit 204 transmits image data and the like to the digital monitor 105 using a wireless LAN or the like, and receives commands related to control of the optical system 205 from the digital monitor 105 or a remote control terminal.

FIG. 3 is a block diagram showing a functional configuration of the digital monitor 300 (105 in FIG. 1) according to the embodiment of the present invention. The control unit 301 is, for example, a CPU, and controls the operation of each block included in the digital monitor 105 by reading the operation program of each block included in the digital monitor 105 from the ROM 302, expanding the program in the RAM 303, and executing the program.

The ROM 302 is a rewritable non-volatile memory, and stores parameters necessary for the operation of each block, in addition to the operation program of each block included in the digital monitor 105. The RAM 303 is a rewritable volatile memory and is used as a temporary storage area for data output in the operation of each block included in the digital monitor 105.

The communication unit 304 is connected by a wireless or wired cable and transmits/receives a video signal and an audio signal. The communication unit 304 can also be connected to a wireless LAN (Local Area Network) and the Internet. The communication unit 304 can also communicate with an external device using Bluetooth (registered trademark) or Bluetooth Low Energy. The communication unit 304 also receives image data and the like from the digital camera 200, and transmits commands and the like related to control of the digital camera 200. The communication unit 304 also receives commands relating to various controls of the digital monitor 300 or the digital camera 200 from a remote control terminal or the like.

The image processing unit 305 generates a viewpoint image of an arbitrary viewpoint using at least a part of the images of the digital cameras 101 to 104 in FIG. A known method may be used as a method of generating the viewpoint image. For example, in Japanese Patent Application Laid-Open No. 2004-242242, a technique for calculating three-dimensional shape data from a multi-directional image, setting a specific viewpoint for data obtained by texture mapping the obtained three-dimensional model, and obtaining image data based on that viewpoint It is disclosed. The direction is instructed by a touch panel, or by detecting the voice of the user, the movement of the finger or the line of sight.

The display unit 306 is, for example, a display element such as a liquid crystal or an organic EL, and displays a viewpoint image generated by the image processing unit 305, an operation screen, information about the image, and the like according to an instruction from the control unit 301.

The operation unit 307 includes various operation members such as buttons and wheels, and functions are assigned in advance. For example, by selecting and operating various function icons displayed on the display unit 306, the operation unit 307 can have appropriate functions for each scene. It is assigned and acts as various function buttons. In addition, a touch panel that can detect contact with the display portion 306 is included as one of the operation portions 307. The touch panel and the display unit 306 can be integrally configured. For example, the touch panel is configured so that the light transmittance does not hinder the display of the display portion 306 and is attached to the upper layer of the display surface of the display portion 306. Then, the input coordinates on the touch panel are associated with the display coordinates on the display unit 306. This makes it possible to configure a GUI (graphical user interface) that allows the user to directly operate the screen displayed on the display unit 306.

The operation unit 307 also includes a receiving device such as a microphone or an infrared sensor that receives voice input or line-of-sight input. The line-of-sight input (line-of-sight detection) is performed by using an infrared light source included in the digital monitor 105 to irradiate a user's eyeball with infrared light to obtain reflected light, for example, in the digital camera 101 which has the same shooting direction as the display direction of the monitor, Light is received by the attached sensor. The control unit 301 detects the line of sight of the user from the received optical image, and accepts an operation instruction by associating the behavior of the line of sight (eye) or the position of the menu screen displayed on the display unit 306 with the position of the line of sight. As described above, the operation unit 307 includes various modes, and various operation instructions may be given from any of the operation modes. However, by using only the touch panel, the voice input or the line-of-sight input as the operation unit of the digital monitor 105 as the operation unit 307, various physical operation members are not exposed, and the appearance is more similar to that of an optical mirror. You can do it. In addition, voice input or line-of-sight input has an advantage that an operation instruction can be given even when the user keeps the body posture or performs some work by hand.

FIG. 4 is a flowchart for explaining the operation of the control unit 301 of FIG. Each step of this flowchart is executed by each unit according to an instruction from the control unit 301 or the control unit 301. It is assumed that this flow is started, for example, when the user turns on the power of the digital monitor 105 and the digital cameras 101 to 104 are also turned on.

First, a change in the distance between the user and the monitor is detected from the image of the camera 101 in FIG. 1 (S400). Here, it is assumed that the image of the camera 101 is input to the image processing unit 305 via the communication unit 304 and acquired. A change in distance between the user and the digital monitor 105 is detected by monitoring the change using the face size of the user captured by the digital camera 101 in FIG. 1 as distance information. A known method can be used for the face size detection method. For example, as a well-known face detection technique, a technique based on learning using a neural network or the like, template matching is used to search for a region having a characteristic shape such as eyes, nose, or mouth from an image, and if the similarity is high, the face is detected. There is a method to consider. In addition, many other methods have been proposed, such as a method using statistical analysis to detect image feature amounts such as skin color and eye shape. It is also possible to improve the accuracy of face detection by combining a plurality of these methods. As a specific example, there is a method for detecting a face using the wavelet transform and the image feature amount described in JP-A-2002-251380. The face detection technique described above is used to detect a face area from each image, and the control unit 301 further detects a change in the size of the detected face area to detect a relative change in distance.

Further, the method of detecting the change in distance between the user and the monitor is not limited to the above. The camera 101 has a plurality of image pickup elements so that images can be picked up from a plurality of viewpoints, and at least relative distance information (defocus amount) is acquired by calculating parallax by a parallax (phase difference) detection method, The change may be detected. Further, one pixel unit is composed of a microlens and a plurality of photoelectric conversion units, and a pair of light fluxes passing through different pupil regions of the image pickup optical system are respectively formed as optical images to form a pair of image data. An image sensor that can output from the photoelectric conversion unit may be used. The output from the image pickup device also has the parallax between the outputs from the photoelectric conversion units of the respective pixels, so that the distance can be acquired by the parallax detection method. Further, the TOF (Time Of Flight) method for measuring the delay time from the projection of the object to the reception of the reflected light to measure the distance information to the object is used to measure the object distance from the imaging device to the object in the image. May be obtained directly. In the TOF method, pulsed light is projected onto a subject (object) by a light projecting unit, reflected light is received by an image pickup unit, and the flight time (delay time) of this pulsed light is measured to determine the subject distance (target). The distance to an object) is measured to obtain depth information. Further, a contrast evaluation method may be used in which the distance is measured by monitoring the contrast evaluation value of the image of the subject.

Next, it is determined whether the distance between the user and the digital monitor 105 is changing (S401). This is performed by determining whether or not the above-described distance change has occurred with a change amount of a predetermined amount or more over a predetermined number of frames. When the control unit 301 determines that the distance between the user and the digital monitor 105 is changing, the control unit 301 mainly displays the image by the clipping process by the image processing unit 305 rather than the zoom control by the optical system of the digital cameras 101 to 104. Control size. That is, the control unit 301 controls the display size by changing the cutout size of the cutout process so as to suppress the change of the display size of the user in the image due to the change of the distance information. For example, the image processing unit 305 generates and displays an enlarged or reduced image so that the viewing angle corresponds to the distance between the user and the digital monitor 105 (S402). The display size is adjusted by the image processing unit 305 so that the display angle becomes a viewing angle according to the distance between the user and the digital monitor 105 as if the optical mirror were at the position of the digital monitor 105. In the present embodiment, the viewing angle (image size, number of pixels) corresponding to the distance measurement result is stored in the ROM 302 in advance and used for adjustment. By adjusting the viewing angle in accordance with the change in distance in this way, the display is made larger when the distance between the user and the digital monitor 105 is closer, and smaller when the distance is far. Specifically, when a viewpoint image taken from the direction in which the digital monitor 105 is present is displayed on the digital monitor 105, the display size is adjusted so as to give the same usability as an optical mirror. Further, in the case of an image displayed on the digital monitor 105 from a direction different from that of the digital monitor 105, for example, the size of the user 106 (the face thereof) does not change as compared with the image from the direction of the digital camera 101. Adjust the display size to. Like the images acquired from the digital camera 101, a table of the viewing angle (image size, the number of pixels) corresponding to the distance measurement result is also stored in the ROM 302 in advance for the digital cameras 102 to 104, and this table is set to the display size. It may be used for adjustment.

When it is determined that the distance change between the user and the digital monitor 105 is less than the predetermined amount, it is determined that the user's movement is stable, and the distance change detected from step S400 until the change is stable, This time, the optical zoom of the digital cameras 101 to 104 is changed. Specifically, the control unit 301 calculates the total distance change from the timing when it is determined that the distance change between the user and the digital monitor 105 is greater than or equal to a predetermined amount to the timing when it is determined that the distance change is less than the predetermined amount. To do. The control unit 301 specifies the focal length of each digital camera corresponding to the distance change based on the calculated total change amount of the distance, and transmits a command to each digital camera as an instruction signal including an instruction to change the focal length ( S403). The digital cameras 102 to 104 receive the instruction signal via the communication unit 204, and the control unit 201 controls each optical system 205 to the corresponding focal length. By the operation in step S403, the display size of the image displayed on the digital monitor 105 becomes the viewing angle according to the distance between the user and the digital monitor 105 as described above. The method of adjusting the viewing angle of each digital camera is not limited to this. The size of the user (subject, face) is detected, and the image data output from each digital camera is controlled by the control unit 301 or so that the size of the user becomes substantially the same when the distance change starts and stabilizes. The control unit 201 may control the optical system 205 while monitoring.

Here, the digital camera 101 changes the distance to the user in the same way as the digital monitor 105 in accordance with the movement of the user, so the optical zoom is not changed according to the change in the distance in step S400.

As described above, the digital cameras 102 to 104 zoom in when the distance between the user and the digital monitor 105 is close, and zoom out when the distance is far.

When the user gives an instruction to end the display through an external device or an operation I/F (not shown), the control unit 301 ends the image display on the display unit 306 (S404).

Although not shown as a flow, it is assumed that the user can instruct the digital cameras 101 to 104 to take an image at any timing between steps S400 and S404. Specifically, the user can wirelessly transmit various instructions from the remote control terminal or a device having a similar function to the communication unit 204 of each digital camera or the communication unit 304 of the digital monitor 105. When the communication unit 204 receives the image capturing instruction signal from the external device as described above, the control unit 201 causes the image capturing unit 206 to perform image capturing according to the signal. Using these configurations, the user can also issue an imaging instruction at any timing during the activation of the digital monitor 105. Further, the digital monitor 105 may have a clean capture function, and the user can instruct transmission of a screen capture execution instruction signal via an external device at any timing.

Further, FIG. 5 is a flowchart for explaining the operation of the digital cameras 101 to 104 of FIG. Each step of this flowchart is executed by each unit according to an instruction from the control unit 201 or the control unit 201. This flow is started, for example, when the user turns on the power of the digital monitor 105 and the digital cameras 101 to 104 are also turned on. At this time, image data taken by at least some of the digital cameras 101 to 104 is transmitted to the digital monitor 105.

It is confirmed whether the optical zoom change command is received from the digital camera digital monitor 105 (S501). If not received, the optical system related to the focal length does not move, the subject (user) is sequentially imaged as it is at the current viewing angle, the image is captured, and the processed image data is sequentially output to the digital monitor 105 (S502). .. When the optical zoom change command is received, the zoom position of the optical system is changed according to the change command (S503). When the user instructs to end the image capturing, the image capturing by the image capturing unit 206 is terminated (S504). By controlling in this way, the display size is smoothly changed only by image processing of the digital monitor 105 while the user is moving. Furthermore, since the high-definition image can be displayed by the optical zoom of the digital camera shown in FIG. 1 when the user is stationary, the visibility of the user can be improved.

In the present embodiment, images from arbitrary directions are generated and displayed by combining images in a plurality of directions, but the present invention is not limited to this, and images may be displayed without image combination depending on the angle direction. It may be displayed. For example, in FIG. 1, images from the digital cameras 101 and 102 are combined in the angular range 107, and images from the digital cameras 102 and 103 are combined in the angular range 108. However, in the angular range 109 near the digital camera 102, the image of the digital camera 102 may be displayed without image combination. In that case, the direction may be adjusted by optically panning the digital camera 102 according to the deviation from the designated direction.

By controlling in this way, it is possible to display a high-definition image without deterioration due to image synthesis in the vicinity of the direction in which the digital camera is arranged.

In the present embodiment, the same digital cameras 101 to 104 of FIG. 1 are used, but the present invention is not limited to this, and the camera specifications may have different optical characteristics depending on the direction. good. For example, for a digital camera in a direction in which the distance between the user and the camera cannot be sufficiently secured due to the restrictions on the location where the camera is installed, a wider-angle zoom lens may be mounted. Further, for example, the digital camera 104 that is the farthest from the digital monitor 105 has a perspective image when the user 106 approaches the digital monitor 105. Therefore, a digital camera equipped with a shift lens to correct it. You may

With this configuration, it is possible to reduce the deterioration of the display image quality depending on the direction.

In this embodiment, the display size is the same as the optical mirror so that the display angle is a viewing angle according to the distance between the user and the digital monitor 105. However, the present invention is not limited to this, and the display angle may be displayed smaller or larger than the viewing angle according to the distance between the user and the digital monitor 105, and the size on the digital monitor 105 may not change. You may scale it. For example, when the distance between the digital monitor 105 and the user 106 in FIG. 1 is not enough, but when the user wants to confirm the whole-body image of the user 106 for the purpose of confirming the form of the sport, the user's instruction causes the visual angle of the optical mirror to be larger than that of the optical mirror. It may be displayed small. Further, for example, when the user wants to make a close-up check of a part of the face for the purpose of makeup, the display may be made larger than the viewing angle of the optical mirror. Further, for example, assuming that the user wants to perform sports form confirmation with the same size as the previous time, the control unit 301 manually or automatically records the distance between the digital monitor 300 and the user 106 in the ROM 302 during use. You may keep it. When displaying an image from the digital camera 200 on the digital monitor 105, the control unit 301 determines whether or not information about the distance between the monitor users regarding the subject size is recorded in the ROM 302. When the information on the distance is recorded, the control unit 301 scales up and down compared with the recorded distance according to the distance between the user and the digital monitor 105 at that time so that the subject size does not change from the previous time. By displaying it, the previous display size is reproduced. Further, the control unit 301 records the size of the subject in the image from each digital camera displayed on the display unit 306, controls the optical zoom of each digital camera so as to have the size, or enlarges or reduces the image. Good. With such a configuration, it is possible to display with a display size that suits the purpose of the user.

Further, the control unit 301 has a function of outputting or recording a moving image from the digital camera 200 to the communication unit 304 or the ROM 302, and displaying a recorded moving image or a still image or a moving image received from an external device via the communication unit 304 on the display unit 306. You may have the function to display. By doing this, you can record the form of the last sports and check the difference from the last time yourself, or share the video captured by the digital camera 200 by connecting to the Internet with the instructor at a remote place and give communication instruction. Will be received.

In the present embodiment, the image from the direction (camera) instructed by the user is displayed on the digital monitor 105, but the present invention is not limited to this. For example, the image from the direction of the digital monitor 105 may always be displayed. For example, the image in the direction of the digital monitor 105 may be displayed on the entire surface, and then the image in the direction designated by the user may be displayed in a small wipe shape, or vice versa. This allows the user to use the digital monitor 105 as a mirror and to check the user's image from the other side without significantly shifting the line of sight. Further, the images displayed on the entire surface and the wipe may be switched in accordance with a user's instruction via the operation unit 307 or the like. Further, the digital monitor 105 may be composed of a plurality of panels such as liquid crystals so that the image in the direction of the digital monitor 105 and the image in the direction designated by the user are displayed on different panels.

Further, in the present embodiment, the digital camera 200 controls the brightness of the captured image by performing the AE control, but at least some of the cameras have an auxiliary light source and are adjusted manually or according to the brightness of the subject. Alternatively, the light amount may be adjusted. Further, the color (color temperature) of the auxiliary light source may also be switched according to an application or the like via the operation unit 307 depending on the application.

With such a configuration, it is possible to provide a digital mirror having both the usability of the conventional optical mirror and the convenience of the digital mirror. In addition, the display unit 306 of the digital monitor 105 may be integrally configured with an actual optical mirror. In this case, the digital camera 101 that captures the angle of view in the display direction of the digital monitor 105 may be omitted.

[Second Embodiment]
An image processing system according to the second embodiment of the present invention will be described below. A digital camera as an example of an image pickup apparatus, a digital monitor as an example of an image processing apparatus, and a digital mirror as an example of an image processing system according to the present embodiment will be described in Example 1 with reference to FIGS. 1 and 2. It is similar to each of the configurations described above.

FIG. 6 is a flow chart for explaining the operation of control unit 301 of FIG. 3 in the present embodiment.

First, it is determined whether or not the distance between the user and the digital monitor 105 is changing (S601). The change in the distance between the user and the digital monitor 105 is detected by monitoring the change in the face size of the user captured by the digital camera 101 in FIG. A known method may be used as the face size detection method.

If the distance between the user and the digital monitor 105 is changing, all the digital cameras 101 to 104 in FIG. 1 are turned on (S602).

If it is determined that the distance between the user and the digital monitor 105 is stable and the user is stationary, the power to the camera outside the directional cover range of 0° to 90° is turned off and the power supply is cut off (S603). .. When the user gives an instruction to end the display, the process ends (S604).

FIG. 7 is a flowchart for explaining the operation of control unit 301 of FIG. 3 in the present embodiment. First, it is determined whether the direction is changing (S701). If the direction is changing, all the digital cameras 101 to 104 in FIG. 1 are turned on (S702). If it is determined that the direction is stable, the power consumption can be reduced by turning off the power of the digital camera that is not referred to when creating the composite image in that direction (S703). When the display of the display is instructed by the user, the display ends (S704).

In the present embodiment, the power of the selected camera is turned off in order to suppress the power consumption (supply power), but the present invention is not limited to this, and the selected camera is set in the power saving mode. You may make it shift to. With this configuration, the recovery time of the selected camera can be shortened, so that the user can be provided with a comfortable operation feeling without the digital mirror freezing.

FIG. 8 is a flow chart for explaining another operation when the digital cameras 101 to 104 operate in response to receiving the power ON/OFF instruction signal from the digital monitor 105. This flowchart is operated in parallel with the flowchart of FIG. 5 of the first embodiment.

It is confirmed whether the power-off command is received from the digital monitor 105 (S801). If the power-off command is received, the power supply to the optical system 205, the imaging unit 206, the A/D conversion unit 207, and the image processing unit 208 in FIG. 2 related to imaging is turned off (S804). Further, it is confirmed whether the power ON command is received from the digital monitor 105 of FIG. 1 (S802). When the power ON command is received, the power supply of the optical system 205, the image pickup unit 206, the A/D conversion unit 207, and the image processing unit 208 of FIG. 2 related to the image pickup is turned on (S803). When the user gives an instruction to end the image capturing, the process ends (S805).

In the present embodiment, the power consumption is suppressed by selecting the camera that can be turned off according to the positional relationship between the user and the digital monitor 105. However, the present invention is not limited to this, and the specified A camera to be turned off may be selected according to an arbitrary direction. Further, the setting operation to be performed for the unnecessary digital cameras 101 to 104 is not power off, but the power is switched by switching the mode to a mode such as a power saving mode in which the function to be operated is limited to reduce power consumption. It may be configured to suppress consumption.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes can be made within the scope of the gist thereof.

(Other embodiments)
The object of the present invention can also be achieved as follows. That is, a storage medium recording a program code of software in which a procedure for realizing the functions of the above-described embodiments is recorded is supplied to a system or an apparatus. Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads and executes the program code stored in the storage medium.

In this case, the program code itself read from the storage medium realizes the novel function of the present invention, and the storage medium and the program storing the program code constitute the present invention.

Moreover, examples of the storage medium for supplying the program code include a flexible disk, a hard disk, an optical disk, and a magneto-optical disk. Further, a CD-ROM, a CD-R, a CD-RW, a DVD-ROM, a DVD-RAM, a DVD-RW, a DVD-R, a magnetic tape, a non-volatile memory card, a ROM, or the like can be used.

Further, the functions of the above-described embodiments are realized by making the program code read by the computer executable. Further, in the case where an OS (operating system) running on the computer performs a part or all of actual processing based on the instructions of the program code, and the processing realizes the functions of the above-described embodiments. Is also included.

Furthermore, the following cases are also included. First, the program code read from the storage medium is written in a memory provided in a function expansion board inserted in the computer or a function expansion unit connected to the computer. After that, based on the instruction of the program code, the CPU or the like included in the function expansion board or the function expansion unit performs a part or all of the actual processing.

101 digital camera 102 digital camera 103 digital camera 104 digital camera 105 digital monitor 106 user 107 angle range 108 angle range 109 angle range

Claims (19)

An image processing device for outputting the image to a display unit that displays an image including the user captured by an image capturing unit toward the user,
An image acquisition unit that acquires from the imaging unit an image of the user viewed from a direction different from the display unit,
A distance acquisition unit that acquires distance information between the user and the display unit,
Control means for enlarging or reducing the image acquired from the acquisition means to display the image on the display unit so as to suppress a change in the display size of the user in the image due to a change in the distance information; An image processing apparatus comprising: The control means controls the optical system of the imaging means based on the distance information so that an image displayed on the display section has a viewing angle corresponding to a distance between the user and the display section. Image processing device. The control means is
When it is determined that the distance between the user and the display unit is changing based on the distance information, an image acquired from the acquisition unit so as to suppress a change in the display size of the user. Scale up or down,
When it is determined that the distance between the user and the display unit is changing based on the distance information, and then it is determined that the distance is stable, the optical system of the image capturing unit is controlled to control the optical system. The image processing apparatus according to claim 2, wherein a change in display size of a user is suppressed. The image pickup unit has a plurality of image pickup units for picking up images of the user from different viewpoints,
The image acquired from the image pickup unit by the control unit is a combined image obtained by combining the images output from the plurality of image pickup units. The image processing device described. The image processing apparatus according to claim 4, wherein the plurality of image capturing units included in the image capturing unit have different optical characteristics from each other. The control unit changes a part of the plurality of image pickup units of the image pickup unit to a mode in which power consumption is suppressed, according to the distance information. The image processing device described. The control unit changes a part of the image capturing units that captures an image of a viewpoint that is not used to generate the composite image to a mode in which power consumption is suppressed. The image processing apparatus according to claim 4. The image pickup apparatus according to claim 7, wherein changing some of the image pickup units of the plurality of image pickup units to a mode in which the power consumption is suppressed means reducing the power supply. 8. The changing of a part of the plurality of image pickup units to a mode in which power consumption is suppressed means turning off the power supply of the image pickup unit and cutting off the power supply. Imaging device. The control unit increases the size of the user of the image displayed on the display unit when the user approaches the display unit, and displays the image on the display unit when the user moves away from the display unit. The image processing apparatus according to claim 1, wherein a size of a user of the image is reduced. The image processing device according to claim 1, wherein the distance information is a face size of the user in an image displayed on the display unit. The image processing apparatus according to claim 1, wherein the distance information is acquired by a Time Of Flight method. The image processing device according to any one of claims 1 to 10, wherein the distance information is obtained by a parallax detection method using an image acquired by the imaging unit. The image processing device according to any one of claims 1 to 10, wherein the distance information is obtained by a contrast evaluation method using an image acquired by the imaging unit. 15. The display unit displays at least two images, an image from the direction of the display unit and an image from a direction different from the direction of the display unit. The image processing device according to 1. The imaging means,
An image processing system comprising: the image processing device according to claim 1. A program for causing a computer to execute as each unit of the image processing apparatus according to claim 1. A computer-readable storage medium that stores a program for causing a computer to execute as each unit of the image processing apparatus according to claim 1. A method of controlling an image processing apparatus, which outputs the image to a display unit that displays an image including the user captured by an image capturing unit toward the user,
An image acquisition step of acquiring from the imaging means an image of the user viewed from a direction different from the display unit;
A distance acquisition step of acquiring distance information between the user and the display unit,
A control step of displaying the image on the display unit by enlarging or reducing the image acquired from the acquisition unit so as to suppress a change in the display size of the user in the image due to a change in the distance information; A method for controlling an image processing apparatus, comprising:

Images