JP2010109415A - Imaging apparatus and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus and an image processing method that prevent an image picked up by a wide-angle imaging unit from decreasing in image quality at no cost even when the view angle of the wide-angle imaging unit becomes large. <P>SOLUTION: A double-lens reflex camera 100 having a plurality of imaging units includes the wide-angle imaging unit 1 which includes a lens having a wider view angle than other imaging units, a zoom imaging unit 2 which includes a lens having a narrower view angle and larger magnification than the wide-angle imaging unit 1, and is turnably provided to the double-lens reflex camera 100, a two-axis actuator 3 for turning the zoom imaging unit 2, a memory 5 for associatively storing image data picked up by the zoom imaging unit 2 and positional information of the image data obtained from the turning state of the zoom imaging unit 2, and a detailed wide-angle image composition unit 75 which combines wide-range image data of a plurality of image data with one another and generates the image data of a wide range, on the basis of a plurality of image data stored in the memory 5 and differing in imaging range, and the positional information of the image data. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and an image processing method, and more particularly to an imaging apparatus such as a surveillance camera and an image processing method in the imaging apparatus.

  Conventionally, surveillance cameras having a wide-angle imaging unit and a zoom imaging unit are known. The wide-angle imaging unit images a relatively wide range. On the other hand, the zoom imaging unit images a part within the imaging range of the wide-angle imaging unit at a high magnification. The wide-angle imaging unit and the zoom imaging unit can be driven integrally. Thereby, the surveillance camera can image a desired portion within the surveillance range. For example, by driving the wide-angle imaging unit and the zoom imaging unit integrally, it is possible to track and image a target moving within the monitoring range. Furthermore, a surveillance camera using two or more imaging units having different angles of view is also known.

For example, Patent Document 1 describes a camera control system that displays a frame indicating a range captured by a zoom imaging unit on a display screen of an image captured by a wide-angle imaging unit.
Such an imaging apparatus having a plurality of imaging units having different magnifications has an advantage that a detailed image of a desired part of the monitoring range can be displayed while monitoring the entire range of the monitoring range.

  Patent Document 2 describes a monitoring device that performs monitoring using a plurality of imaging units arranged so that parts of the imaging range overlap each other. The monitoring device detects the position of the moving object in three dimensions.

Patent Document 3 describes a monitoring device that captures a color image using two left and right imaging units. In addition, the monitoring apparatus calculates a color ratio histogram of the center of gravity of the moving body included in the color image for each successive frame. Then, by comparing the color ratio histograms, it is determined whether or not the moving body of each frame is the same moving body.
JP 2000-341574 A JP 2006-041939 A JP 2004-355601 A

However, in recent years, the angle of view of wide-angle lenses tends to increase, and for example, wide-angle lenses with an angle of view of 180 degrees have been manufactured. Therefore, it is considered that the difference between the imaging range of the wide-angle imaging unit and the imaging range of the zoom imaging unit will increase in the future. In other words, the difference between the magnification of the wide-angle imaging unit and the magnification of the zoom imaging unit is expected to increase in the future.
If the angle of view of the wide-angle imaging unit is increased without changing the number of pixels of the image sensor used in the wide-angle imaging unit, the image quality of the image captured by the wide-angle imaging unit is degraded. Therefore, if the number of pixels of the image sensor used in the wide-angle imaging unit is increased by increasing the angle of view of the wide-angle imaging unit, the image quality of the image captured by the wide-angle imaging unit can be prevented from being degraded. However, the cost increases due to the use of an image sensor with a large number of pixels.

  The present invention has been made to solve such a problem, and prevents a reduction in image quality of an image captured by the wide-angle imaging unit without incurring costs even when the angle of view of the wide-angle imaging unit increases. An object of the present invention is to provide an imaging apparatus and an image processing method capable of performing the above.

  The imaging device according to the first aspect of the present invention is an imaging device including a plurality of imaging units each including a lens and an imaging element. Further, at least one of the plurality of imaging units is a wide-angle imaging unit having the lens having a wider angle of view than the other imaging units. In addition, at least one of the plurality of imaging units is a rotating imaging unit that includes the lens having a narrower angle of view and a larger magnification than the wide-angle imaging unit and is rotatably provided in the imaging device. . Further, the imaging apparatus includes a drive unit that rotates the rotational imaging unit so as to change an imaging range of the rotational imaging unit. In addition, the imaging apparatus includes an image storage unit that stores the image data captured by the rotational imaging unit and the positional information of the image data obtained from the rotational state of the rotational imaging unit in association with each other. . Furthermore, the imaging apparatus synthesizes a plurality of image data based on a plurality of the image data having different imaging ranges stored in the image storage unit and the position information of the image data. Is provided with an image synthesizing unit.

In the first aspect of the present invention, the rotating imaging unit captures a plurality of image data in different imaging ranges at a higher magnification than the wide-angle imaging unit. Further, the image synthesis unit synthesizes a plurality of pieces of image data with different imaging ranges captured at a high magnification to generate a wide range of image data. Therefore, it is possible to generate a wide range of image data by synthesizing a plurality of image data obtained by the rotating imaging unit dividing the imaging range of the wide-angle imaging unit into a plurality of ranges. The wide-range image data is image data in a range substantially the same as the imaging range of the wide-angle imaging unit. In addition, the wide-range image data has good image quality because the rotary imaging unit is imaging at a high magnification. Then, the wide range image data can be used instead of the image data captured by the wide-angle imaging unit. As a result, it is possible to prevent the image quality of the image data of the wide-angle imaging unit from being deteriorated due to an increase in the angle of view of the lens of the wide-angle imaging unit.
In addition, as the angle of view of the lens of the wide-angle imaging unit increases, it is not necessary to use an imaging device with a large number of pixels. Therefore, no cost is required.

Further, the entire imaging range of the wide-angle imaging unit is divided into a plurality of imaging ranges, and the rotating imaging unit is rotated by the driving unit, and the plurality of imaging ranges are sequentially imaged on the rotating imaging unit. And an initial operation control unit that controls an initial operation process for storing the plurality of image data captured by the rotational imaging unit in association with the position information of the image data in the image storage unit. preferable.
Further, the image synthesis unit synthesizes the plurality of image data stored in the image storage unit in the initial operation process based on the position information of the image data, and is approximately the imaging range of the wide-angle imaging unit. It is preferable to generate image data in the same range.
Thereby, in the initial operation, the rotating imaging unit can capture the entire imaging range of the wide-angle imaging unit at a high magnification. For this reason, image data in the same range as the imaging range of the wide-angle imaging unit can always be used instead.

Furthermore, the imaging device preferably includes an image recognition unit and a drive control unit. The image recognition unit detects a moving body in an image captured by the wide-angle imaging unit by analyzing image data captured by the wide-angle imaging unit. The drive control unit controls the drive unit to rotate the rotation imaging unit so as to image the moving body detected by the image recognition unit.
Thereby, the moving body can be tracked and an image near the moving body can be taken at a high magnification. That is, it is possible to track the moving body and take a detailed image.

The imaging apparatus preferably includes an image update control unit. The image update control unit, in the image storage unit, based on the moving body image data obtained by the rotary imaging unit imaging the moving body and the position information of the moving body image data, The image data stored in the image storage unit is updated by replacing the image data having the position information corresponding to the position information of the moving body image data with the moving body image data.
Thereby, only the part in which the change of the image data memorize | stored in the image memory | storage part in the initial operation process produced by the moving body can be replaced with moving body image data. For this reason, image data that is in the same range as the imaging range of the wide-angle imaging unit and that reflects the latest state can always be used instead.

  The image processing method according to the second aspect of the present invention is an image processing method in an imaging apparatus including a plurality of imaging units each including a lens and an imaging element. Further, at least one of the plurality of imaging units is a wide-angle imaging unit having the lens having a wider angle of view than the other imaging units. Further, at least one of the plurality of the imaging units is a rotating imaging unit that includes the lens having a narrower angle of view and a larger magnification than the wide-angle imaging unit and is rotatably provided in the imaging device. . Furthermore, the imaging device includes a drive unit and an image storage unit. The said drive part rotates the said rotation imaging unit so that the imaging range of the said rotation imaging unit may be changed. The image storage unit stores the image data captured by the rotational imaging unit and the positional information of the image data obtained from the rotational state of the rotational imaging unit in association with each other. In the image processing method, a plurality of the image data are synthesized based on the plurality of image data having different imaging ranges stored in the image storage unit and the position information of the image data, and a wide range Generate image data.

In the second aspect of the present invention, the rotating imaging unit captures a plurality of image data in different imaging ranges at a higher magnification than the wide-angle imaging unit. Further, a wide range of image data is generated by synthesizing a plurality of image data with different imaging ranges captured at a high magnification. Therefore, it is possible to generate a wide range of image data by synthesizing a plurality of image data obtained by the rotating imaging unit dividing the imaging range of the wide-angle imaging unit into a plurality of ranges. The wide-range image data is image data in a range substantially the same as the imaging range of the wide-angle imaging unit. In addition, the wide-range image data has good image quality because the rotary imaging unit is imaging at a high magnification. Then, the wide range image data can be used instead of the image data captured by the wide-angle imaging unit. As a result, it is possible to prevent the image quality of the image data of the wide-angle imaging unit from being deteriorated due to an increase in the angle of view of the lens of the wide-angle imaging unit.
In addition, as the angle of view of the lens of the wide-angle imaging unit increases, it is not necessary to use an imaging device with a large number of pixels. Therefore, no cost is required.

Further, the entire imaging range of the wide-angle imaging unit is divided into a plurality of imaging ranges, and the rotating imaging unit is rotated by the driving unit, and the plurality of imaging ranges are sequentially imaged on the rotating imaging unit. It is preferable to perform an initial operation process in which the image storage unit stores a plurality of the image data captured by the rotational imaging unit in association with the position information of the image data.
Furthermore, a plurality of the image data stored in the image storage unit in the initial operation processing is synthesized based on the position information of the image data, and image data in a range substantially the same as the imaging range of the wide-angle imaging unit is obtained. It is preferable to produce.
Thereby, in the initial operation, the rotating imaging unit can capture the entire imaging range of the wide-angle imaging unit at a high magnification. For this reason, image data in the same range as the imaging range of the wide-angle imaging unit can always be used instead.

Furthermore, it is preferable to detect a moving body in an image captured by the wide-angle imaging unit by analyzing image data captured by the wide-angle imaging unit. Further, it is preferable to control the driving unit so as to rotate the rotating imaging unit so as to image the moving body.
Thereby, the moving body can be tracked and an image near the moving body can be taken at a high magnification. That is, it is possible to track the moving body and take a detailed image.

Further, based on the moving body image data obtained by the rotating imaging unit imaging the moving body and the position information of the moving body image data, the moving image data is stored in the image storage unit. Preferably, the image data stored in the image storage unit is updated by replacing the image data having the position information corresponding to the position information with the moving body image data.
Thereby, only the part in which the change of the image data memorize | stored in the image memory | storage part in the initial operation process produced by the moving body can be replaced with moving body image data. For this reason, image data that is in the same range as the imaging range of the wide-angle imaging unit and that reflects the latest state can always be used instead.

  According to the present invention, even when the angle of view of the wide-angle imaging unit is increased, it is possible to prevent a reduction in image quality of an image captured by the wide-angle imaging unit without incurring costs.

Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings. Note that the present invention is not limited to the following embodiments.
FIG. 1 shows an example of a binocular camera 100 (imaging device) according to an embodiment of the present invention. The twin-lens camera 100 has a plurality of imaging units. Each imaging unit includes a lens (not shown), an imaging element (not shown), and the like. The imaging device is configured by a solid-state imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor).
Specifically, the binocular camera 100 includes a wide-angle imaging unit 1 and a zoom imaging unit 2 (rotating imaging unit). In FIG. 1, the imaging range of the wide-angle imaging unit 1 is indicated by a solid line, and an example of the imaging range of the zoom imaging unit 2 is indicated by a broken line and a dotted line.

  FIG. 2 is a block diagram illustrating a schematic configuration of the twin-lens camera 100. As shown in FIG. 2, the twin-lens camera 100 further includes a biaxial spherical actuator 3 (drive unit), a controller 4, a memory 5 (image storage unit), an interface 6, a signal processing unit 7, and the like.

  The wide-angle imaging unit 1 has a lens with a wider angle of view than the zoom imaging unit 2. Specifically, the wide-angle imaging unit 1 has a combined lens with an angle of view of 90 degrees as an imaging lens. The wide-angle imaging unit 1 has an image sensor of 1 / 7.4 inch, 640 × 480 pixels (330,000 pixels) as an imaging device.

  The zoom imaging unit 2 has a lens with a narrower angle of view and a larger magnification than the wide-angle imaging unit 1. Specifically, the zoom imaging unit 2 has a combined lens with an angle of view of 30 degrees as an imaging lens. The zoom imaging unit 2 has an image sensor of 1 / 7.4 inch and 640 × 480 pixels (330,000 pixels) as an imaging device. Therefore, as shown in FIG. 1, the imaging range (field of view) of the wide-angle imaging unit 1 is nine times the imaging range of the zoom imaging unit 2. Further, the spatial resolution of the zoom imaging unit 2 is nine times the spatial resolution of the wide-angle imaging unit 1.

  In addition, the zoom imaging unit 2 is rotatably provided to the binocular camera 100. Specifically, the zoom imaging unit 2 is provided in the binocular camera 100 via the biaxial spherical actuator 3. Thereby, the zoom imaging unit 2 can be rotated in the vertical direction and the horizontal direction of the twin-lens camera 100. Further, the rotation range of the zoom imaging unit 2 is 30 degrees in the vertical and horizontal directions of the twin-lens camera 100.

  The biaxial spherical actuator 3 rotates the zoom imaging unit 2 so as to change the imaging range of the zoom imaging unit 2. Specifically, the biaxial spherical actuator 3 includes a biaxial automatic stage (not shown), a motor (not shown) such as a stepping motor or a DC motor (Direct-Current motor), and the like. The zoom imaging unit 2 is connected to the biaxial spherical actuator 3 via the biaxial gonio automatic stage. The biaxial gonio automatic stage is rotatable in the vertical direction and the horizontal direction of the twin-lens camera 100.

The controller 4 controls the operation of the biaxial spherical actuator 3. Then, based on the control of the controller 3, the biaxial spherical actuator 3 rotates the biaxial gonio automatic stage. As a result, the zoom imaging unit 2 mounted on the two-axis goniometer automatic stage rotates.
The biaxial actuator 3 inputs rotation angle data of the biaxial gonio automatic stage to the controller 4. Then, the controller 4 inputs the rotation angle data to the signal processing unit 7.

  The memory 5 stores image data captured by the zoom imaging unit 2 and position information of the image data in association with each other. The position information of the image data is obtained from the rotation angle (rotation state) of the zoom imaging unit 2.

Here, the relationship between the imaging range of the wide-angle imaging unit 1 and the imaging range of the zoom imaging unit 2 is shown in FIG. In FIG. 3, the range indicated by the thick solid line is the imaging range of the wide-angle imaging unit 1. In FIG. 3, the range indicated by the thin solid line is the imaging range of the zoom imaging unit 2.
As shown in FIG. 3, the imaging range of the wide-angle imaging unit 1 is nine times the imaging range of the zoom imaging unit 2. Therefore, when the zoom imaging unit 2 captures the entire imaging range of the wide-angle imaging unit 1, the imaging range of the wide-angle imaging unit 1 is divided into nine as shown in FIG. Then, the zoom imaging unit 2 images nine imaging ranges obtained by dividing the imaging range of the wide-angle imaging unit 1 one by one.

Here, as shown in FIG. 3, nine imaging ranges obtained by dividing the imaging range of the wide-angle imaging unit 1 into nine are set as imaging ranges G1 to G9 from the upper left to the lower right as viewed in the drawing. Since each of the imaging ranges G1 to G9 is detected by an image sensor having 640 × 480 pixels, the number of pixels of the image data in the imaging ranges G1 to G9 is 640 × 480 pixels, respectively. Further, in the memory 5, data for one pixel of the image data is stored with a capacity of 1 bit of the memory 5, and the image data is stored in the order of the imaging ranges G1 to G9. Thereby, for example, the image data of the imaging range G1 is stored in the addresses 1 to 640 × 480 of the memory 5. Similarly, the image data of the imaging range G2 is stored at addresses 640 × 480 + 1 to 640 × 480 × 2 of the memory 5.
Of course, the memory 5 may store data for one pixel with a capacity of several bits of the memory 5.

  When the zoom imaging unit 2 captures the imaging range G1, the zoom imaging unit 2 is rotated 30 degrees upward and 30 degrees leftward. Therefore, based on the rotation angle of the zoom imaging unit 2, it is possible to obtain the position information of the image data being captured by the zoom imaging unit. Specifically, when the zoom imaging unit 2 is rotated 30 degrees upward and 30 degrees to the left, the position information of the image data captured by the zoom imaging unit 2 is the imaging range G1. Furthermore, the address of the memory 5 in which the image data of the imaging range G1 is stored is 1 to 640. Therefore, the rotation angle of the zoom imaging unit 2 is associated with the imaging ranges G1 to G9 in advance, and the rotation of the zoom imaging unit 2 is performed by associating the imaging ranges G1 to G9 with the addresses of the memory 5. The imaging range and the address of the memory 5 can be obtained from the angle. In other words, the imaging range as the position information of the image data can be obtained from the rotation angle of the zoom imaging unit 2. Further, the address of the memory 5 where the image data having the position information is stored can be obtained from the imaging range as the position information. Accordingly, by storing the image data in advance in the address of the memory 5 associated with the imaging range of the image data, the image data and the position information of the image data can be associated with each other and stored in the memory 5. .

  The interface 6 is a connection unit that connects the signal processing unit 7 of the binocular camera 100 and an external device connected to the binocular camera 100. For example, the interface 6 connects an input unit (not shown) such as a keyboard or a display unit (not shown) such as an LCD (Liquid Crystal Display) to the binocular camera 100. Then, recognition information, an initialization command, a detailed wide-angle image request command, and the like are input from the input unit to the signal processing unit 7 of the binocular camera 100 via the interface 6. Further, image data is input to the display unit from the signal processing unit 7 of the twin-lens camera 100 via the interface 6.

The signal processing unit 7 has an FPGA (Field Programmable Gate Array) and stores various programs for controlling each unit of the binocular camera 100. And the signal processing part 7 controls each part of the binocular camera 100 by executing the various programs. The signal processing unit 7 may be configured by an LSI (Large Scale Integration) or the like. When the signal processing unit 7 is configured by an FPGA or the like, various programs stored in the signal processing unit 7 can be rewritten.
As shown in FIG. 2, the signal processing unit 7 includes an image recognition unit 71, an actuator control unit 72, an initialization operation control unit 73, an image update control unit 74, a detailed wide-angle image composition unit 75, and the like.

  The image recognizing unit 71 detects the moving object in the image captured by the wide-angle imaging unit 1 by analyzing the image data captured by the wide-angle imaging unit 1. Specifically, for example, the image recognition unit 71 first detects a difference between the image data of the current frame and the image data of the previous frame. Next, the image recognition unit 71 detects a portion where the difference is a predetermined value or more as a moving body.

  The actuator control unit 72 controls the biaxial spherical actuator 3 to rotate the zoom imaging unit 2 so as to image the moving body detected by the image recognition unit 71. Specifically, the actuator control unit 72 inputs a control signal for rotating the zoom imaging unit 2 to the controller 4 so as to image the moving body detected by the image recognition unit 71. The controller 4 calculates an angle (rotation angle) for rotating the zoom imaging unit 2 based on the control signal. Next, the controller 4 rotates the biaxial goniometer automatic stage by the angle. Thereby, the zoom imaging unit 2 is rotated.

The initialization operation control unit 73 divides the entire imaging range of the wide-angle imaging unit 1 into a plurality of imaging ranges, and rotates the zoom imaging unit 2 by the biaxial spherical actuator 3 to zoom the plurality of imaging ranges. The image pickup unit is made to pick up images sequentially. Next, the initialization operation control unit 73 stores a plurality of image data captured by the zoom imaging unit 2 in the memory 5 in association with the position information of the image data. Specifically, as shown in FIG. 3, the memory 5 stores the image data of the imaging range G1 in association with G1 as position information. More specifically, the imaging ranges G1 to G9 and the addresses of the memory 5 in which image data of the imaging ranges G1 to G9 are stored are associated in advance. For example, the image data of the imaging range G1 is stored at addresses 1 to 640 × 480. Similarly, the image data of the imaging range G2 is stored at addresses 640 × 480 + 1 to 640 × 480 × 2 of the memory 5. Therefore, the initialization operation control unit 73 stores the image data at the address of the memory 5 associated in advance with the position information (imaging range) of the image data, thereby obtaining the image data and the position information of the image data. The data are stored in the memory 5 in association with each other.
The initialization operation control unit 73 divides the imaging range of the wide-angle imaging unit 1 into a plurality of imaging ranges based on the angle of view of the wide-angle imaging unit 1 and the angle of view of the zoom imaging unit 2.

  The image update control unit 74 includes image data of an imaging range including the moving body (hereinafter referred to as moving body image data) obtained by the zoom imaging unit 2 capturing an image of the moving body, and the moving body image data. The image data stored in the memory 5 is updated based on the position information. Specifically, the image update control unit 74 is stored in the memory 5 by replacing image data having position information corresponding to the position information of the moving object image data with the moving object image data in the memory 5. Update existing image data. More specifically, the image update control unit 74 calculates the address of the memory 5 in which the moving body image data is stored from the imaging range of the moving body image data. Next, the image update control unit 74 replaces the image data stored at the address with the moving body image data. As a result, the image data in the memory 5 is updated.

  The detailed wide-angle image combining unit 75 generates a wide range of image data by combining a plurality of image data based on a plurality of image data having different imaging ranges stored in the memory 5 and position information of the image data. To do. Specifically, the detailed wide-angle image composition unit 75 determines an imaging range as position information from the address of the memory 5 and assigns image data stored at the address to the imaging range. Thereby, image data is assigned to each imaging range, and a wide range of image data is generated. More specifically, as shown in FIG. 3, the detailed wide-angle image composition unit 75 assigns the image data stored at addresses 1 to 640 × 480 of the memory 5 to the imaging range G1. Similarly, the detailed wide-angle image composition unit 75 assigns the image data stored in the addresses 640 × 480 + 1 to 640 × 480 × 2 of the memory 5 to the imaging range G2. In this way, the detailed wide-angle image composition unit 75 assigns the image data stored in the memory 5 to the imaging ranges G1 to G9, and synthesizes a wide range of image data.

Next, an image processing method of the twin-lens camera 100 according to the present embodiment will be described. The image processing according to the present embodiment includes initial operation processing described below, memory 5 update processing, detailed wide-angle image data (described later) composition processing, and the like.
First, an initial operation process in the twin-lens camera 100 according to the present embodiment will be described with reference to a flowchart shown in FIG. In the following description, the position information of the image data in the imaging ranges G1 to G9 is represented by an x value, and the address of the memory 5 in which the image data is stored is represented by a y value. For example, the value of x representing the position information of the image data in the imaging range G1 is 1, and the value of y representing the address of the memory 5 in which the image data is stored is 1 to 640 × 480. Similarly, the value of x representing the position information of the image data in the imaging range G2 is 2, and the value of y representing the address of the memory 5 in which the image data is stored is 640 × 480 + 1 to 640 × 480 × 2. .

First, the initial operation control unit 73 sets x and y as x = 1 and y = 1 to 640 × 480, respectively (step S1).
Next, the initial operation control unit 73 rotates the zoom imaging unit 2 with the biaxial spherical actuator 3 to cause the zoom imaging unit to image the imaging range G1 (step S2).
Next, the initial operation control unit 73 stores the image data acquired in step S2 at addresses y = 1 to 640 × 480 of the memory 5 (step S3).

  Next, the initial operation control unit 73 sets a new value of x by adding 1 to the current value of x. At the same time, the initial operation control unit 73 sets a new y value by adding 640 × 480 to the current y value (step S4).

Next, the initial operation control unit 73 determines whether or not the new value of x obtained in step S4 is greater than 9 (step S5).
In step S5, when the initial motion control unit 73 determines that the new value of x obtained in step S4 is 9 or less (step S5; No), the process returns to step S2.
In step S5, when the initial motion control unit 73 determines that the new value of x obtained in step S4 is greater than 9 (step S5; Yes), the process ends.

Next, update processing of the memory 5 in the twin-lens camera 100 according to the present embodiment will be described with reference to the flowchart shown in FIG.
First, the image recognition unit 71 analyzes the image data captured by the wide-angle imaging unit 1 to determine whether or not there is a motion in the image captured by the wide-angle imaging unit 1 (whether there is a moving body). Is determined (step S101).

In step S101, when the image recognition unit 71 determines that there is no movement (no moving body) in the image captured by the wide-angle imaging unit 1 (step S101; No), the process of step S101 is performed again. .
In step S101, when the image recognition unit 71 determines that there is a motion (there is a moving body) in the image captured by the wide-angle imaging unit 1 (step S101; Yes), the actuator control unit 72 The imaging range of the zoom imaging unit 2 is controlled so as to image the moving object detected by the recognition unit 71 (step S102).

Next, the zoom imaging unit 2 acquires image data of the imaging range including the moving body detected by the image recognition unit 71 (hereinafter referred to as moving body image data) (step S103).
Next, the image update control unit 74 updates the image data stored in the memory 5 based on the moving object image data obtained in step S103 and the position information of the moving object image data (step S104). ), This process is terminated.

Next, a process of synthesizing detailed image data (hereinafter referred to as detailed wide-angle image data) in the same range as the imaging range of the wide-angle imaging unit in the binocular camera 100 according to the present embodiment is shown in FIG. This will be described with reference to a flowchart.
First, the detailed wide-angle image composition unit 75 sets the value of x to 1 (step S201).
Next, the detailed wide-angle image composition unit 75 reads the image data at addresses y = 1 to 640 × 480 in the memory 5 and arranges the image data at the position of x = 1, that is, the position of the image range G1 (step S202).

Next, the detailed wide-angle image composition unit 75 sets a new value of x by adding 1 to the current value of x (step S203).
Next, the detailed wide-angle image composition unit 75 determines whether or not the new value of x obtained in step S203 is greater than 9 (step S204).

  In step S204, when the detailed wide-angle image composition unit 75 determines that the new value of x obtained in step S203 is 9 or less (step S204; No), the address y of the memory 5 = 640 × 480. Image data of × (x−1) +1 to 640 × 480 × (x) is read and arranged at the position of x (step S205).

  In step S204, when the detailed wide-angle image composition unit 75 determines that the new value of x obtained in step S203 is greater than 9 (step S204; Yes), the process ends.

As described above, in the binocular camera 100 and the image processing method of the binocular camera 100 according to the present embodiment, the zoom imaging unit 2 captures image data in a different imaging range at a higher magnification than the wide-angle imaging unit 1. Take multiple images. Further, the detailed wide-angle image synthesis unit 75 synthesizes a plurality of pieces of image data with different imaging ranges captured at a high magnification to generate a wide range of image data. Therefore, it is possible to generate detailed wide-angle image data by synthesizing a plurality of image data obtained by imaging the zoom imaging unit 2 by dividing the imaging range of the wide-angle imaging unit 1 into a plurality of ranges. The detailed wide-angle image data is image data in substantially the same range as the imaging range of the wide-angle imaging unit 1. The detailed wide-angle image data has good image quality because the zoom imaging unit 2 captures images at a high magnification. The detailed wide-angle image data can be used instead of the image data captured by the wide-angle imaging unit 1. As a result, it is possible to prevent the image quality of the image data of the wide-angle imaging unit 1 from being deteriorated due to an increase in the field angle of the lens of the wide-angle imaging unit 1.
In addition, as the angle of view of the lens of the wide-angle imaging unit 1 increases, it is not necessary to use an imaging element with a large number of pixels. Therefore, no cost is required.

The initial operation control unit 73 controls initial operation processing. Specifically, the initial operation control unit 73 divides the entire imaging range of the wide-angle imaging unit 1 into a plurality of imaging ranges, rotates the zoom imaging unit 2 with the biaxial spherical actuator 3, and The zoom imaging unit 2 sequentially captures the imaging range. Further, the initial operation control unit 73 stores a plurality of image data captured by the zoom imaging unit 2 in the memory 5 in association with position information of the image data.
Furthermore, the detailed wide-angle image composition unit 75 synthesizes a plurality of image data stored in the memory 5 based on the position information of the image data in the initial operation process. As a result, detailed wide-angle image data in the same range as the imaging range of the wide-angle imaging unit 1 is generated.
Thereby, in the initial operation, the zoom imaging unit 1 can capture the entire imaging range of the wide-angle imaging unit 2 at a high magnification. Therefore, detailed wide-angle image data in the same range as the imaging range of the wide-angle imaging unit 1 can always be used instead.

Furthermore, the binocular camera 100 includes an image recognition unit 71 and an actuator control unit 72. The image recognizing unit 71 detects the moving object in the image captured by the wide-angle imaging unit 1 by analyzing the image data captured by the wide-angle imaging unit 1. In addition, the actuator control unit 72 controls the biaxial spherical actuator 3 to rotate the zoom imaging unit 2 so as to image the moving body detected by the image recognition unit 71.
Thereby, the moving body can be tracked and an image near the moving body can be taken at a high magnification. That is, it is possible to track the moving body and take a detailed image.

The binocular camera 100 also includes an image update control unit 74. The image update control unit 74 uses the moving body image data obtained by the zoom imaging unit 2 to pick up the moving body and the position information of the moving body image data in the memory 5 based on the position information of the moving body image data. The image data stored in the memory 5 is updated by replacing the image data having the position information corresponding to the mobile object image data.
Thereby, only the part in which the change of the image data stored in the memory 5 has occurred in the initial operation process can be replaced by the moving object image data. For this reason, it is possible to always use the detailed wide-angle image data that is the image data in substantially the same range as the imaging range of the wide-angle imaging unit 1 and reflects the latest state.

It is a perspective view which shows an example of the twin-lens camera concerning embodiment of this invention. It is a block diagram which shows schematic structure of the twin-lens camera concerning embodiment of this invention. It is a figure which shows the relationship between the imaging range of a wide angle imaging unit and the imaging range of a zoom imaging unit in embodiment of this invention. It is a flowchart explaining the initial stage operation process concerning embodiment of this invention. It is a flowchart explaining the update process of the memory concerning embodiment of this invention. It is a flowchart explaining the synthetic | combination process of the detailed wide-angle image data concerning embodiment of this invention.

Explanation of symbols

1 Wide-angle imaging unit 2 Zoom imaging unit (rotating imaging unit)
3 Biaxial spherical actuator (drive unit)
5 Memory (image storage unit)
71 Image recognition unit 72 Actuator control unit (drive control unit)
73 Initialization control unit 74 Image update control unit 75 Detailed wide-angle image composition unit (image composition unit)
100 binocular camera (imaging device)
G1-G9 imaging range

Claims (10)

An imaging apparatus including a plurality of imaging units each including a lens and an imaging element,
At least one of the plurality of imaging units is a wide-angle imaging unit having the lens having a wider angle of view than the other imaging units,
At least one of the plurality of imaging units is a rotating imaging unit that includes the lens having a narrower angle of view and a larger magnification than the wide-angle imaging unit, and is provided rotatably on the imaging device.
A drive unit that rotates the rotational imaging unit so as to change the imaging range of the rotational imaging unit;
An image storage unit that stores the image data captured by the rotational imaging unit and the positional information of the image data obtained from the rotational state of the rotational imaging unit in association with each other;
An image composition unit that synthesizes the plurality of image data based on the plurality of image data having different imaging ranges stored in the image storage unit and the position information of the image data, and generates a wide range of image data; ,
An imaging apparatus comprising:   The entire imaging range of the wide-angle imaging unit is divided into a plurality of imaging ranges, the rotating imaging unit is rotated by the driving unit, and the plurality of imaging ranges are sequentially imaged by the rotating imaging unit, The image storage unit includes an initial operation control unit that controls an initial operation process of storing a plurality of the image data captured by the rotational imaging unit in association with the position information of the image data. The imaging device described.   The image synthesizing unit synthesizes the plurality of image data stored in the image storage unit in the initial operation process based on the position information of the image data, and substantially the same range as the imaging range of the wide-angle imaging unit. The imaging device according to claim 2, wherein the image data is generated. An image recognition unit for detecting a moving body in an image captured by the wide-angle imaging unit by analyzing image data captured by the wide-angle imaging unit;
A drive control unit that controls the drive unit to rotate the rotation imaging unit so as to image the moving body detected by the image recognition unit;
An imaging apparatus according to any one of claims 1 to 3.   Based on the moving object image data obtained by the rotating imaging unit imaging the moving object and the position information of the moving object image data, the position of the moving object image data in the image storage unit. The imaging according to claim 4, further comprising an image update control unit configured to update the image data stored in the image storage unit by replacing the image data having the position information corresponding to information with the moving body image data. apparatus. An image processing method in an imaging apparatus including a plurality of imaging units each including a lens and an imaging element,
At least one of the plurality of imaging units is a wide-angle imaging unit having the lens having a wider angle of view than the other imaging units,
At least one of the plurality of imaging units is a rotating imaging unit that includes the lens having a narrower angle of view and a larger magnification than the wide-angle imaging unit, and is provided rotatably on the imaging device.
The imaging device includes a drive unit that rotates the rotating imaging unit so as to change an imaging range of the rotating imaging unit;
An image storage unit that stores the image data captured by the rotational imaging unit and the positional information of the image data obtained from the rotational state of the rotational imaging unit in association with each other;
An image processing method for generating a wide range of image data by combining a plurality of the image data based on the plurality of image data having different imaging ranges stored in the image storage unit and the position information of the image data.   The entire imaging range of the wide-angle imaging unit is divided into a plurality of imaging ranges, the rotating imaging unit is rotated by the driving unit, and the plurality of imaging ranges are sequentially imaged by the rotating imaging unit, The image processing method according to claim 6, wherein an initial operation process is performed in which the image storage unit stores a plurality of the image data captured by the rotational imaging unit in association with the position information of the image data.   A plurality of the image data stored in the image storage unit in the initial operation process is synthesized based on the position information of the image data to generate image data in a range substantially the same as the imaging range of the wide-angle imaging unit. The image processing method according to claim 7. By analyzing the image data captured by the wide-angle imaging unit, a moving object in the image captured by the wide-angle imaging unit is detected,
The image processing method according to any one of claims 6 to 8, wherein the drive unit is controlled to rotate the rotation imaging unit so as to image the moving body.   Based on the moving object image data obtained by the rotating imaging unit imaging the moving object and the position information of the moving object image data, the position of the moving object image data in the image storage unit. The image processing method according to claim 9, wherein the image data stored in the image storage unit is updated by replacing the image data having the position information corresponding to information with the moving body image data.

Images