JP2021180455A - Device and method for controlling imaging apparatus, and program - Google Patents

Abstract

To create a smooth image in an imaging apparatus comprising a driving unit that changes an imaging direction of an imaging unit.SOLUTION: A control device controls an imaging apparatus 100 comprising an imaging unit and driving units (pan motor 106 and tilt motor 107) that change an imaging direction of the imaging unit, and comprises: an image processing unit 103 that executes segmentation processing on image data; and a system control unit 104 that, during the stoppage of the driving units, controls the image processing unit 103 to execute the segmentation processing on imaging data acquired from the imaging unit, and during the drive of the driving units, controls the image processing unit 103 to execute the segmentation processing on imaging data before the drive stored in a storage unit 108 that stores imaging data acquired by the imaging unit.SELECTED DRAWING: Figure 1

Description

The present invention relates to a control device, a method and a program for controlling an image pickup device.

In recent years, imaging devices including a surveillance camera and a streaming camera, which are provided with a drive unit that changes the imaging direction in the pan direction or the tilt direction, are known. A stepping motor may be used as the drive unit of the image pickup apparatus from the viewpoint of ease of position control and the like.
In such an image pickup device, when an image is taken while rotating the stepping motor at a low speed, a moving image may be shaken due to a power transmission mechanism such as a stepping motor or a gear.
For example, in Patent Document 1, the required performance at high speed rotation and low speed rotation is satisfied by adjusting the pressing force for removing the backlash of the power transmission mechanism for pan rotation or tilt rotation of the camera unit. An image pickup device has been proposed.

Japanese Unexamined Patent Publication No. 2012-225949

An object to be solved by the present invention is to enable a smooth image to be generated in an image pickup apparatus provided with a drive unit for changing the image pickup direction of the image pickup unit.

The control device according to one aspect of the present invention is a control device that controls an image pickup device including an image pickup unit and a drive unit that changes the image pickup direction of the image pickup unit, and is an image that performs cutout processing on image data. While the processing means and the drive unit are stopped, the image pickup data acquired by the image pickup unit is cut out by the image processing means, and while the drive unit is being driven, the image pickup acquired by the image pickup unit is performed. It is characterized by including a control means for controlling the imaged data before driving stored in the storage unit for storing the data so as to perform the cutting process by the image processing means.

It is a figure which shows the structure of the image pickup apparatus which concerns on embodiment. It is an external view of the image pickup apparatus which concerns on embodiment. It is a characteristic diagram which shows an example of the angle change of a pan rotation part with respect to a speed command. It is a figure for demonstrating an example of an imaging range. It is a figure for demonstrating the cut-out process carried out by the image pickup apparatus which concerns on embodiment. It is a flowchart which shows the process which the image pickup apparatus which concerns on embodiment perform.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.
FIG. 1 is a diagram showing a configuration example of the image pickup apparatus 100 according to the embodiment.
The image pickup device 100 includes an image pickup optical system 101, an image pickup element 102, an image processing unit 103, a system control unit 104, a drive control unit 105, a pan motor 106, a tilt motor 107, a storage unit 108, and a communication unit. It is equipped with 109. In the present embodiment, an example in which the image pickup apparatus 100 incorporates a control apparatus to which the present invention is applied will be described.

The image pickup optical system 101 includes a plurality of lenses (not shown) and a holding member (not shown). The configuration of the imaging optical system 101 is not limited to this, and may be configured to include, for example, a lens driving motor so that the zoom magnification and the focus position can be adjusted, and visible light or infrared light. It may be configured so that light containing a plurality of wavelength components such as light can be focused.
The image pickup device 102 is an image pickup device having sensitivity to light in the visible light region. The light transmitted through the image pickup optical system 101 is imaged by the image pickup element 102 and converted into an electrical image pickup signal. The configuration of the image pickup device 102 is not limited to this, and may be one having sensitivity in an invisible light region such as infrared light.

The image processing unit 103 receives the image pickup signal output by the image pickup element 102, performs various image processes such as development processing, color balance processing, gamma processing, and noise reduction processing, and generates image data. Further, the image processing unit 103 can perform cutting processing on the image data.

The pan motor 106 and the tilt motor 107 are drive units that change the image pickup direction of the image pickup unit composed of the image pickup optical system 101 and the image pickup element 102. The pan motor 106 and the tilt motor 107 can be driven by the drive control unit 105 based on the instructions of the system control unit 104. A stepping motor is used for the pan motor 106 and the tilt motor 107, but the stepping motor is not limited to this, and may be configured by, for example, a brushless motor or the like.
The drive control unit 105 is composed of an electric circuit for driving the motors 106 and 107, such as an H-bridge circuit.

The system control unit 104 performs comprehensive control of each component of the image pickup apparatus 100, setting of various parameters, and the like.
The storage unit 108 is composed of a RAM, a FLASH (registered trademark) memory, and the like. The storage unit 108 can temporarily store and read the image data output from the image processing unit 103. The storage unit 108 is also used as a storage area for a program executed by the system control unit 104 and a work area during program execution. Further, the storage unit 108 is also used for the system control unit 104 to store initial values of various parameters for controlling each component of the image pickup apparatus 100.

The communication unit 109 converts the image data in accordance with the communication protocol and then distributes the image data to the client device 121 via the LAN 120. Communication unit 109 is H.I. 264 and H. Performs compression coding processing such as 265. Further, the communication unit 109 receives commands for setting various parameters for the image pickup device 100 from the client device 121 and outputs them to the system control unit 104, and at the same time, transmits a response to the client device 121.

The LAN (Local Area Network) 120 on the network is composed of routers, switches, cables and the like that satisfy communication standards such as Ethernet (registered trademark). The image pickup device 100 can be connected to the client device 121, another image pickup device (not shown), a server, or the like via the LAN 120. The LAN 120 is not limited in its communication standard, scale, configuration, etc., as long as it can communicate between the image pickup device 100 and the client device 121. For example, the LAN may be configured by a wired LAN, a wireless LAN, a WAN (Wide Area Network), or the like. It may also be configured via the cloud on the network.

The client device 121 is composed of a personal computer, a mobile terminal, or the like. The client device 121, which is an external device, is communicably connected to the image pickup device 100 via the LAN 120, and can receive, display, and record image data from the image pickup device 100. Further, the client device 121 can transmit various setting commands for controlling the image pickup device 100. For example, the pan motor 106 and the tilt motor 107 can be operated by the Joyce Tech (not shown) connected to the client device 121 or the GUI on the display screen to change the image pickup direction of the image pickup device 100. Further, the client device 121 can acquire the information of the image pickup device 100 by receiving the response output by the communication unit 109.

2A and 2B are external views of the image pickup apparatus 100, where FIG. 2A shows a side view and FIG. 2B shows a top view. The image pickup apparatus 100 includes a fixed portion 201, a pan rotating portion 202 supported by the fixed portion 201, and a tilt rotating portion 203 supported by the pan rotating portion 202. The pan rotation unit 202 rotates using the pan motor 106 as a drive source. Further, the tilt rotation unit 203 rotates using the tilt motor 107 as a drive source. The tilt rotation unit 203 is equipped with an image pickup optical system 101 and an image pickup element 102 that form an image pickup unit.

In the image pickup device 100 as described above, when the pan motor 106 rotates, the pan rotating portion 202 rotates in the arrow 204 direction with respect to the fixed portion 201 via a power transmission mechanism having a predetermined reduction ratio. Similarly, when the tilt motor 107 rotates, the tilt rotating portion 203 rotates in the arrow 205 direction with respect to the pan rotating portion 202 via a power transmission mechanism having a predetermined reduction ratio. By rotating the pan motor 106 and the tilt motor 107 in this way, it is possible to change the imaging direction of the imaging unit to a desired direction.

FIG. 3 is a characteristic diagram showing an example of an angle change of the pan rotation unit 202 with respect to the speed command α input to the image pickup apparatus 100, where the horizontal axis shows the time [seconds] and the vertical axis shows the angle [°]. .. The characteristic line 301 shown by the dotted line is a graph of the speed command α [° / sec] for the pan motor 106. Further, the characteristic line 302 shown by a solid line is a graph of the actual angle change [°] of the pan rotating portion 202. Although the pan rotation unit 202 will be described here, the same applies to the tilt rotation unit 203.

The speed command α is output from, for example, a value according to the user operation from the client device 121, and is received by the system control unit 104 of the image pickup device 100. The generation of the speed command α is not limited to the above method, and may be, for example, a method of executing a drive program stored in advance in the storage unit 108.
Upon receiving the speed command α, the system control unit 104 calculates the movement interval T1 from the angular resolution θ1 of the pan rotation unit 202 according to the equation (1).
T1 = θ1 ÷ α… (1)

The angle resolution θ1 of the pan rotation unit 202 can be calculated by the equation (2) from the step angle β, which is the movement angle per unit pulse signal of the pan motor 106, and the reduction ratio γ of the power transmission mechanism.
θ1 = β ÷ γ… (2)
For example, when the step angle β is 3.6 ° and the reduction ratio γ is 10 times, the angular resolution θ1 of the pan rotating portion 202 is 0.36 °. The step angle β and the reduction ratio γ are parameters stored in advance in the storage unit 108, and can be changed to arbitrary values according to the configuration of the image pickup apparatus 100.

After that, the pan rotation unit 202 moves to the next angle θ (n) represented by the equation (4) at each time t = T (n) represented by the equation (3). n is an integer of 1 or more.
T (n) = T (n-1) + T1 ... (3)
θ (n) = θ1 × n ... (4)
For example, when the speed command α is 1.44 ° / sec, the movement interval T1 is calculated to be 0.25 seconds from the equation (1). Then, the pan rotation unit 202 moves in the pan direction, with the angle θ1 at time t = T1 = 0.25 seconds being 0.36 ° and the angle θ2 at time t = T2 = 0.5 seconds being 0.72 °. Will be done. The speed command α is not limited to the above-mentioned value, and may be any value.

Here, when a pulse signal, which is a drive signal, is applied from the system control unit 104 to the drive control unit 105, the drive control unit 105 starts moving the pan motor 106 to the next step angle according to the pulse signal. The section 303 is a section indicating the travel time τ in which the pan motor 106 is being driven. The movement time τ is a predetermined time from the application of the pulse signal to the drive control unit 105 to the end of the change in the image pickup direction of the image pickup unit. Specifically, the movement time τ is a time obtained by adding the time from when the pulse signal is applied to the drive control unit 105 until the pan motor 106 starts to move, the time when the pan motor 106 moves, and the settling time. The settling time is the time required for the overshoot and ringing that occur after the pan motor 106 has moved to sufficiently converge. The travel time τ is a time that can be predicted by measuring the pulse response in advance, and is stored in the storage unit 108 as a predetermined parameter, for example, 0.12 seconds. However, this is not the case, and the travel time τ may be updated sequentially by performing a predetermined parameter adjustment operation.

After the pulse signal is applied from the system control unit 104 to the drive control unit 105 in this way, there is a delay time defined by the movement time τ from the time when the pan rotation unit 202 stably stops at the next angle. do. Therefore, the pulse signal is output from the system control unit 104 at the time represented by the equation (5) (hereinafter referred to as the pulse signal application time) Tp (n).
Tp (n) = T (n) −τ… (5)

Hereinafter, with reference to FIGS. 4 and 5, an output image when a pulse signal is applied from the system control unit 104 to the drive control unit 105 and the pan rotation unit 202 rotates will be described.
FIG. 4 is a diagram for explaining an example of the image pickup range of the image pickup apparatus 100, and FIG. 400 shows a landscape over a wide area including the image pickup range of the image pickup apparatus 100. 401 and 402 are the image pickup range of the image pickup apparatus 100, and the image pickup range 402 is the image pickup range when the pan rotation unit 202 is moved from the image pickup range 401 by the angle resolution θ1. The distance 403 indicates the moving distance between the imaging range 401 and the imaging range 402.

Here, when the imaging data in the imaging range is used as the output image as it is, when the speed command α = 1.44 ° / sec described above is input, the imaging direction changes only every time T1 = 0.25. , Discontinuous moving images are displayed. In addition, during the section 303 shown in FIG. 3, the moving image may fluctuate due to overshoot, ringing, or the like of the pan rotating portion 202.

Therefore, as described in detail below, the image pickup apparatus 100 performs a cutting process to generate image data to be an output image. FIG. 5 is a diagram for explaining the cutting process performed by the image pickup apparatus 100. In FIG. 5, as in FIG. 4, a moving image when the image pickup range of the image pickup apparatus 100 moves from the image pickup range 401 to the image pickup range 402 based on the speed command α is taken as an example. The cutout region of the cutout process by the image processing unit 103 can be set more finely than the change of the image pickup region that changes with the angular resolution.

Based on the speed command α, the system control unit 104 calculates the movement interval T1 to the imaging range 402 from the equation (1), and then calculates the pulse signal application time Tp1 from the equation (5). When the speed command α is 1.44 ° / sec and the movement interval T1 is 0.25 seconds, the pulse signal application time Tp1 is 0.13 seconds.

During time t <Tp1, the pan motor 106 is stopped, and the image pickup range of the image pickup apparatus 100 is the image pickup range 401. 5 (A) to 5 (C) show output images 501 to 503 during time t <Tp1. FIG. 5A shows image data 501 generated by performing a cutting process on the imaging data 401-a at time t = a. Further, FIG. 5B shows image data 502 generated by performing a cutting process on the imaging data 401-b at time t = b (a <b). Further, FIG. 5C shows image data 503 generated by performing a cutting process on the imaging data 401-c at time t = c (b <c). As shown in FIGS. 5A to 5C, during the time t <Tp1 when the pan motor 106 is stopped and stable imaging can be performed, the cutting position is set for each imaging frame based on the speed command α. Adjustments are made. For example, when the speed command α is 1.44 ° / sec, the time a is 0 seconds, and the time b is 0.05 seconds, the cutout position difference between the image data 501 and the image data 502 pseudo-pan-rotates the pan rotation unit 202. It corresponds to the movement distance that occurs when rotated by 0.072 °. Similarly, when the time c is 0.1 second, the cutout position difference between the image data 502 and the image data 503 corresponds to the movement distance generated when the pan rotation unit 202 is rotated by 0.072 ° in a pseudo manner. ..

In this way, when the speed command α is received, while the pan motor 106 is stopped, the cutout area is adjusted for each image pickup frame, so that even if the angular resolution of the pan rotation unit 202 is large, the speed command α is pseudo. It becomes possible to deliver a smooth moving image according to the situation.
It is assumed that each image pickup data at time t <Tp1 is stored in the storage unit 108. The saving method may be overwrite saving.

Next, when the time t = Tp1, the system control unit 104 outputs a pulse signal to the drive control unit 105, and the pan motor 106 starts driving so as to rotate at the step angle β. After that, during Tp1 <time t <T1, the cutout region is obtained with respect to the image pickup data of the last image pickup frame at time t <Tp1 stored in the storage unit 108 (that is, the image pickup data immediately before outputting the pulse signal). The image data is generated by performing the cutting process while changing the above. 5 (D) and 5 (E) show output images 504 and 505 during Tp1 <time t <T1. FIG. 5D shows image data 504 generated by performing a cutout process on the image pickup data 401-c of the last image pickup frame at time t <Tp1 stored in the storage unit 108 at time t = d. show. Further, in FIG. 5E, at time t = e (d <e), the image pickup data 401-c of the last image pickup frame at time t <Tp1 stored in the storage unit 108 is cut out. The image data 505 generated in the above is shown. By calculating the number of image pickup frames required during the movement time τ, the number of images required during Tp1 <time t <T1 is known, and the cutout region is determined based on the speed command α. For example, when the speed command α is 1.44 ° / sec, the time d is 0.15 seconds, and the time e is 0.2 seconds, the cutout position difference between the image data 504 and the image data 505 is a pseudo pan rotation unit. It corresponds to the movement distance generated when the 202 is rotated by 0.072 °.

Next, when the time t = T1, the pan rotation unit 202 finishes moving to the position where the image pickup range 402 is imaged, and the image pickup range of the image pickup apparatus 100 becomes the image pickup range 402. FIG. 5F shows an output image 506 at time f corresponding to time t = T1. In this case, the image data 402-f at time t = f is cut out to generate image data 506. The image data 506 is obtained by moving the distance generated when the pan rotation unit 202 is rotated by 0.072 ° with respect to the image data 505.
Here, the process until the time t = T1 is reached has been described, but the same applies to the subsequent T2, T3, ....

As described above, while the pan motor 106 is stopped, the image pickup data at each time is cut out, and while the pan motor 106 is being driven, the last image pickup data before the drive is stored in the storage unit 108. The cutout process is performed, and the image data whose cutout area is changed according to the speed command α is distributed. As a result, even when a relatively slow speed command α is input in an image pickup device having a large angle resolution of the pan rotation unit 202, the angle resolution of the pan rotation unit 202 is pseudo-increased and smooth according to the speed command α. It is possible to generate a moving image.
Although the pan motor 106 has been described here, it goes without saying that the same applies to the tilt motor 107.

FIG. 6 is a flowchart showing a process executed by the image pickup apparatus 100 according to the embodiment. Each process in the flowchart of FIG. 6 is realized by the system control unit 104 executing the program stored in the storage unit 108.
For example, when the system control unit 104 receives the speed command, the process of this flowchart is started.
In step S1, the system control unit 104 calculates various parameters. Specifically, the system control unit 104 calculates the equations (1) to (5) from the speed command α and each parameter stored in the storage unit 108.

In step S2, the system control unit 104 determines whether or not the pan motor 106 and the tilt motor 107 are stopped. While the pan motor 106 is stopped, as shown in FIG. 3, it is the time obtained by subtracting the travel time τ from the off time of the pulse signal. The same applies to the tilt motor 107. If the system control unit 104 determines that the motor is stopped, the process proceeds to step S3. While the motor is stopped, it is before the pulse signal is applied, and the image pickup apparatus 100 is stationary and can take an image in a stable state. On the other hand, if the system control unit 104 determines that the motor is not stopped, that is, if it is determined that the motor is being driven, the process proceeds to step S5. The state in which the motor is not stopped means that the pulse signal application time Tp (n) has been reached.

In step S3, the system control unit 104 acquires the imaging data in the current imaging range.
In step S4, the system control unit 104 stores the image pickup data acquired in step S3 in the storage unit 108.

In step S5, the system control unit 104 determines whether or not the pulse signal application time Tp (n) is reached. If the system control unit 104 determines that the pulse signal application time Tp (n), the process proceeds to step S6. If the system control unit 104 determines that the pulse signal application time Tp (n) is not set, the process proceeds to step S7.
In step S6, the system control unit 104 outputs a pulse signal to the target motor to the drive control unit 105.
In step S7, the system control unit 104 reads out the imaging data stored in the storage unit 108 immediately before outputting the pulse signal.

In step S8, the system control unit 104 determines a cutout region for the target image pickup data based on the speed command α, and performs a cutout process so as to cut out the cutout region. As described with reference to FIG. 5, while the motor is stopped, the imaging data at each time is cut out, and while the motor is being driven, the last imaging data before driving stored in the storage unit 108 is cut out. Carry out the process.
In step S9, the system control unit 104 distributes the image data cut out in step S8.
In step S10, the system control unit 104 determines whether or not the drive stop command has been received. When it is determined that the system control unit 104 has received the drive stop command, this flowchart processing ends. If the system control unit 104 determines that the drive stop command has not been received, the process returns to step S2.

As described above, while the motor is stopped, the imaging data at each time is cut out, and while the motor is being driven, the last imaging data before driving stored in the storage unit 108 is cut out. This is performed, and the image data whose cutout area is changed according to the speed command α is distributed. This makes it possible to generate a smooth image in an image pickup apparatus provided with a drive unit that changes the image pickup direction of the image pickup unit. Moreover, no special measures are required for the motor, the power transmission mechanism, the electric circuit for driving the motor, and the like, and it is possible to avoid an increase in the size and cost of the image pickup apparatus.

Although the present invention has been described above with the embodiments, the above embodiments are merely examples of embodiment of the present invention, and the technical scope of the present invention is limitedly interpreted by these. It shouldn't be. That is, the present invention can be implemented in various forms without departing from the technical idea or its main features.
For example, in the above-described embodiment, the example in which the image pickup device 100 incorporates the control device to which the present invention is applied has been described, but the present invention is not limited to this, and the control device to which the present invention is applied is the image pickup device. It may be configured as a separate body.
(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100: Image pickup device, 101: Image pickup optical system, 102: Image pickup element, 103: Image processing unit, 104: System control unit, 105: Drive control unit, 106: Pan motor, 107: Tilt motor, 108: Storage unit, 109: Communication department

Claims (11)

A control device that controls an image pickup device including an image pickup unit and a drive unit that changes the image pickup direction of the image pickup unit.
An image processing means for performing cutout processing on image data, and
While the drive unit is stopped, the image pickup data acquired by the image pickup unit is cut out by the image processing means, and while the drive unit is being driven, the image pickup data acquired by the image pickup unit is stored. A control device including a control means for controlling the imaged data before driving stored in the storage unit so as to perform cutting processing by the image processing means. While the driving unit is being driven, the control means controls to perform cutting processing by the image processing means on the image pickup data of the last image pickup frame before driving stored in the storage unit. The control device according to claim 1. The control device according to claim 1 or 2, wherein the control means determines a cutout region of the cutout process by the image processing means based on a speed command to the drive unit. The control device according to claim 3, further comprising a communication means for receiving the speed command from an external device. The driving unit is characterized by a predetermined time from the application of the driving signal to the circuit for driving the driving unit to the end of the change in the imaging direction of the imaging unit. The control device according to any one of claims 1 to 4. The control device according to claim 5, wherein the predetermined time is stored in the storage unit as a predetermined parameter. The control device according to claim 5 or 6, wherein the drive signal is a pulse signal. The control device according to any one of claims 5 to 7, wherein the stop of the drive unit is a time obtained by subtracting the predetermined time from the off time of the drive signal. The angular resolution of the drive unit is obtained from the movement angle per unit pulse signal.
The control device according to any one of claims 1 to 8, wherein the cut-out region of the cut-out process by the image processing means can be set more finely than the change of the image pickup region that changes with the angular resolution. A control method for an image pickup apparatus including an image pickup unit and a drive unit for changing the image pickup direction of the image pickup unit.
While the driving unit is stopped, the imaging data acquired by the imaging unit is cut out, and while the driving unit is being driven, the imaging data acquired by the imaging unit is stored in a storage unit. A control method for an image pickup apparatus, which comprises a step of controlling the image pickup data before driving so as to perform a cutting process. A program for controlling an image pickup apparatus including an image pickup unit and a drive unit for changing the image pickup direction of the image pickup unit.
While the driving unit is stopped, the imaging data acquired by the imaging unit is cut out, and while the driving unit is being driven, the imaging data acquired by the imaging unit is stored in a storage unit. A program for causing a computer to execute a process for controlling the cutout process for the imaged data before driving.

Images