WO2022059584A1

WO2022059584A1 - Camera system and camera system control method

Info

Publication number: WO2022059584A1
Application number: PCT/JP2021/033109
Authority: WO
Inventors: 嵩臣神田
Original assignee: 株式会社日立国際電気
Priority date: 2020-09-18
Filing date: 2021-09-09
Publication date: 2022-03-24
Also published as: JPWO2022059584A1; JP7472299B2

Abstract

The present invention provides a camera system for magnifying and monitoring a plurality of objects to be monitored, wherein the objects to be monitored are magnified without a decrease in resolution, and a loss from a screen region is minimized.　A first camera device (50a) acquires wide-angle video data. A second camera device (50b) is controlled for panning, tilting, and zooming, and acquires magnified video data having a designated angle of view. A computer system (300): (a) extracts an object to be monitored from captured data from the first camera device or the second camera device; (b) calculates an expected showing time with respect to the object to be monitored; and (c) transmits a control signal to the second camera device to capture a magnified image with respect to an object to be monitored that has a shorter showing time among the objects to be monitored.

Description

Camera system and camera system control method

The present invention relates to a camera system and a control method for the camera system.

In recent years, there has been an increase in demand for human face recognition and face recognition using camera systems in the surveillance field. Conventionally, in response to these demands, it is common to acquire an image having a fixed angle of view with a fixed focus lens and perform face recognition or face recognition on the image. However, there is a problem that high accuracy cannot be obtained for a distant object in an image obtained with such a fixed angle of view.

On the other hand, it is conceivable to perform zoom control to expand the monitored object. For example, Patent Document 1 (Japanese Unexamined Patent Publication No. 2013-219556) proposes a method of performing electronic zooming in order to zoom to a position of interest.

Japanese Unexamined Patent Publication No. 2013-219556

The problem with the electronic zoom technology described in Patent Document 1 is that the resolution of the human face, which is the object to be monitored, is inferior to that of the optical zoom, and the accuracy is expected to decrease when applied to face recognition or face recognition. was there.
On the other hand, with optical zoom, there is a problem that generally, zooming can be performed only at the center of the angle of view, and images outside the region cannot be captured while zooming. Therefore, it is an object of the present invention to provide a technique for solving the above-mentioned problem that the angle of view is limited in the optical zoom and the problem that the resolution is inferior in the electronic zoom.

In order to solve the above problems, one of the representative camera systems of the present invention has a first camera device and an image pickup area overlapping the image pickup area of the first camera device, and is a monitored object. It is provided with a second camera device capable of acquiring an enlarged image of the above, and a control device for controlling the second camera device.
Then, the control device (a) extracts the monitored object from the image pickup data of the first camera device or the second camera device, and (b) calculates the expected projection time for the monitored object. (C) A control signal is transmitted to the second camera device so as to capture an enlarged image of the monitored object having a shorter reflection time among the monitored objects.

According to the present invention, in the area monitored by the camera system, the first camera device acquires the monitored area image, while the second camera device acquires the enlarged image of the monitored object, so that the image data for the monitored area can be obtained. There is no omission, and a clear image of the monitored object can be obtained.

Issues, configurations and effects other than those mentioned above will be clarified by the explanation in the form for carrying out the following.

It is a figure which shows the outline structure of the camera system 100 which concerns on embodiment of this invention. It is a figure which shows an example of the internal structure of the camera apparatus 50 used in the camera system 100 which concerns on embodiment of this invention. It is a block diagram of a computer system 300. It is a figure explaining the setting example of the camera system 100 which concerns on embodiment of this invention. It is a figure explaining the preparation procedure at the time of installation of the camera system 100 which concerns on embodiment of this invention. It is a figure explaining the concept of the projection expected time table in the camera system 100 which concerns on embodiment of this invention. It is a figure explaining an exemplary projection time exemplarily. It is a figure which shows the flowchart of the face recognition (face recognition) processing in the camera system 100 which concerns on embodiment of this invention. It is a figure which shows an example of the acquired wide-angle moving image data. It is a figure which shows the figure explaining the acquisition of the expected reflection time. It is a figure which shows an example of the acquired enlarged moving image data. It is a figure which shows the flowchart of the table update process in the camera system 100 which concerns on embodiment of this invention. It is a figure which shows the outline structure of the camera system 100 which concerns on other embodiment of this invention. It is a figure which shows the angle of view by each camera device 50 of the camera system 100 which concerns on another embodiment. It is a figure which shows the operation example of the camera system 100 which concerns on other embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment) FIG. 1 is a diagram showing an outline configuration of a camera system 100 according to an embodiment when two cameras are used. Further, the camera system 100 according to the present invention acquires moving image data having the maximum angle of view that can be acquired by the camera system 100 with one camera device (hereinafter, may be referred to as "wide-angle moving image data"). One of the purposes is to acquire a magnified image (hereinafter, may be referred to as "enlarged moving image data") by a local optical zoom lens with another camera device and monitor a predetermined area. ..
The number of camera devices used in the camera system 100 according to the present invention is not particularly limited. Therefore, the maximum angle of view that can be acquired by the camera system 100 in the case of a system using three or more cameras is the sum of the angles of view that can be acquired by three or more cameras.

In the camera system of the present disclosure, at least one camera device 50 includes a moving image imager 25 provided with an image pickup lens 22 capable of optical zoom operation, and pan / tilt electrically with respect to the moving image imager 25. A pan head mechanism 30 for operating is included. According to such a camera device 50, the enlarged moving image data can be acquired.
Further, in the present invention, together with such a single camera device 50, a camera device 50 in which the pan head mechanism 30 and the zoom operation of the lens are omitted can be used. The camera device 50 in which the pan head mechanism 30 and the zoom operation of the lens are omitted is configured to acquire wide-angle moving image data.
In the embodiment shown in FIG. 1, two pan head mechanisms 30 and two camera devices 50 capable of zooming the lens are used.

(Hardware Configuration) In the camera system 100 according to the present invention, a control command is transmitted to each of the plurality of camera devices 50 via the signal cable 9 together with the plurality of camera devices 50 as described above. A computer system 300 that receives moving image data acquired by the moving image imaging device 25 of a plurality of camera devices 50 is included.
In the present embodiment, there is an example in which a computer system 300 outside the camera device is used as a higher-level control device that sends a control command to a plurality of camera devices 50, receives moving image data from each camera device 50, and performs analysis. As will be described, it is not always essential to use such a computer system 300. For example, such an upper control device may be provided inside one of a plurality of camera devices 50 so that the upper control device can play the role of the upper control device.

In the camera system 100 in the present disclosure, the computer system 300 transmits a control command related to pan / tilt / zoom to the camera device 50. Based on such a control command, the camera device 50 sets the turning angle and elevation angle of the pan head mechanism 30, and the zoom magnification of the image pickup lens 22, and the moving image image pickup device 25 acquires the moving image data. Data is transmitted to the computer system 300. As a result, on the computer system 300 side, it is possible to acquire moving image data having a designated arbitrary angle of view. The zoom operation of the image pickup lens 22 is realized by being driven by a motor (not shown) based on a control command from the computer system 300.

In the camera system 100 according to the present embodiment, the two camera devices 50 are communicably connected to the computer system 300 via the signal cable 9. Hereinafter, suffixes a and b may be used to distinguish between the two camera devices 50. In FIG. 1, the power cable 43 is not shown.

In the embodiment shown in FIG. 1, the camera device 50a and the camera device 50b are set so that the wide-angle side angle of view of each image pickup lens 22 is common. It should be noted that it is not essential in the present invention to set the angle of view on the wide-angle side of each camera device 50 to be common in this way. In the camera system 100 according to the present embodiment, at least one of the camera device 50a and the camera device 50b constantly acquires moving image data having an angle of view on the widest angle side.

In the present embodiment, of the camera device 50a and the camera device 50b, for example, the camera device 50a constantly acquires moving image data of the angle of view on the widest angle side, and the camera device 50b pans and tilts from the computer system 300. -Receives control commands related to zooming and acquires moving image data at the specified angle of view.

Here, the former moving image data is referred to as wide-angle moving image data, and the latter moving image data is referred to as enlarged moving image data. Further, a camera device that acquires wide-angle moving image data is referred to as a first camera device, and a camera device that is pan / tilt / zoom controlled and acquires enlarged moving image data is referred to as a second camera device. In the case of the present embodiment, as for the camera device 50a, a camera device in which the pan head mechanism 30 or the zoom operation of the lens is omitted can be used as long as the moving image data of the wide-angle side angle of view can be acquired.

(Camera device configuration) Next, the internal configuration of the camera device 50 will be described in more detail. FIG. 2 is a diagram showing an example of the internal configuration of the camera device 50 included in the camera system 100 according to the present invention. In FIG. 2, 1 is a fixed portion, 2 is a horizontal rotating portion, and 3 is a camera housing. At this time, the horizontal rotating portion 2 is arranged on the fixed portion 1, and the camera housing is placed on the upper side surface of the horizontal rotating portion 2. 3 is arranged, and a moving image imaging device 25 provided with an image pickup lens 22 capable of optical zoom, that is, a video camera is housed in the camera housing 3 to form a camera housing.

The fixed portion 1 which is the basis of the pan head mechanism 30 serves as a pedestal for installing the camera device 50 at a required place, and a horizontal rotation shaft 6 is vertically attached to the pedestal, whereby the camera device 50 is horizontally rotated. The unit 2 can be rotated to an arbitrary turning angle position, and the moving image imaging device 25 is provided with a pan (PAN: turning angle) operation. Then, by holding the vertical rotation shaft 8 rotatably horizontally in the horizontal rotation portion 2 and attaching the camera housing 3 to the vertical rotation shaft 8, the camera housing 3 can be rotated to an arbitrary elevation angle position. , The moving image imaging device 25 is provided with a tilt (TILT: elevation angle) operation.

Therefore, in addition to the horizontal rotation shaft 6 and the vertical rotation shaft 8, the horizontal rotation unit 2 includes a horizontal rotation pulse motor 4, a vertical rotation pulse motor 5, a horizontal rotation worm gear 7, and a camera control. A circuit 10, a vertical rotation belt 13, a power supply unit 40, an origin turning angle sensor 44 and an origin elevation angle sensor 45, a horizontal rotation motor drive circuit 46, a vertical rotation motor drive circuit 47, and the like are provided.

The horizontal rotation motor drive circuit 46 and the vertical rotation motor drive circuit 47 are each controlled by the camera control circuit 10. At this time, the horizontal rotation shaft 6 is made hollow so that the two cables of the signal cable 9 and the power cable 43 can be drawn into the horizontal rotation portion 2 from the outside via the fixing portion 1. It has become.

First, the signal cable 9 is for transmitting a video signal (moving image data to the computer system 300) and a control signal (control command from the computer system 300), and is connected to the camera control circuit 10 after being drawn from the outside. Further, the cable 14 is also connected to the moving image imaging device 25 in the camera housing 3 via the vertical rotation shaft 8 which is also made hollow. On the other hand, the power cable 43 is for supplying AC power, is connected to the power supply unit 40, and functions to supply power to the power cable 43.

At this time, the power supply unit 40 functions to supply electric power for operation to the camera control circuit 10, the horizontal rotation motor drive circuit 46, and the vertical rotation motor drive circuit 47. Then, the horizontal rotation motor drive circuit 46 supplies a drive pulse to the horizontal rotation pulse motor 4 via the horizontal rotation motor cable 11, and the vertical rotation motor drive circuit 47 passes through the vertical rotation pulse motor cable 12. And supplies a drive pulse to the vertical rotation pulse motor 5.

Therefore, when a drive pulse is supplied from the horizontal rotation motor drive circuit 46, the horizontal rotation pulse motor 4 rotates, and as a result, the horizontal rotation worm gear 7 rotates, so that the entire horizontal rotation unit 2 rotates horizontally. The camera housing 3 can be panned by rotating around the shaft 6 and moving the camera housing 3 to an arbitrary turning angle. The origin turning angle sensor 44 functions to generate a detection signal when the turning angle reaches the origin turning angle (a predetermined turning angle set in advance).

Similarly, when a drive pulse is supplied from the vertical rotation motor drive circuit 47, the vertical rotation pulse motor 5 rotates, whereby the vertical rotation shaft 8 is rotated via the vertical rotation belt 13. The camera housing 3 rotates about the vertical rotation shaft 8, and the camera housing 3 can be moved to an arbitrary elevation angle to perform a tilt operation. The origin elevation angle sensor 45 functions to generate a detection signal when the origin elevation angle reaches the origin elevation angle (predetermined predetermined elevation angle).

At this time, although not shown, for example, a coaxial type reduction mechanism is interposed in the rotation transmission system from the vertical rotation pulse motor 5 to the vertical rotation shaft 8 via the vertical rotation belt 13. Therefore, a predetermined reduction ratio such as 100: 1 is given between the rotation speed of the vertical rotation pulse motor 5 and the displacement speed of the tilt angle of the camera housing 3. It should be noted that it is not necessary to explain that this predetermined reduction ratio is similarly given by the horizontal rotation worm gear 7 with respect to the displacement speed of the pan angle.

Next, the operation of setting the imaging direction by the camera device 50 will be described. In such a device, as described above, a pulse motor is used to drive the moving image imaging device 25, and the turning angle and the elevation angle are controlled at the origin. Since the so-called open-loop control method is adopted, in which only the number of pulses supplied to the pulse motor is used, the turning angle and elevation angle of the moving image imaging device 25 are set in advance as initial processing. It is necessary to move to a certain origin (origin turning angle and origin elevation angle).

This initial processing is performed as follows. That is, the camera control circuit 10 rotationally drives the horizontal rotation pulse motor 4 and the vertical rotation pulse motor 5 while observing the detection signals of the origin rotation angle sensor 44 and the origin elevation angle sensor 45, and the origin rotation angle sensor 44 and the origin elevation angle sensor. By stopping when the detection signal of 45 is obtained, the turning angle and the elevation angle of the moving image imaging device 25 are set to the origin. As the origin at this time, for example, for the turning angle, the front is the origin turning angle, and for the elevation angle, the origin elevation angle is when the camera housing 3 is set to the maximum angle downward (or upward) from the horizontal position. good.

After the initial processing is completed, the process shifts to the above-mentioned imaging direction setting operation, and includes information indicating the horizontal rotation angle (turning angle) and the vertical rotation angle (elevation angle) from the computer system 300 via the signal cable 9. A signal, that is, a horizontal rotation angle command and a vertical rotation angle command is supplied to the camera control circuit 10.

Then, the number of step pulses corresponding to each of the horizontal rotation angle command and the vertical rotation angle command is generated from the camera control circuit 10 and supplied to the horizontal rotation pulse motor 4 and the vertical rotation pulse motor 5. As a result, the horizontal rotation pulse motor 4 and the vertical rotation pulse motor 5 start to rotate, and the moving image imaging device 25 in the camera housing 3 starts to move toward the turning angle and the elevation angle commanded from the origin.

The horizontal rotation pulse motor 4 and the vertical rotation pulse motor 5 are finally stopped after step rotation from the origin position by the number of commanded pulses. At this time, as a characteristic of the pulse motor, Since the angle from the origin uniquely corresponds to the number of given pulses, the moving image imaging device 25 in the camera housing 3 will be stopped when it is correctly oriented in the commanded direction. , The imaging direction of the moving image imaging device 25 is set here without feedback control.

(Computer system configuration) Next, a configuration example of the computer system 300 will be described with reference to the drawings. FIG. 3 is a block diagram of a computer system 300 for implementing aspects of the embodiments of the present disclosure. The mechanisms and devices of the various embodiments disclosed herein may be applied to any suitable computing system. The main components of the computer system 300 include one or more processors 302, memory 304, terminal interface 312, storage interface 314, I / O (input / output) device interface 316, and network interface 318. These components may be interconnected via a memory bus 306, an I / O bus 308, a bus interface unit 309, and an I / O bus interface unit 310.

The computer system 300 may include one or more general purpose programmable central processing units (CPUs) 302A and 302B collectively referred to as processors 302. In one embodiment, the computer system 300 may include a plurality of processors, and in another embodiment, the computer system 300 may be a single CPU system. Each processor 302 may execute an instruction stored in memory 304 and include an onboard cache.

The memory 304 may store all or part of a program, module, and data structure that implements the functions described herein. For example, the memory 304 may store a camera system application 350 for controlling the camera system 100 according to the present invention. In one embodiment, the camera system application 350 may include an instruction or description that performs a function described below on the processor 302, or may include an instruction or description that is interpreted by another instruction or description. In certain embodiments, the application 350 for a camera system replaces or in addition to a processor-based system, semiconductor devices, chips, logic gates, circuits, circuit cards, and / or other physical hardware. It may be implemented in hardware via the device. In certain embodiments, the camera system application 350 may include data other than instructions or descriptions. In certain embodiments, other data input devices (not shown) may be provided to communicate directly with the bus interface unit 309, processor 302, or other hardware of the computer system 300.

The computer system 300 may include a processor 302, a memory 304, a display system 324, and a bus interface unit 309 that communicates between the I / O bus interface units 310. The computer system 300 may also include devices such as one or more sensors configured to collect the data and provide the data to the processor 302. The display memory may be a dedicated memory for buffering video data. The display system 324 may be connected to a display device 326 such as a stand-alone display screen, television, tablet, or portable device. In certain embodiments, the display device 326 may include a speaker to render the audio. Alternatively, the speaker for rendering the audio may be connected to the I / O interface unit. In other embodiments, the functionality provided by the display system 324 may be implemented by an integrated circuit that includes a processor 302. Similarly, the functionality provided by the bus interface unit 309 may be implemented by an integrated circuit that includes a processor 302.

The I / O interface unit has a function of communicating with various storages or I / O devices. For example, the terminal interface unit 312 may be a user output device such as a video display device or a speaker TV, or a user input device such as a keyboard, mouse, keypad, touchpad, trackball, button, light pen, or other pointing device. It is possible to attach such a user I / O device 320. The user inputs input data and instructions to the user I / O device 320 and the computer system 300 by operating the user input device using the user interface, and receives output data from the computer system 300. May be good. The user interface may be displayed on the display device via the user I / O device 320, reproduced by the speaker, or printed via the printer, for example. The I / O device interface 316 and the network interface 318 can be used for connection with the camera device 50 included in the camera system 100. It was

In certain embodiments, the computer system 300 is a device that receives a request from another computer system (client) that does not have a direct user interface, such as a multi-user mainframe computer system, a single user system, or a server computer. There may be. In other embodiments, the computer system 300 may be a desktop computer, a portable computer, a laptop computer, a tablet computer, a pocket computer, a telephone, a smartphone, or any other suitable electronic device.

(Setting of the camera system and preparation for starting use) Next, a setting method at the time of installation of the camera system 100 according to the present invention having the above system configuration will be described. FIG. 4 is a diagram for explaining a setting example of the camera system 100 according to the embodiment of the present invention, and FIG. 5 is a diagram for explaining a preparation procedure at the time of installing the camera system 100 according to the embodiment of the present invention. The preparation procedure in FIG. 5 is a diagram for explaining an action performed by the user, and is not a flowchart executed by the computer system 300.

FIG. 4 shows an example of moving image data acquired at the widest angle of view that can be captured by the camera device 50a and the camera device 50b by installing the camera system 100 according to FIG. In the moving image data (wide-angle moving image data) having the widest angle of view in the present embodiment, the imaging regions of the camera device 50a, which is the first camera device, and the camera device 50b, which is the second camera device, overlap. Therefore, even while the camera device 50b is performing zoom imaging, it can be acquired by the camera device 50a, which is the first camera device.

In the present embodiment, the camera system 100 will be described based on the settings for face recognition (face recognition if a matching database exists) of a person on the road in a landscape as shown in the figure. The magnified moving image data for such face recognition (face recognition) is provided with an image pickup lens 22 equipped with a zoom and a pan head mechanism 30, pan / tilt / zoom controlled, and an enlarged moving image having a specified angle of view. It is acquired by the camera device 50b, which is a second camera device that acquires image data.

In such an example, when the camera system 100 is installed, the designated effective area is set as shown in step S1 of FIG. Such a designated effective region will be described with reference to FIG. The designated effective area means an area in which the camera system can capture an image and captures a monitored object or performs image processing.

For example, in FIG. 4, when performing face recognition (face recognition) processing, an area excluding an area of interest (for example, a tree area) among the areas that can be imaged by the camera system, that is, a person on the road. An area where face recognition (face recognition) may be performed can be set as a designated effective area. By setting the designated effective area in this way, the image processing area can be limited and efficient processing can be performed.
However, it is not always necessary to set the designated effective area as described above, and it goes without saying that the monitored object may be supplemented or image processing may be performed on the entire area where the camera system can capture an image. ..

Next, when the designated effective area is set, this area is divided into an arbitrary number of blocks (when the designated effective area is not set, the area that can be imaged by the camera system is divided into an arbitrary number of blocks. do it). In this example, it is divided into 3 vertical blocks and 8 horizontal blocks. For each of these blocks, the estimated time until the monitored object disappears from the designated effective area is set according to the direction in which the monitored object (for example, a moving object such as a person or a car) moves. .. In the present disclosure, such an estimated time is referred to as an "estimated reflection time". (If the designated effective area is not set, the estimated time until the monitored object disappears from the area where the camera system can take an image is the "estimated reflection time".)

The expected reflection time as described above is used to predict how long the monitored object to be image-analyzed will be reflected in the wide-angle image. As a result, when there are a plurality of monitored objects in the area where the camera system can take an image, information for determining which monitored object should be preferentially subjected to image processing such as zooming is obtained. be able to.

Next, as shown in step S2 of FIG. 5, the user sets the expected projection time for each block in the designated effective area and for each moving direction of the monitored object. The estimated projection time may be set by a preparation period after the camera system is constructed and by an update method during operation, which will be described later. The estimated projection time set for each block and each movement direction of the monitored object is stored in the estimated projection time table. FIG. 6 is a diagram illustrating the concept of such an expected projection time table.

In FIG. 6, when the monitored object recognized by the block B _mn having the designated effective area is moving in the eight directions from (a) to (h), the monitored object disappears from the designated effective area. It shows that the expected time to do is set for each direction. For example, when it is recognized that the monitored object is moving in the direction of (a) in the block B _mn , it indicates that the expected projection time is set to T _{mn (0, 1)} . There is. The setting of these estimated times will be described later, but it may be set appropriately for each setting direction based on the average moving speed of the monitored object, or it is specified as changing according to the moving speed of the monitored object. You may.

In the present embodiment, the expected reflection time of the monitored object is set in every eight directions as the moving direction of the monitored object, but the number of setting directions of the estimated time is not limited to eight directions. For example, it may be set in four directions in the vertical and horizontal directions when looking at the paper surface, or it may be set in more than eight directions, for example, 16 directions.

Furthermore, if the moving direction of the monitored object extracted by image analysis does not correspond to the direction in which the estimated time is set, the projected projection time in the direction closest to the moving direction of the monitored object among the set directions is set. May be applied. Further, the moving direction of the monitored object may be vector-decomposed in the direction in which the expected time is set, and calculated as a combination of the estimated times in these decomposed directions.
Further, when the monitored object is stationary, the average of the expected projection times in all directions may be applied. If the monitored object spans multiple blocks, the estimated projection time of the block containing the center point of the monitored object (for example, the center of gravity when the monitored object is viewed as a two-dimensional figure) is applied. Alternatively, the average expected projection time of a plurality of blocks may be applied.

Such an estimated projection time will be illustrated by way of FIG. 7. In FIG. 7, assuming that the object to be monitored is moving in the direction of (c) in the block B ₂₂ , the distance to the end of the region where the camera system can take an image is relatively long. Therefore, the expected reflection time of the monitored object moving in the direction (c) is relatively long. On the other hand, if this monitoring object is moving in the direction (g), the distance to the end of the area where the camera system can take an image is relatively short, so the monitoring object moving in the direction (g). The expected projection time of an object is relatively short.

As described above, an expected projection time table in which the estimated projection time is set for each block in the designated effective area and for each movement direction of the monitored object is prepared, and the camera system 100 according to the present invention is ready for installation and start of use. Complete.

(Example of Face Recognition Processing) Next, an example in which face recognition (face recognition) processing is performed for monitoring a predetermined area using the camera system 100 according to this embodiment will be described.
FIG. 8 is a diagram showing a flowchart of face recognition (face recognition) processing in the camera system 100 according to the embodiment of the present invention. A program based on such a flowchart can be stored in the memory 304 as an application 350 for a camera system and configured to be executed on the processor 302.

In FIG. 8, when the face recognition (face recognition) process is started in step S100, the process proceeds to step S101, and wide-angle moving image data is acquired from the camera device 50a, which is the first camera device. FIG. 9 is a diagram showing an example of acquired wide-angle moving image data.

In step S102, the monitored object is extracted by image analysis of the acquired wide-angle moving image data. For the extraction of the monitored object by such image analysis, a known method such as an object extraction process based on background subtraction can be adopted. When the object to be monitored has a specific shape such as a vehicle or a person, it may be extracted by a method such as pattern matching or skeleton detection.

In the next step S103, the position where the monitored object is extracted, more specifically, the block is specified, and further, the moving direction of the monitored object is specified. To specify the moving direction of the monitored object, a known method of calculating the moving direction from the feature points of the monitored object can be used. Explaining with reference to FIG. 9, in such a step, it is specified that the monitored object exists in each of the block B ₂₃ and the block B ₂₄ , and each monitored object is moved in the direction of the arrow. Identify that.

In step S104, the expected projection time table is referred to from the block and the moving direction of the specified monitored object, and the estimated projection time stored in the estimated projection time table is acquired. FIG. 10 is a diagram illustrating acquisition of the expected projection time. In FIG. 10, T _{23 (-1, -1)} is acquired as the expected projection time of the monitored object extracted in the block B ₂₃ from the estimated projection time table, and the monitored object extracted in the block B ₂₄ is obtained. It is shown that T _{24 (1,0)} was acquired as the expected projection time of. Here, it is assumed that T _{23 (-1, -1)} <T _{24 (1, 0)} .

In step S105, it is determined whether or not the expected projection time has been acquired for all the monitored objects in the wide-angle moving image data. If the determination in step S105 is NO, the process proceeds to step S112, paying attention to the next monitored object, and looping to perform the processes of steps S103 and S104. On the other hand, if the determination in step 105 is YES, the process proceeds to step S106.

In step S106, a control command is transmitted to the camera device 50b, which is the second camera device, so as to acquire the magnified moving image data of the monitored object having the shorter expected projection time. According to the examples of FIGS. 9 and 10, the block that is expected to disappear from the designated effective area first due to the relationship of T _{23 (-1, -1)} <T _{24 (1, 0)} of the estimated projection time. A control command is transmitted to the camera device 50b so as to acquire magnified moving image data for the monitored object extracted by B ₂₃ . This control command commands the camera device 50b to acquire magnified moving image data by optical zoom of the monitored object extracted by the block B ₂₃ by pan / tilt / zoom control. The camera device 50b acquires enlarged moving image data based on such a control command.

In step S107, image analysis of the enlarged moving image data acquired from the camera device 50b, which is the second camera device, is performed. FIG. 11 shows an example of enlarged moving image data acquired from the camera device 50b. Here, in the camera system 100 according to the present embodiment, the camera device 50b is pan / tilt / zoom controlled to acquire magnified moving image data by optical zoom of the monitored object, so that face recognition and face recognition and face are performed. It is possible to improve the accuracy of authentication.

In step S108, it is determined whether or not the monitored object is a human being based on the image analysis of the enlarged moving image data. If this determination is NO, the process proceeds to step S111, and the process of quantifying the feature amount of the human face and the like is skipped.

On the other hand, if the determination in step S108 is YES, the process proceeds to step S109, and the facial feature amount of the monitored object, which is a human, is quantified. Conventional techniques can be appropriately used as the algorithm for quantifying the facial features. In step S110, it is determined whether or not the facial feature amount of a predetermined value or more can be acquired.

If the determination in step S110 is YES, it is determined that sufficient information (facial feature amount) has been acquired for the monitored object, and the process proceeds to step S111. On the other hand, if the determination in step S110 is NO, the process proceeds to step S114, and it is first determined whether or not a predetermined predetermined time (time-out time) has elapsed.

Such a time-out time is set even if the monitored object is a person, but if the face is not shown in the magnified moving image data, it can be determined by adjusting the zoom of the image pickup lens 22. Because it is impossible. When there are a plurality of monitored objects in the area where the camera system can take an image, it is more efficient to direct the processing to the acquisition of the magnified moving image data of the other monitored objects. Therefore, if the determination in step S114 is YES, the acquisition of the facial feature amount for the monitored object is given up, and the process proceeds to step S111.

On the other hand, if the determination in step S114 is NO, the process proceeds to step S115, the angle of view is adjusted by the zoom operation of the image pickup lens 22, the enlarged moving image data is acquired, and the facial feature amount is numerically measured again in step S109. In step S110, a process for determining whether or not a facial feature amount of a predetermined value or more can be acquired is looped. As described above, in the camera system 100 according to the present invention, the setting is made so as to try to acquire the facial feature amount of a predetermined value or more while adjusting the angle of view until the time-out time elapses.

In step S111, it is determined whether a sufficient feature amount has been acquired for all the monitored objects, or whether a time-out has occurred because a sufficient feature amount could not be acquired. If the determination in step S111 is NO, the process proceeds to step S113, focusing on the next monitored object, and looping to perform a series of processes for acquiring facial features of a predetermined value or more.

In the examples of FIGS. 9 and 10, when the acquisition process of the facial feature amount of the monitored object extracted in the block B ₂₃ is completed, subsequently, in step S113, the monitored object extracted in the block B ₂₄ Focusing on this, pan / tilt / zoom control is performed on the camera device 50b in order to acquire magnified moving image data. On the other hand, if the determination in step S111 is YES, the process proceeds to step S116, and the face recognition (face recognition) process is terminated.
When the enlarged image of the next monitored object is acquired in step S113, the next monitored object is moving while the enlarged image of the first monitored object is acquired. In the camera system of the present embodiment, since the first camera device continues to shoot the next monitored object even while the second camera device performs pan / tilt / zoom control, the following Never lose track of what you are monitoring.
Further, even if a new monitored object is generated while the magnified image of the first monitored object is acquired, the first camera device can supplement this. When a new monitoring target is generated in this way, in the flowchart of FIG. 8, the transition destination of step S113 is set between steps S101 and S102, and the new monitoring target and the recognized monitoring target are set. After comparing the expected projection times again, the acquisition target of the next enlarged image may be determined.

As described above, in the camera system 100 according to the present embodiment, the monitoring objects extracted by the image analysis are prioritized based on the expected projection time acquired from the estimated projection time table. Then, pan / tilt / zoom control is performed on the camera device 50b so as to give priority to the monitored object that is expected to disappear from the moving image data (designated effective area) earlier and acquire the enlarged moving image data. Is configured to do. According to such a configuration, accurate face recognition and face recognition can be realized without deteriorating the resolution of the human face in the monitoring area, and human face recognition and face recognition and face can be realized in the monitoring area. It is possible to reduce the probability that authentication will fail.

(Setting of estimated projection time) When the camera system 100 according to the present invention is installed and set, the estimated projection time is set for each block in the designated effective area and for each moving direction of the monitored object. I explained that the estimated time table should be prepared in advance. It is conceivable that the estimated time initially set in such an estimated projection time table is different from the actual estimated projection time when the camera system 100 is actually operated. Therefore, it is preferable that the expected projection time table is configured to be appropriately updated with the actual operation of the camera system 100.

Hereinafter, the update process of the expected projection time table in the camera system 100 will be described. FIG. 12 is a diagram showing a flowchart of table update processing of the expected projection time table in the camera system 100 according to the embodiment of the present invention. A program based on such a flowchart can be stored in the memory 304 as an application 350 for a camera system and configured to be executed on the processor 302.

In FIG. 12, when the table update process is started in step S200, the process proceeds to step S201, and wide-angle moving image data is acquired from the camera device 50a, which is the first camera device.

In step S202, the monitored object is extracted by image analysis of the acquired wide-angle moving image data. For the extraction of the monitored object by such image analysis, a known method such as an object extraction process based on background subtraction can be adopted.

In the next step S203, the position where the monitored object is extracted, more specifically, the block is specified, and the moving direction of the monitored object is further specified. To specify the moving direction of the monitored object, a known method of calculating the moving direction from the feature points of the monitored object can be used.

In step S204, attention is paid to the monitored object extracted in the previous step, and the time until the monitored object disappears from the designated effective area is timed.

Subsequently, in step S205, the projected projection time table is based on the position (block) specified in step S203, the moving direction of the monitored object of interest, and the time until the monitored object disappears measured in step S204. Make an update.

When updating the projected expected time table, the method of updating so as to overwrite with the newly acquired timed time, or the newly acquired timed time and the estimated time already described in the table. It is possible to adopt a method of taking the average value of and updating with this average value. Further, when taking the average value, the newly acquired timekeeping time and the expected time already described in the table may be weighted and the average value may be taken. Then, in step S206, the table update process is terminated.

By updating the expected projection time table as described above, even if there is a discrepancy between the initially set estimated projection time table and the actual expected projection time, the cumulative total of the camera system 100 As the operating time increases, the expected projection time with higher accuracy will be described in the estimated projection time table.

(Second Embodiment) Next, the second embodiment of the present invention will be described. In the previous embodiment, the operation of the camera system 100 including the two camera devices 50 has been described, but the camera system 100 according to the second embodiment can also use three or more camera devices 50. Here, an embodiment of a camera system 100 including four camera devices 50 will be described.
In the following, the differences from the embodiments described above will be mainly described. The points not described below are the same as those described in the previous embodiment.

FIG. 13 is a diagram showing an outline configuration of the camera system 100 according to the second embodiment. As the four camera devices 50 used in the camera system 100 according to the second embodiment, four cameras similar to those described in detail in FIG. 2 can be used. The suffixes a, b, c and d will be used to distinguish these four camera devices 50.

FIG. 14 is a diagram showing the angles of view of the four

camera devices

50a, 50b, 50c, and 50d. FIG. 14 shows the angle of view on the widest angle side of the image pickup lens 22 of each of the

camera devices

50a, 50b, 50c, and 50d. Further, in FIG. 14, different types of lines indicating the angle of view are used so that the angle of view of each camera device 50 can be understood, and the lines indicating the angle of view are intentionally shifted in the vertical direction. There is. Therefore, the angles of view in the vertical direction of each camera device may be the same.

In the camera system 100 according to the second embodiment, the wide-angle moving image data is always used for the image having the widest angle of view that can be covered by the

camera devices

50a, 50b, 50c, 50d constituting all the camera systems 100. To get. Of the total angle of view shown in FIG. 13, the wide-angle moving image data at both ends can be acquired only by the camera device 50a and the camera device 50d. Therefore, among the four camera devices, the camera device 50a and the camera device 50a The camera device 50d will be used as a camera device dedicated to the first camera device.

While the camera device 50b is acquiring wide-angle moving image data on the widest angle side, the areas shown in FIGS. 13 (X) and 13 (Y) can be covered by the camera device 50b. Is pan / tilt / zoom controlled and can be used to acquire enlarged moving image data of a specified angle of view. In this case, the camera device 50b is used as the first camera device, and the camera device 50c is used as the second camera device.

While the camera device 50c is acquiring wide-angle moving image data on the widest angle side, the areas shown in FIGS. 13 (Y) and (Z) can be covered by the camera device 50c. Is pan / tilt / zoom controlled and can be used to acquire enlarged moving image data of a specified angle of view. In this case, the camera device 50c is used as the first camera device, and the camera device 50b is used as the second camera device.

Therefore, in the second embodiment, the overlapping imaging regions of the first camera device and the second camera device are the regions (X), (Y), and (Z) of FIG. 13, and the camera device 50b and the camera device 50c. Is utilized as a camera device for both the first camera device and the second camera device, so that the same operation as that of the previous embodiment can be performed. Here, the area that can be set as the designated effective area by the user of the camera system 100 is within the areas (X), (Y), and (Z) of FIG.
Similarly to the previous embodiment, the camera system 100 according to the second embodiment also has accuracy corresponding to a wider monitoring area by setting the designated effective area and the expected projection time table. You can often perform face recognition and face recognition.
When setting the expected projection time table in the second embodiment, the areas (X), (Y), and (Z) in FIG. 13 capture the monitored object or perform image processing. Since it is an area that can be used, the estimated time until the monitored object disappears from the areas (X), (Y), and (Z) in FIG. 13 is set.
Then, in the second embodiment, by using the camera devices 50a to 50d in conjunction with each other by the control device, it is possible to monitor a wider range as compared with the previous embodiment.

FIG. 15 is a diagram showing an operation example of the camera system 100 according to the second embodiment. FIG. 15 shows the entire moving image data acquired by the camera system 100 according to the second embodiment, and shows a situation in which the monitored object A and the monitored object B are present in the moving image data. There is. Hereinafter, the expected projection time will be described on the premise that the monitored object A is shorter than the monitored object B.

In FIG. 15A, for the monitored object A having a shorter expected projection time, the camera device 50c acquires the magnified moving image data with the pan / tilt / zoom controlled camera device 50b, while the camera device 50c obtains the wide-angle moving image data. Have acquired. In FIG. 15A, the camera device 50c is used as the first camera device, and the camera device 50b is used as the second camera device. As described above, as described above, when the feature amount of the face equal to or higher than the predetermined value can be acquired for the monitored object A in FIG. 15 (A), the camera system 100 subsequently obtains an enlarged moving image of the monitored object B. Execute the process to acquire the data.

FIG. 15B shows a situation after FIG. 15A, and the camera device 50b is obtained with the pan / tilt / zoom controlled camera device 50c while acquiring the magnified moving image data of the monitored object B. Is acquiring wide-angle moving image data. In FIG. 15B, the camera device 50b is used as the first camera device, and the camera device 50c is used as the second camera device.

Also in the camera system 100 according to the second embodiment composed of three or more camera devices as described above, the angle of view of the camera device used as the second camera device for acquiring the magnified moving image data. Is complemented by another camera device, so that the enlarged moving image data of the monitoring target is acquired without degrading the resolution, while the monitoring area is obtained by another camera device used as the first camera device. It is possible to continue to acquire the entire wide-angle moving image data.

As described above, according to the present invention, in the area monitored by the camera system 100, the first camera device acquires wide-angle moving image data, and the second camera device acquires enlarged moving image data of the monitored object of interest. Therefore, the moving image data of the entire monitoring area is not lost, and the resolution of the human face is not lowered, so that accurate face recognition and face recognition can be realized.
Further, in the present embodiment, the object to be monitored is a person, but the object to be monitored may be a vehicle. When the object to be monitored is a vehicle, the information on the license plate of the vehicle may be acquired instead of quantifying the facial features from the enlarged image data.

1 ... Fixed part 2 ... Horizontal rotation part 3 ... Camera housing 4 ... Horizontal rotation pulse motor 5 ... Vertical rotation pulse motor 6 ... Horizontal rotation shaft 7 ... Warm gear for horizontal rotation 8 ... Shaft for vertical rotation 9 ... Signal cable 10 ... Camera control circuit 13 ... Belt for vertical rotation 14 ... Cable 22 ... Imaging lens 25 ... Video imaging Device 30 ... Cloud stand mechanism 40 ... Power supply unit 43 ... Power cable 44 ... Origin turning angle sensor 45 ... Origin elevation angle sensor 46 ... Horizontal rotation motor drive circuit 47 ... Vertical Rotating motor drive circuit 50 ... Camera device 100 ... Camera system 300 ... Computer system 302 ...

Processors

302A, 302B ... General-purpose programmable central processing device (CPU) 304 ... Memory 306 ... -Memory bus 308 ... I / O bus 309 ... Bus interface unit 310 ... I / O bus interface unit 312 ... Terminal interface 314 ... Storage interface 316 ... I / O (ON) Output) Device interface 318 ... Network interface 320 ... User I / O device 322 ... Storage device 324 ... Display system 326 ... Display device 330 ... Network 350 ... Camera system application

Claims

The first camera device and
A second camera device having an image pickup area overlapping with the image pickup area of the first camera device and capable of acquiring an enlarged image of a monitored object, and a second camera device.
A control device for controlling the second camera device is provided.
The control device (a) extracts a monitored object from the image pickup data of the first camera device or the second camera device, and (b) calculates the expected projection time for the monitored object, and (b) C) A camera system that transmits a control signal to the second camera device so as to capture an enlarged image on the monitored object having a shorter reflection time among the monitored objects.
The camera system according to claim 1, wherein the control device includes the expected projection time table according to the position and moving direction of the monitored object in order to calculate the estimated reflection time.
The camera system according to claim 2, wherein the expected projection time table is provided for each block in which the overlapping imaging regions of the first camera device and the second camera device are divided into a plurality of blocks.
The first camera device and
A second camera device having an image pickup area overlapping with the image pickup area of the first camera device and capable of acquiring an enlarged image of a monitored object, and a second camera device.
In a camera system including a control device for controlling the second camera device, (1) a step of extracting a monitored object in an overlapping imaging region of the first camera device and the second camera device (2). ) The step of calculating the expected projection time for the monitored object based on the position where the monitored object is recognized and the moving direction of the monitored object, and (3) the projection in the monitored object. For a monitoring target with a shorter time, the monitoring target that was the shooting target in the step of the second camera device taking a magnified image and (4) if there are a plurality of monitoring targets, in the step (3). A method for controlling a camera system, which comprises a step of taking a magnified image of a monitored object other than an object, which has a shorter reflection time.