JP2020194125A - Information processing device, imaging device, camera system, information processing method, a program, and storage medium - Google Patents

Abstract

To provide an information processing device capable of determining a tilt angle suitable for recognizing a subject.SOLUTION: An information processing device includes: acquisition means (1015) that acquires a piece of installation information of an imaging device; first calculation means (1016) that calculates the limit distance at which a subject can be recognized based on the installation information and a piece of attribute information of the subject; second calculation means (1017) that calculates a forward depth of field and a rear depth of field when focusing on the subject; and third calculation means (1009) that calculates the tilt angle based on the limit distance and the rear depth of field.SELECTED DRAWING: Figure 8

Description

The present invention relates to an imaging device capable of adjusting the tilt angle.

Patent Document 1 describes a method of expanding the depth of field range by detecting the amount of defocus in a plurality of focal detection regions of an imaging unit and adjusting the tilt angle of the image sensor based on the detected amount of defocus. It is disclosed. Patent Document 2 discloses a method of displaying a range in which a camera can be installed based on shooting conditions such as an angle of view, resolution, and aspect ratio when a position, range, and conditions of a shooting target are specified.

Japanese Unexamined Patent Publication No. 2017-173802 Japanese Unexamined Patent Publication No. 2018-014553

However, the method disclosed in Patent Document 1 does not consider a region in which a subject such as a person or a face can be recognized. Therefore, when the purpose is to recognize the subject, the subject in an originally unnecessary area is also focused. In the method disclosed in Patent Document 2, the area where the subject can be recognized is considered, but it is premised that the subject is in focus, and the case where the subject is out of focus is not considered. Further, in the method disclosed in Patent Document 2, tilting is not taken into consideration. Further, in the methods disclosed in Patent Document 1 and Patent Document 2, when the depth of field is extended by adjusting the tilt angle, the blur peculiar to tilt that appears for a subject at a short distance to a medium distance is also taken into consideration. It has not been.

Therefore, an object of the present invention is to provide an information processing device, an imaging device, a camera system, an information processing method, a program, and a storage medium capable of determining a tilt angle suitable for recognizing a subject.

The information processing device as one aspect of the present invention calculates the limit distance at which the subject can be recognized based on the acquisition means for acquiring the installation information of the image pickup device and the installation information and the attribute information of the subject. Based on the calculation means of, the second calculation means for calculating the front depth of field and the rear depth of field when the subject is focused, and the limit distance and the rear depth of field. It has a third calculation means for calculating the tilt angle of the image pickup element or the optical system.

The image pickup apparatus as another aspect of the present invention includes an image pickup means for imaging a subject and the information processing apparatus.

A camera system as another aspect of the present invention includes the image pickup device and an input means for inputting the installation information of the image pickup device, and a control device for transmitting the installation information to the image pickup device via a network. Has.

The information processing method as another aspect of the present invention includes a step of acquiring installation information of the imaging device and a step of calculating a limit distance at which the subject can be recognized based on the installation information and the attribute information of the subject. , A step of calculating the front depth of field and the rear depth of field when the subject is focused, and a step of calculating the tilt angle based on the limit distance and the rear depth of field. Has.

A program as another aspect of the present invention causes a computer to execute the information processing method.

A computer-readable storage medium as another aspect of the invention stores the program.

Other objects and features of the present invention will be described in the following embodiments.

According to the present invention, it is possible to provide an information processing device, an imaging device, a camera system, an information processing method, a program, and a storage medium capable of determining a tilt angle suitable for recognizing a subject.

It is a block diagram of the camera system in this embodiment. It is a block diagram of the camera system in this embodiment. It is explanatory drawing of the tilt control in this embodiment. It is explanatory drawing of the depth of field in this embodiment. It is explanatory drawing of the depth of field in this embodiment. It is explanatory drawing of the depth of field in this embodiment. It is explanatory drawing of the recognition limit surface in this embodiment. It is a flowchart of the information processing method in this embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

A surveillance camera (imaging device) may be installed at a high place, and the optical axis of the surveillance camera may be directed diagonally downward to monitor people passing by on the road or to photograph a car or its license plate. In this case, since the optical axis of the surveillance camera faces diagonally downward, the focal plane that is in focus when performing imaging is the plane perpendicular to the optical axis and does not match the subject plane to be actually imaged. .. Therefore, the focusing depth range is only a part of the screen (within the depth of field), and the other areas are out of focus.

To solve this problem, there is a method of increasing the depth of field by reducing the aperture of the optical system to prevent out-of-focus. However, in surveillance cameras that shoot under low illuminance, there are many cases where the aperture is opened close to the maximum aperture. As a result, the depth of field becomes shallow, the focus is out of focus within the depth of the entire screen, and shooting is performed in a defocused state.

Therefore, there is a method of tilting the lens relative to the image sensor to widen the depth of field range. On the other hand, there is a technique of tilting the image sensor relative to the lens to widen the depth of field range. Hereinafter, tilting the relative angle between the lens and the image sensor is referred to as "aoru". By applying this technology to a surveillance camera, the depth of field becomes deep even when the aperture is open, and one camera can monitor a long distance.

For example, in the case of a structure in which the tilt of the image sensor is controlled by driving the stepping motor, the tilt angle per pulse is determined by the drive of the stepping motor and the gear ratio of the gears. Therefore, the image sensor can be controlled to have a desired tilt angle by designating the tilt angle (hereinafter referred to as a tilt angle) of the image sensor with respect to the lens by the number of pulses. The rotation axis of the tilt of the image sensor is various, such as the vertical tilt that rotates the image sensor around the center line in the long side direction of the image sensor, and the left and right tilt that rotates the image sensor around the center line in the short side direction of the image sensor. There is a rotation axis.

By the way, when adjusting the tilt angle, blurring peculiar to tilt may appear in a part (upper or lower part) of a subject at a short to medium distance. For example, when using a telephoto lens with a shallow depth of field, a part of a short-distance to medium-distance subject is used before the depth of field is extended from a short-distance subject to a long-distance subject by adjusting the tilt angle. Blurring peculiar to tilting appears for (upper or lower).

Hereinafter, embodiments for solving such problems will be described.

<Network configuration>
First, the configuration of the camera system according to the present embodiment will be described with reference to FIG. FIG. 1 is a configuration diagram of a camera system including a surveillance camera according to the present embodiment. The camera system 100 includes a surveillance camera (imaging device) 100, a control device 1100 for the surveillance camera 1000, and a network 1200. The surveillance camera 1000 and the control device 1100 are connected to each other via the network 1200 so as to be able to communicate with each other. The control device 1100 transmits various commands to the surveillance camera 1000. The surveillance camera 1000 transmits the response to those commands to the control device 1100.

The network 1200 is a plurality of routers, switches, cables, etc. that satisfy communication standards such as Ethernet (registered trademark). In the present embodiment, the communication standard, scale, and configuration are not limited as long as they can communicate between the surveillance camera 1000 and the control device 1100. Therefore, the network 1200 may be, for example, a wired LAN (Local Area Network), a wireless LAN, a WAN (Wide Area Network), or the like.

In this figure, the number of surveillance cameras is one for the sake of simplification of the description, but two or more surveillance cameras may be included. Further, in addition to the control device 1100, another control device that accesses the surveillance camera 1000 to receive and store images may be included.

<Surveillance camera>
Next, the configuration and functions of the camera system 100 will be described with reference to FIG. FIG. 2 is a block diagram of the camera system 100.

First, the configuration and function of each part of the surveillance camera 1000 will be described. The surveillance camera 1000 includes an imaging unit (imaging means) 1001, an image processing unit 1002, a system control unit 1003, and a storage unit 1004. Further, the surveillance camera 1000 includes a lens driving unit 1005, an image pickup angle of view control unit 1006, a focus control unit 1007, an image pickup element drive unit (angle control means) 1008, and an image pickup element control unit (third calculation means) 1009. .. The surveillance camera 1000 also includes a pan drive unit 1010, a tilt drive unit 1013, a pan / tilt control unit 1011 and a communication unit 1012, and a program memory 1014. Further, the surveillance camera 1000 includes an installation information management unit (acquisition means) 1015, a recognition limit distance calculation unit (first calculation means) 1016, a depth of field calculation unit (second calculation means) 1017, and a rear depth of field. It has a surface designation unit (determining means) 1018.

The image pickup unit 1001 includes a lens (imaging optical system) and an image pickup element, and performs imaging of a subject and conversion into an electric signal. The image processing unit 1002 performs predetermined image processing, resolution conversion processing, and compression coding processing of the signal imaged and photoelectrically converted by the imaging unit 1001 to generate image data.

The system control unit 1003 analyzes the command (camera control command) transmitted from the control device 1100 and performs processing according to the command. The system control unit 1003 mainly receives a request command for a live image from the control device 1100, and distributes the image data generated by the image processing unit 1002 via the communication unit 1012. In addition, the system control unit 1003 sets the installation information of the surveillance camera 1000 from the control device 1100, the resolution setting, the zoom setting, the focus setting, the tilt angle setting, and the pan / tilt. Receives the request command for the set value. Further, the system control unit 1003 sets values from the installation information management unit 1015, the image processing unit 1002, the imaging angle of view control unit 1006, the focus control unit 1007, the image pickup element control unit 1009, and the pan / tilt control unit 1011 respectively. read. Then, the system control unit 1003 distributes each of the read set values to the control device 1100 via the communication unit 1012.

When the system control unit 1003 receives various setting commands, the system control unit 1003 causes the installation information management unit 1015, the image processing unit 1002, the imaging angle of view control unit 1006, the focus control unit 1007, the image sensor control unit 1009, and the pan / tilt control unit 1011. On the other hand, control based on each set value is instructed. Here, the various setting commands are, for example, installation information, resolution, zoom, focus, tilt angle, and pan / tilt setting commands. The image processing unit 1002, the imaging angle of view control unit 1006, and the focus control unit 1007 control the imaging unit 1001 and the lens driving unit 1005 based on the instructions of the system control unit 1003. The image sensor control unit 1009 and the pan / tilt control unit 1011 control the image sensor drive unit 1008, the pan drive unit 1010, and the tilt drive unit 1013 based on the command of the system control unit 1003. As a result, the resolution, zoom, focus, tilt angle, and pan / tilt set values set by the control device 1100 are reflected in the surveillance camera 1000.

The storage unit 1004 stores images in the internal storage and the external storage. The imaging angle of view control unit 1006 instructs the lens drive unit 1005 to change the zoom lens position based on the zoom setting value output from the system control unit 1003. The imaging angle of view control unit 1006 manages, for example, the focal length as a zoom setting value. The focus control unit 1007 instructs the lens drive unit 1005 to change the focus lens position based on the focus setting value output from the system control unit 1003. By changing the position of the focus lens, the focus position is changed.

Here, the focus modes of the surveillance camera 1000 include manual, autofocus (AF), fixed at infinity, and one-shot AF. Manual is a mode in which the user manually adjusts the lens position of the focus lens. The autofocus is a mode in which the lens position of the focus lens is constantly and automatically adjusted so that the surveillance camera 1000 focuses on, for example, the central region of the imaging angle of view. The infinity fixed mode is a mode in which the surveillance camera 1000 moves the focus lens to a lens position recorded on the focus lens so that the focus lens can be focused at infinity. This mode is mainly used for shooting a distant landscape or a distant subject, and when fixed at infinity, the subject in the vicinity is out of focus. Further, the one-shot AF is a function in which the surveillance camera 1000 executes AF only once for the imaging angle of view when the focus mode is manual, and then returns the focus mode to manual.

When AF is performed by the contrast detection method, the focus control unit 1007 gives an instruction to change the focus lens position little by little and discretely. At the same time, the focus control unit 1007 acquires image data from the image processing unit 1002 and calculates a contrast evaluation value for a region to be focused. As a result, a table of contrast evaluation values for each of the discretely specified focus lens positions can be obtained, and the focus lens position having the highest contrast evaluation value is set from this table.

The image sensor control unit 1009 instructs the image sensor drive unit 1008 to change the tilt angle based on the set value of the tilt angle output from the system control unit 1003 or various data used for calculating the tilt angle. .. The pan / tilt control unit 1011 instructs the pan drive unit 1010 and the tilt drive unit 1013 to change the pan / tilt based on the pan / tilt set value output from the system control unit 1003. The system control unit 1003 executes various operations based on the computer program stored in the program memory 1014. The installation information management unit 1015 stores installation information such as a camera installation position and a subject position (focus position) to be focused on the optical axis, and acquires and sets the installation information according to an instruction from the system control unit 1003.

The recognition limit distance calculation unit 1016 stores attribute information such as the width, orientation, and required number of pixels for each attribute of the subject to be recognized, and acquires and sets the attribute information according to an instruction from the system control unit 1003. To do. Here, the attribute information may be set in advance by the computer program in the program memory 1014. Further, the recognition limit distance calculation unit 1016 calculates the recognition limit distance by using the attribute information, the installation information, and various data according to the instruction from the system control unit 1003.

The depth of field calculation unit 1017 stores optical information such as an effective aperture and a permissible circle of confusion diameter, and acquires and sets the optical information according to an instruction from the system control unit 1003. Further, the depth of field calculation unit 1017 calculates the depth of field using the installation information, the optical information, and various data according to the instruction from the system control unit 1003. The rear depth-of-field designation unit 1018 designates the optimum rear depth-of-field surface for the attributes of the subject to be recognized by using the installation information and the depth of field according to the instruction from the system control unit 1003.

The communication unit 1012 distributes the image data to the control device 1100 via the network 1200. Further, the communication unit 1012 receives various commands transmitted from the control device 1100 and outputs them to the system control unit 1003. The commands transmitted from the control device 1100 mainly include live image request commands, surveillance camera 1000 installation information, resolution, zoom, focus, tilt angle, and pan / tilt setting value request commands, and setting commands. included.

The lens drive unit 1005 is composed of a drive system for a focus lens and a zoom lens and a motor for the drive source thereof, and is controlled by the imaging angle of view control unit 1006 and the focus control unit 1007. The image sensor driving unit 1008 changes (tilts) the angle of the sensor surface (imaging surface) of the image sensor such as CCD or CMOS with respect to the optical axis of the lens. The pan drive unit 1010 is composed of a mechanical drive system that performs a pan operation and a motor of a drive source, and the operation is controlled by the pan / tilt control unit 1011. The tilt drive unit 1013 is composed of a mechanical drive system that performs a tilt operation and a motor of a drive source, and the operation is controlled by the pan / tilt control unit 1011.

<Control device>
Next, the configuration and function of each part of the control device 1100 will be described. The control device 1100 typically assumes a general-purpose computer such as a personal computer, but is not limited thereto. The control device 1100 may be, for example, a mobile terminal which is an external device. The control device 1100 includes a communication unit 1101, a display unit 1102, a system control unit 1103, and an input unit (input means) 1104.

The communication unit 1101 transmits various commands issued from the control device 1100, and receives various data distributed from the surveillance camera 1000. The various commands include a request command for live video, installation information of the surveillance camera 1000, a request command for resolution, zoom, focus, tilt angle, and pan / tilt setting values. In addition, various commands include installation information of the surveillance camera 1000, resolution, zoom, focus, tilt angle, and pan / tilt setting commands. The various data mainly include information on the installation of the surveillance camera 1000, information on the resolution, information on the imaging angle of view including zoom, information on the focus, information on the tilt angle, information on pan / tilt, image data, and the like.

The display unit 1102 uses a liquid crystal display device or the like to display an image acquired from the surveillance camera 1000 and display a graphical user interface (GUI) for controlling the camera. The system control unit 1103 executes various operations based on the computer program stored in the program memory 1014 received from the surveillance camera 1000 via the communication unit 1101. The system control unit 1103 may execute various operations based on the computer program stored in the program memory (not shown) held by the control device 1100. The system control unit 1103 generates a camera control command according to the GUI operation of the user and transmits it to the surveillance camera 1000 via the communication unit 1101. Further, the system control unit 1103 displays image data received from the surveillance camera 1000 via the communication unit 1101 and data representing set values of the imaging angle of view including installation information and zoom, focus, tilt angle, and pan / tilt. Displayed in unit 1102. A pointing device such as a keyboard or a mouse is used as the input unit 1104, and the user of the control device 1100 operates the GUI via the input unit 1104.

<Mechanism of tilt control>
Next, the mechanism of tilt control will be described with reference to FIG. FIG. 3 is an explanatory diagram of tilt control, and shows an example of a usage state of the surveillance camera 1000. FIG. 3 shows a case where the environment imaged by the surveillance camera 1000 is viewed from the side. In FIGS. 3 to 6 thereafter, a surface including the optical axis 3002, which is lowered directly below the optical axis 3002 from the optical axis 3002, is described, and the line of intersection between the surfaces is represented as an intersection in the drawings. .. In FIG. 3, the surveillance camera 1000 captures the range of the angle of view 3001. The surveillance camera 1000 includes a lens (imaging optical system) 1050 and an image sensor 1051.

The focus surface 3003 (3003 [a], 3003 [b]) is the focus focusing position. The focus surface 3003 [a] in the non-fluttered state is perpendicular to the optical axis 3002 of the surveillance camera 1000. FIG. 3 shows an example in which the focus position is aligned with the focus position 3011 on the horizontal plane 3004. The horizontal plane 3004 may be, for example, the ground or a horizontal plane at the height of the attribute to be recognized, for example, the height of the face (height of 1.5 m from the ground). In FIG. 3, the distance between the surveillance camera 1000 and the subject in focus (focus position 3011) is defined as the subject distance 3005 (Do), and the distance between the lens 1050 and the image sensor 1051 is defined as the focal length 3006 (f).

In FIG. 3, the sensor surface 3008 shows the sensor surface when the tilt is not performed. The sensor surface 3010 shows the sensor surface when the image sensor 1051 is tilted by the tilt angle 3009 (b) from the sensor surface 3008 in the non-raised state. Considering the vertical tilt that rotates the image sensor 1051 around the center line in the long side direction of the image sensor 1051 as the rotation axis of the tilt of the image sensor 1051, the focal length on the optical axis 3002 even if the image sensor 1051 is tilted. 3006 does not change. The surface parallel to the lens 1050 is referred to as the lens surface 3013. Assuming that the intersection of the lens surface 3013 and the sensor surface 3010 when the image sensor 1051 is tilted is the intersection 3012, the focus surface 3003 [b] coincides with the surface connecting the intersection 3012 and the focus position 3011. At this time, the angle formed by the optical axis 3002 and the focus surface 3003 [b] is set to the angle 3007 (a). The height of the focus position 3011 can be matched with the height of the attribute of the subject to be recognized.

The tilt angle b when the image sensor 1051 is tilted so that the focus surface 3003 [b] and the optical axis 3002 form an angle a as described above is expressed by the following equation (1) based on the Scheimpflug principle. ..

b = arctan (f / (Do × tan (a)))… (1)
That is, it is possible to obtain the tilt angle 3009 (b) based on the subject distance 3005 (Do) and the angle 3007 (a) formed by the optical axis 3002 and the focus surface 3003 [b]. In particular, when the focus surface 3003 is tilted so as to overlap the horizontal plane 3004, the tilt angle b is the same as the angle a when the depression angle 3015 (c) of the surveillance camera 1000 is used. Therefore, the following equation (2) ).

b = arctan (f / (Do × tan (c)))… (2)
<Depth of field>
Next, the depth of field will be described with reference to FIGS. 4 to 6. Hereinafter, since the same reference numerals among the reference numerals used in FIGS. 3 to 7 have the same meaning, their description will be omitted. 4 to 6 are explanatory views of the depth of field, and like FIG. 3, show a case where the environment imaged by the surveillance camera 1000 is viewed from the side.

FIG. 4 is a schematic view showing an example of the depth of field in a non-fluttering state. In FIGS. 4 to 6, the horizontal plane 3004 is a horizontal plane at the height of the attribute to be recognized, or the height of the face in the present embodiment. The height of the horizontal plane 3004 with respect to the ground 4008 is defined as the height of the face 4001. The depth of field on the front side of the focus surface 3003 when viewed from the surveillance camera 1000 is the front depth of field 4002 (d2), and the surface on the front side of the focus surface 3003 by the front depth of field 4002 is the front depth of field surface. It is set to 4004. The intersection of the front depth of field surface 4004 and the optical axis 3002 is defined as the intersection 4006. The depth of field beyond the focus surface 3003 is referred to as the rear depth of field 4003 (d1), and the surface beyond the focus surface 3003 by the rear depth of field 4003 is referred to as the rear depth of field 4005. The intersection of the rear depth of field surface 4005 and the optical axis 3002 is defined as the intersection 4007.

Assuming that the effective aperture is D and the permissible circle of confusion diameter is δ, the hyperfocal length H is expressed by the following equation (3).

H = f × D / δ… (3)
At this time, the rear depth of field d1 and the front depth of field d2 are represented by the following equations (4) and (5), respectively. Here, ^ 2 represents the square. For example, Do ^ 2 = Do × Do.

d1 = Do ^ 2 / (H-Do) ... (4)
d2 = Do ^ 2 / (H + Do) ... (5)
The depth of field is represented by the sum of the rear depth of field d1 and the front depth of field d2. In the present embodiment, the equations (3) to (5) are approximate equations when the distance Do to the subject is sufficiently larger than the focal length f and the focal length f is sufficiently larger than the allowable confusion circle diameter δ. is there.

When the distance to the subject is more than a certain distance, the resolution is insufficient and there is a distance at which the attributes of the subject cannot be recognized, and this surface is called the recognition limit surface 4009. The recognition limit surface 4009 will be described later with reference to FIG. 7. The attributes of the subject to be recognized include the height, length, orientation, and required number of pixels of the attributes, which are called attribute information. In the present embodiment, a person is taken as a subject, and a human face is taken as an example of a recognition attribute. The intersection of the recognition limit surface 4009 and the optical axis 3002 is defined as the intersection 4010. Subjects 4011 to 4016 represent people. In FIG. 4, since the subjects 4013 and 4014 are located within the depth of field, they are in focus, and the other subjects 4011, 4012, 4015, and 4016 are out of focus. Subject 4013 is located on the stairs 4017. The intersection 4018 is the intersection of the horizontal plane 3004 and the recognition limit plane 4009. The recognition limit distance 4019 (LL) is the distance from the surveillance camera 1000 to the recognition limit surface 4009.

The x-axis 4100 is set on the optical axis 3002 with the center of the lens 1050 as the origin and the imaging direction as positive. The y-axis 4101 is set on the lens surface 3013 with the center of the lens 1050 as the origin and the upper side as the positive.

FIG. 5 is a schematic view showing an example of the depth of field in a state where the focus surface 3003 is set so as to overlap the horizontal plane 3004 by tilting the image sensor 1051. The intersection 3012 in FIG. 5 is located on the horizontal plane 3004, and the angle a and the angle c coincide with each other. The focal length 3006 on the optical axis 3002 does not change even if the image sensor is blown up. Therefore, if the tilt angle b is sufficiently small, the positions of the intersections 4006 and 4007 do not change. The front depth of field surface 4004 is a surface connecting the intersection 3012 and the intersection 4006, and the rear depth of field surface 4005 is a surface connecting the intersection 3012 and the intersection 4007. In FIG. 5, at least the face portion of the subjects 4013 and 4016 is located outside the depth of field, indicating that they are out of focus. On the other hand, at least the face portion of the subjects 4011, 4012, 4014, and 4015 is located within the depth of field, indicating that they are in focus.

FIG. 6 is a schematic view showing an example of the depth of field for obtaining the optimum tilt angle for face recognition. If the surface is farther than the recognition limit surface 4009, even if the subject is in focus, the required resolution cannot be obtained, so that the face cannot be recognized. In the present embodiment, in order to recognize the face of the subject, it is considered sufficient that the focus is in the region above the face height 4001 and in front of the recognition limit surface 4009. However, the present embodiment is not limited to this standard, and may be another standard, for example, a region above the height of the ground. In order to satisfy these criteria, as shown in FIG. 6, the tilt angle b can be set so that the rear depth of field surface 4005 passes through the intersection 4018, so that the tilt angle can be minimized. Here, it can be seen that the depth of field shown in FIG. 6 is wider in the vertical direction than the depth of field shown in FIG. For example, the subjects 4013 and 4016 that are out of focus in FIG. 5 are in focus.

In addition, instead of the linear front depth of field surface 4004 and the rear depth of field surface 4005, a non-linear front depth of field surface 4004'and a rear depth of field depth of field 4005' may be used. The front depth-of-field surface 4004'and the rear depth-of-field surface 4005' are calculated as follows. Assuming that the subject exists on the focus surface 3003 other than the focus position 3011, the distance between the surveillance camera 1000 and the focus surface 3003 can be regarded as the subject distance Do. When the subject distance Do is sufficiently long with respect to the angle of view 3001, the line connecting the surveillance camera 1000 and the focus surface 3003 can be regarded as parallel to the optical axis (not shown). At this time, the equation (4) of the rear depth of field d1 and the equation (5) of the front depth of field d2 hold for the front and rear of the focus surface 3003. For example, in FIG. 6, the focus surface 3003 becomes farther as it goes upward. That is, the subject distance Do becomes far. When the subject distance Do is sufficiently smaller than the hyperfocal length H, the depth of field increases non-linearly in proportion to the square of the subject distance Do from the equations (4) and (5).

<Calculation of maximum distance required for subject recognition>
Next, the recognition limit surface 4009 will be described with reference to FIG. 7. FIG. 7 is an explanatory diagram of the recognition limit surface 4009. The horizontal angle of view 7001 (ho_deg) and the vertical angle of view 7002 (ve_deg) represent the horizontal and vertical angles of view of the angle of view 3001, respectively. The recognition limit surface 4009 shown in FIGS. 4 to 6 is represented by a line, and the recognition limit surface 4009 is a surface as shown in FIG. The subject 7003 exists on the recognition limit surface 4009. The horizontal line segment 7004 (length w, resolution w_pix) and the vertical line segment 7005 (length h, resolution h_pix) indicate a range that can be imaged on the recognition limit surface 4009, and the optical axis 3002 is located at the center of the range. Pass. The horizontal line segment 7004 and the vertical line segment 7005 represent the length as well as the resolution. For example, when the surveillance camera 1000 acquires an image at 1080p for recognition purposes, the resolution w_pix of the horizontal line segment 7004 is 1920, and the resolution h_pix of the vertical line segment 7005 is 1080. The length w of the horizontal line segment 7004 and the length h of the vertical line segment 7005 are represented by the following equations (6) and (7), respectively.

w = 2π × LL × (ho_deg / 360)… (6)
h = 2π × LL × (ve_deg / 360)… (7)
The equations (6) and (7) are approximate equations when the angle of view 3001 is small.

Here, consider recognizing a certain attribute of the subject. As the attribute information, the length (meter) of the attribute is set to attr_l, the direction of the attribute is set to the horizontal direction, and the resolution (pixels) required for recognizing the attribute is set to attr_pix. For example, when recognizing a human face, attr_pix = 20 pixels is required for a face width of attr_l = 0.16 meters in the horizontal direction. Considering the case of recognizing horizontal attributes such as face recognition, the following equation (8) holds.

attr_pix = w_pix × (attr_l / w)… (8)
Similarly, considering the case of recognizing the attribute in the vertical direction, the following equation (9) holds.

attr_pix = h_pix × (attr_l / h)… (9)
From the equations (6) to (9), the following equations (10) and (11) are established in the horizontal direction and the vertical direction, respectively.

LL = attr_l × (w_pix / attr_pix) / (2π × (ho_deg / 360))… (10)
LL = attr_l × (h_pix / attr_pix) / (2π × (ve_deg / 360))… (11)
<Flowchart that realizes tilting optimized for face recognition>
Next, with reference to FIG. 8, a method (information processing method) for realizing tilting optimized for face recognition will be described. FIG. 8 is a flowchart of the information processing method in the present embodiment. In this embodiment, the surveillance camera 1000 is mainly used for the purpose of acquiring (imaging) an image that is premised on analysis using a computer program, such as age / gender recognition of a person, vehicle number recognition, and number counting. It is applicable when installing. Each step of FIG. 8 is mainly executed by each unit based on the instruction of the system control unit 1003 or the system control unit 1103 of the camera system 100.

First, in step S8001, the user operates the input unit 1104 of the control device 1100 to set installation information such as the installation position of the surveillance camera 1000 and the focus position 3011. The system control unit 1103 of the control device 1100 transmits the installation information to the system control unit 1003 of the surveillance camera 1000 via the network 1200. The installation information management unit 1015 sets and manages the installation information of the surveillance camera 1000 based on the instruction of the system control unit 1003. When setting the focus position, the system control unit 1003 sets the focus value for the focus control unit 1007. The system control unit 1003 acquires installation information such as the installation position and focus position of the surveillance camera 1000 by using various sensors (not shown) provided in the surveillance camera 1000, and sets them in the installation information management unit 1015. May be good.

Subsequently, in step S8002, the recognition limit distance calculation unit 1016 has attribute information (height, length, orientation, number of pixels, etc.) stored in the recognition limit distance calculation unit 1016 based on the instruction of the system control unit 1003. To read. Further, the recognition limit distance calculation unit 1016 reads the resolution from the image processing unit 1002 and the angle of view from the imaging angle of view control unit 1006. The recognition limit distance calculation unit 1016 calculates the recognition limit distance 4019 from the various data read by the equations (10) and (11) described with reference to FIG. 7.

Subsequently, in step S8003, the depth of field calculation unit 1017 receives optical information (effective diameter, allowable confusion circle diameter, etc.) stored in the depth of field calculation unit 1017 based on the instruction of the system control unit 1003, and The focal lengths are read from the imaging angle of view control unit 1006. Further, the depth of field calculation unit 1017 reads the installation information (installation position of the surveillance camera 1000, focus position 3011, etc.) from the installation information management unit 1015 and the depression angle from the pan / tilt control unit 1011. Further, the depth of field calculation unit 1017 calculates the subject distance Do, which is the distance from the surveillance camera 1000 to the focus position 3011. Then, the depth of field calculation unit 1017 calculates the depth of field from these various data by the equations (4) and (5) described with reference to FIG. From the depth of field (rear depth of field d1 and front depth of field d2) calculated here and the focus position 3011, the intersection 4006 and the intersection 4007 can be obtained.

Subsequently, in step S8004, the rear depth-of-field surface designation unit 1018 acquires the recognition limit surface 4009 from the recognition limit distance 4019 calculated in step S8002 based on the instruction of the system control unit 1003. Further, the rear depth of field designation unit 1018 acquires the horizontal plane 3004 from the height of the attribute, calculates the intersection 4018, and designates the intersection 4018 as the rear depth of field surface 4005.

Subsequently, in step S8005, the image sensor control unit 1009 calculates the optimum tilt angle for face recognition based on the instruction of the system control unit 1003. First, the image sensor control unit 1009 obtains a straight line passing through two points, the intersection 4007 calculated in step S8003 and the intersection 4018 designated in step S8004. This straight line is the rear depth of field surface 4005 shown in FIG.

Hereinafter, an xy plane formed by the x-axis 4100 and the y-axis 4101 will be considered. Generally, a straight line passing through two points (x1, y1) and (x2, y2) can be represented by the following equation (13).

(X2-x1) (y−y1) = (y2-y1) (x−x1)… (13)
Here, assuming that the coordinates of the intersection 4007 are (x4007, 4007) and the coordinates of the intersection 4018 are (x4018, y4018), the following equation (14) is established from the equation (13).

y = {(y4018-y4007) / (x4018-x4007)} x + {(-x4007) (y4018-y4007) / (x4018-x4007) + y4007} ... (14)
The equation (14) can be summarized as the following equation (15).

y = Ax + B ... (15)
here,
A = {(y4018-y4007) / (x4018-x4007)}
B = {(-x4007) (y4018-y4007) / (x4018-x4007) + y4007}
Is.

Next, the intersection 3012 between the rear depth of field surface 4005 and the lens surface 3013 shown in FIG. 6 is obtained. Generally, assuming that the two straight lines are y = ax + b and y = cx + d, when a ≠ c, the intersection of the two straight lines is represented by the following equation (16).

((Db) / (ac), (ad-bc) / (ac)) ... (16)
Since the rear depth of field surface 4005 is represented by the equation (15) and the lens surface 3013 is represented by x = 0, (0, B) can be obtained from the two straight lines as the intersection point 3012.

Next, when the focus surface 3003 is formed into the form of a straight line equation (15) passing through two points of the intersection 3012 (0, B) and the focus position 3011 (x3011, 0), the following equation (17) is obtained. expressed.

y = (-B / x3011) x + B ... (17)
At this time, the angle 3007a formed by the focus surface 3003 and the x-axis 4100 is expressed by the following equation (18) from the equation (17).

tan (a) = −B / x3011… (18)
Next, the image sensor control unit 1009 reads the focal length f stored in the image pickup angle of view control unit 1006. Further, the image sensor control unit 1009 calculates the tilt angle b by using the subject distance Do calculated in step S8003 and the equations (18) and (1). The tilt angle b is expressed by the following equation (19).

b = arctan ((f × x 3011) / (Do × (−B)))… (19)
Subsequently, in step S8006, the image sensor control unit 1009 controls the image sensor drive unit 1008 based on the instruction of the system control unit 1003, and determines the tilt angle of the image sensor 1051 as the tilt angle b calculated in step S8005. Set to. The image sensor control unit 1009 and the image sensor drive unit 1008 form an angle control means for controlling the tilt angle.

If there is an object different from the subject such as an obstacle within the range in which the subject can be recognized, the rear depth of field may be specified so that the obstacle is outside the depth of field. Further, by designating the front depth of field plane, the recognition of the subject in front may be considered. Here, the front subject is imaged larger than the rear subject. Therefore, even if the blur peculiar to the tilt appears, the subject in front is less susceptible to the blur than the subject in the rear.

As described above, in the present embodiment, the information processing apparatus includes an acquisition means (installation information management unit 1015), a first calculation means (recognition limit distance calculation unit 1016), and a second calculation means (depth of field calculation unit 1017). ), And a third calculation means (imaging element control unit 1009). The acquisition means acquires the installation information of the imaging device (surveillance camera 1000). The first calculation means calculates the limit distance at which the subject can be recognized based on the installation information and the attribute information of the subject. The second calculation means calculates the front depth of field and the rear depth of field when the subject is in focus. The third calculation means calculates the tilt angle based on the limit distance and the rear depth of field.

Preferably, the information processing apparatus determines the recognition limit surface 4009 located at the limit distance and the rear depth of field surface 4005 corresponding to the rear depth of field (rear depth of field designation unit 1018). Has. The third calculation means calculates the tilt angle based on the recognition limit plane and the rear depth of field plane. More preferably, the determining means determines the forward depth of field surface 4004 corresponding to the forward depth of field. The third calculation means calculates the tilt angle based on the front depth of field plane. Further, preferably, the third calculation means calculates the tilt angle so as to focus on the limit distance. Further, preferably, the third calculation means calculates the tilt angle based on the position of the subject existing at the limit distance. Further, preferably, the third calculation means calculates the tilt angle based on the position of an obstacle existing before the limit distance.

Preferably, the attribute information includes at least one of the height, length, orientation, or required number of pixels of the attribute (eg, face) of the subject (eg, person). Also preferably, the installation information includes at least one of the installation position or the focus position of the imaging device. Further, preferably, the information processing apparatus has angle control means (1008, 1009) for controlling the tilt angle calculated by the third calculation means.

Preferably, the camera system 100 has an input means (input unit 1104) for inputting installation information of the image pickup device, and a control device 1100 for transmitting the installation information to the image pickup device via the network 1200.

According to the information processing device of the present embodiment, the minimum tilt angle can be set so as to focus on the limit distance that the subject can recognize, so that the optimum tilt angle for subject recognition can be obtained. In the present embodiment, the tilt angle of the image sensor is changed, but the present invention is not limited to this, and the tilt angle of the lens (imaging optical system) may be changed.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

According to the present embodiment, since the minimum tilt angle can be set so as to focus to the limit distance at which the subject can be recognized, it is possible to obtain the optimum tilt angle for recognizing the subject. Further, in order to suppress the blurring peculiar to the tilt within the range in which the subject can be recognized, the range in which the subject can be recognized can be expanded in the vertical direction. Therefore, according to the present embodiment, it is possible to provide an information processing device, an imaging device, a camera system, an information processing method, a program, and a storage medium capable of determining a tilt angle suitable for recognizing a subject. ..

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and modifications can be made within the scope of the gist thereof.

1000 Surveillance camera (information processing device)
1009 Image sensor control unit (third calculation means)
1015 Installation Information Management Department (Acquisition means)
1016 Recognition limit distance calculation unit (first calculation means)
1017 Depth of field calculation unit (second calculation means)

Claims (14)

Acquisition means for acquiring installation information of the imaging device,
A first calculation means for calculating the limit distance at which the subject can be recognized based on the installation information and the attribute information of the subject, and
A second calculation means for calculating the front depth of field and the rear depth of field when the subject is focused, and
An information processing apparatus comprising: a third calculation means for calculating a tilt angle of an image sensor or an optical system based on the limit distance and the rear depth of field. Further having a determination means for determining a recognition limit surface located at the limit distance and a rear depth of field surface corresponding to the rear depth of field.
The information processing apparatus according to claim 1, wherein the third calculation means calculates the tilt angle based on the recognition limit surface and the rear depth of field surface. The determining means determines a forward depth of field surface corresponding to the forward depth of field.
The information processing apparatus according to claim 2, wherein the third calculation means calculates the tilt angle based on the front depth of field plane. The information processing apparatus according to any one of claims 1 to 3, wherein the third calculation means calculates the tilt angle so as to focus on the limit distance. The information processing apparatus according to any one of claims 1 to 4, wherein the third calculation means calculates the tilt angle based on the position of the subject existing at the limit distance. The information processing according to any one of claims 1 to 4, wherein the third calculation means calculates the tilt angle based on the position of an obstacle existing in front of the limit distance. apparatus. The information processing according to any one of claims 1 to 6, wherein the attribute information includes at least one of the height, length, orientation, or required number of pixels of the attribute of the subject. apparatus. The information processing device according to any one of claims 1 to 7, wherein the installation information includes at least one of the installation position or the focus position of the image pickup device. The information processing apparatus according to any one of claims 1 to 8, further comprising an angle control means for controlling the tilt angle calculated by the third calculation means. An imaging means that captures the subject,
An image pickup apparatus comprising the information processing apparatus according to any one of claims 1 to 9. The imaging device according to claim 10,
A camera system comprising an input means for inputting the installation information of the image pickup apparatus, and a control device for transmitting the installation information to the image pickup apparatus via a network. Steps to acquire installation information of the image pickup device and
A step of calculating the limit distance at which the subject can be recognized based on the installation information and the attribute information of the subject, and
A step of calculating the front depth of field and the rear depth of field when the subject is focused, and
An information processing method comprising: a step of calculating a tilt angle based on the limit distance and the rear depth of field. A program comprising causing a computer to execute the information processing method according to claim 12. A computer-readable storage medium that stores the program according to claim 13.