JP2023063765A - Image processing device, image processing method, image processing system, and program - Google Patents

Abstract

To control a camera by quickly detecting an object not related to photographing an image that has entered a photographing range of the camera.SOLUTION: An image processing device acquires an image photographed by imaging means, sets a detection area, which is an area up to a constant width inside from an outer frame of the image in the image, detects an object that has entered the detection area, determines whether the object is an obstacle not related to photographing an image by the imaging means, and controls the angle of view of the imaging means so as to remove the object from the photographing range by the imaging means when the object is determined to be an obstacle.SELECTED DRAWING: Figure 6

Description

The present invention relates to an image processing device, an image processing method, an image processing system, and a program.

2. Description of the Related Art Currently, in order to produce images such as movies and music videos, there is an increasing number of systems that remotely control cameras (for example, web cameras) connected to a network. In such a system, an application that can change the settings of each camera and perform PTZ (pan/tilt/zoom) control of the camera can be used while continuing image capturing by the camera. A camera operator can use a device such as a joystick to perform operations for changing the settings and controlling the PTZ.

Objects (including people) other than the performers that should appear in the video, or objects unrelated to the video (competitions (information sent by the staff to the performers ), etc.) may be unintentionally reflected in the shooting range, resulting in "out of sight". In such a case, the camera operator directs the camera in a pan/tilt direction to avoid the object, or controls the camera so that the object is not captured by zooming. Further, Japanese Patent Application Laid-Open No. 2002-200001 discloses a technique that allows an operator to receive a notification when a boundary where "out of sight" occurs is set in advance and "out of sight" is likely to occur.

JP 2018-125729 A

When operating the camera remotely, the following problems may arise if an unrelated object (including a person) is likely to suddenly enter the shooting range. .
First of all, in remote operation, it is difficult to operate instantaneously in response to the occurrence of "cut-out". You may miss it.
Further, according to the technique disclosed in Patent Document 1, it is possible for the operator to receive a notification when "out of sight" is likely to occur, but it is necessary to set in advance the boundary at which "out of sight" will occur. In addition, in the literature, the operator can receive a notification when the camera is operated to cross a preset boundary, but when an object unrelated to shooting suddenly enters the shooting range, not taken into account.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a technique for quickly detecting an object that has entered the shooting range of the camera and is not related to shooting and controlling the camera.

As one means for achieving the above object, the image processing apparatus of the present invention has the following configuration. Acquisition means for acquiring an image captured by an imaging means; setting means for setting a detection area, which is an area within a certain width from an outer frame of the image in the image; detecting means for detecting; determining means for determining whether the object is an obstacle unrelated to photographing by the imaging means; and if the object is determined to be the obstacle by the determining means, and a control means for controlling the angle of view of the imaging means so as to exclude it from the imaging range of the imaging means.

According to the present invention, it is possible to quickly detect an object that has entered the shooting range of the camera and is not related to shooting, and to control the camera.

FIG. 1 shows a configuration example of the entire image processing system. FIG. 2 shows an example hardware configuration of the camera 102 . FIG. 3 shows an example of the software functional configuration of the camera 102 . FIG. 4 shows an example hardware configuration of the client device 103 . FIG. 5 shows an example of software functional configuration of the client device 103 . FIG. 6 is a flowchart of processing executed by the client device 103 according to the first embodiment. FIG. 7 is a diagram for explaining an imaging range according to the first embodiment. FIG. 8 is a flowchart of processing executed by the client device 103 according to the first embodiment. FIG. 9 explains the imaging range according to the third embodiment.

Embodiments for carrying out the present invention will be described in detail below with reference to the accompanying drawings. The embodiments described below are examples of means for realizing the present invention, and should be appropriately modified or changed according to the configuration of the apparatus to which the present invention is applied and various conditions. It is not limited to the embodiment. Also, not all combinations of features described in the present embodiment are essential for the solution means of the present invention.

<First embodiment>
[System configuration]
FIG. 1 is a diagram showing a configuration example of the entire image processing system according to the first embodiment. The image processing system according to this embodiment includes a network 101, cameras 102-1 to 102-n (n>1), and a client device 103. FIG. Cameras 102-1 through 102-n may be collectively referred to as cameras 102 in the following description. The camera 102 and client 103 function as image processing devices.

A network 101 is a network that connects a camera 102 and a client device 103 . The network 101 is implemented, for example, by a plurality of routers, switches, cables, etc. conforming to communication standards such as ETHERNET (registered trademark). Note that the network 101 may be implemented by the Internet, a wired LAN (Local Area Network), a wireless LAN, a WAN (Wide Area Network), or the like.

The camera 102 captures an image through a lens (not shown) to obtain a captured image. The captured image is an image captured within the imaging range (imaging range) of the camera 102 . In addition, in this embodiment, an image shall contain a still image and/or a moving image. The camera 102 also manages information of the camera 102 (such as camera type information). The camera 102 can transmit captured images and information about the camera 102 via the network 101 in response to a request from the client device 103 . Further, the camera 102 may actively transmit captured images acquired by the camera 102 and information of the camera 102 to the client device 103 when the client device 103 is connected in advance.

The client device 103 is a general client terminal such as a personal computer (PC), tablet, smart phone, or the like. The client device 103 can transmit a request to the camera 102 according to the user's operation.

[Configuration of camera 102]
The hardware configuration and software functional configuration of the camera (imaging device) 102 according to the present embodiment will be described with reference to FIGS. 2 and 3. FIG.
FIG. 2 is a diagram showing a hardware configuration example of the camera 102 according to this embodiment.
The camera 102 has an internal bus 201 inside. The camera 102 includes a CPU (Central Processing Unit) 202, a primary storage device 203, a secondary storage device 204, an imaging unit 205, a pan/tilt/zoom (PTZ) drive unit 206, which are connected to an internal bus 201. And, it is configured to have a network I/F 207 .

The CPU 202 comprehensively controls operations in the camera 102 . That is, the CPU 202 can control the primary storage device 203, the secondary storage device 204, the imaging unit 205, the PTZ driving unit 206, and the network I/F 207, respectively.

The primary storage device 203 is a writable high-speed storage device represented by, for example, RAM (Random Access Memory). The primary storage device 203 is loaded with, for example, an OS (operating system), various programs, various data, and the like. The primary storage device 203 can also be used as a work area when the CPU 202 executes the OS and various programs.

The secondary storage device 204 is a non-volatile storage device represented by, for example, an HDD (Hard Disc Drive), flash memory, SD card, and the like. The secondary storage device 204 can be used not only as a permanent storage area for the OS, various programs, and various data, but also as a short-term storage area for various data.
The type information of the camera 102 is stored in the secondary storage device 204 . In the present embodiment, information indicating whether the camera 102 is capable of pan/tilt/zoom functions is stored as the type information. For example, the information may be a PTZ machine if the camera is capable of panning, tilting, and zooming, a zooming machine if the camera is only capable of zooming, and a fixed machine if the camera is not capable of panning, tilting, or zooming. is.

The imaging unit 205 includes, for example, an optical lens unit, an optical system for controlling aperture, zoom, focus, etc., and an imaging element for converting light (image) introduced through the optical lens unit into an electric signal. . As the imaging device, a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge Coupled Device) is generally used. The imaging unit 205 converts the subject light imaged by the lens included in the imaging unit 102 into digital data (electrical signal) by the imaging element and outputs the digital data (electrical signal).

A PTZ driving unit 206 controls a motor or the like (not shown) to control the PTZ (pan, tilt, zoom) of the camera 102, thereby controlling the angle of view of the camera 102, that is, the shooting range. The angle of view is a photographing range (imaging range) expressed as an angle. Pan control moves the angle of view of the camera 102 in the horizontal direction. Tilt control moves the angle of view of the camera 102 in the vertical direction. By controlling the zoom, the angle of view of the camera 102 is enlarged/reduced.
Also, the PTZ driving unit 206 may be configured to control not only PTZ control but also settings and control of the camera 102 such as focus and white balance. Also, if the camera 102 is a video camera that can only zoom or an interchangeable lens camera that does not have a zoom function, the camera 102 does not need to include the PTZ control unit 206 .
A network I/F 207 is an interface for connecting with the network 101 .

Next, a software functional configuration executed by the CPU 202 of the camera 102 will be described. FIG. 3 is a diagram showing an example of the software function configuration of the camera 102 according to this embodiment. Processing by the functional configuration shown in FIG. 3 can be performed by the CPU 202 executing various programs stored in the primary storage device 203 and/or the second storage device 204 .

A communication control unit 301 controls communication with the client device 103 via the network I/F 207 through the network 101 . Specifically, the communication control unit 301 controls transmission and reception via the network I/F 207 . For example, the communication control unit 301 can receive a captured image acquisition request and a PTZ control request (hereinafter also referred to as a PTZ command) to the PTZ control unit 207 of the camera 102 from the client device 103 . The request may use a common communication protocol, such as HTTP (Hypertext Transfer Protocol).

The image processing unit 302 performs image processing (conversion processing) on the electrical signal generated by the imaging unit 205, and generates (acquires) the processed image data as a captured image. For example, the image processing unit 302 uses image data such as JPEG, H.264, etc., as a photographed image. 264 video data is generated. The image processing can be performed, for example, in response to a captured image acquisition request received from the client device 103 via the communication control unit. The image processing unit 302 may be configured to perform the image processing in response to any other instruction.

The PTZ control unit 303 controls the PTZ control unit 207 based on PTZ commands received from the client device 103 via the communication control unit 301 . Note that the PTZ control unit 303 not only physically controls the operation of the PTZ control unit 207, but also cuts out a captured image and changes the cut-out position to implement digital PTZ in a pseudo manner. good too.

[Configuration of client device 103]
Next, the hardware configuration and software functional configuration of the client device (control device) 103 according to this embodiment will be described with reference to FIGS. 4 and 5. FIG.
FIG. 4 is a diagram showing a hardware configuration example of the client device 103 according to this embodiment.
The client device 103 has an internal bus 401 therein. The client device 103 includes a CPU 402, a primary storage device 403, a secondary storage device 404, a user input/output I/F 405, and a network I/F 406 connected to the internal bus 401. .

The CPU 402 comprehensively controls operations in the client device 103 . That is, the CPU 402 can control each of the primary storage device 403 , secondary storage device 404 , user input/output I/F 405 and network I/F 406 .

The primary storage device 403 is, for example, a writable high-speed storage device represented by RAM. The primary storage device 403 is loaded with, for example, an OS, various programs, various data, and the like. The primary storage device 403 can also be used as a work area when the CPU 402 executes the OS and various programs.
The secondary storage device 404 is a nonvolatile storage device represented by, for example, an HDD, flash memory, SD card, and the like. The secondary storage device 404 can be used not only as a permanent storage area for the OS, various programs, and various data, but also as a short-term storage area for various data.

A user input/output I/F 405 is an I/F for the client device 103 to receive operations from users, input user information, and output images and the like to the user. Examples of the user input/output I/F 405 include a display (display unit), touch panel, keyboard, mouse, microphone, speaker, and the like. Here, the touch panel and the display may be configured as an integrated type such as a touch panel-equipped display (touch display) that shows the touch position.
A network I/F 406 is an interface for connecting with the network 101 .

Next, a software functional configuration executed by the CPU 402 of the client device 103 will be described. FIG. 5 is a diagram showing an example of the software functional configuration of the client device 103 according to this embodiment. Processing by the functional configuration shown in FIG. 5 can be performed by the CPU 402 executing various programs stored in the primary storage device 403 and/or the second storage device 404 .

A communication control unit 501 controls communication with the camera 102 from the network I/F 406 through the network 101 . Specifically, the communication control unit 501 performs transmission and reception control via the network I/F 406 . For example, the communication control unit 501 can transmit an acquisition request for a captured image acquired by the camera 102 and a PTZ control request (PTZ command) to the PTZ control unit 207 of the camera 102 . The request may use a common communication protocol, such as HTTP.

A display control unit 502 performs control for displaying captured images acquired from the camera 102 and buttons for controlling the camera 102 on a display, which is one form of the user input/output I/F 405 . The display control unit 502 can receive input via the mouse and keyboard as the user input/output I/F 405 and input by touch operation via the touch display, and perform various displays.

The object detection unit 503 sets a detection area (detection area) for the captured image (shooting range) acquired via the communication control unit 501, and detects an object that enters the detection area (hereinafter also referred to as an “intruding object”). ). Then, the object detection unit 503 determines whether or not the intruding object is an object unrelated to shooting (an object to be excluded from the shooting range; hereinafter also referred to as an “obstacle”). In addition, in this embodiment, the "object" can also include a person.

In this embodiment, the detection area is defined as an area from the outer frame of the captured image/capturing range (angle of view) of the camera 102 to the inner side of a certain width. The detection area may be set by the camera 102, and the object detection unit 503 may acquire information on the detection area via the communication control unit 501 and set the detection area.

The object detection unit 503 or the camera 102 can determine whether an object in the captured image is an object to be detected by the object detection unit 503 . When the determination is made by the object detection unit 503, for example, the object detection unit 503 analyzes the captured image acquired via the communication control unit 501 by image processing, and determines whether or not the object in the captured image is the object to be detected. can be determined. Further, when the determination is made by the camera 102, for example, the image processing unit 302 of the camera 102 analyzes the captured image by image processing, and determines whether or not the object in the captured image is the object to be detected. Then, the camera 102 can notify the object detection unit 503 of information on the object to be detected through communication.

The object detection unit 503 instructs the camera control unit 505 to control the PTZ of the camera 102 when it determines that the intruding object is an obstacle (object unrelated to shooting). For example, the object detection unit 503 instructs the camera control unit 505 to remove the intruding object from the shooting range.

A timer unit 504 performs timing control for predetermined determination of an intruding object. Timer section 504 is used by object detection section 503 . For example, when the timer unit 504 measures time and confirms that a predetermined time has elapsed since an intruding object was present in the detection area, the object detecting unit 503 determines that the intruding object is an obstacle.

A camera control unit 505 controls the camera 102 according to instructions from the object detection unit 503 . For example, when the object detection unit 503 instructs the camera control unit 505 to move the intruding object out of the shooting range, the camera control unit 505 issues a PTZ command for controlling pan/tilt/zoom of the camera 102 in accordance with the instruction. Generate. The camera control unit 505 then transmits the PTZ command to the camera 102 via the communication control unit 501 . The control of pan/tilt/zoom may be control of at least one of pan/tilt/zoom. For example, it may be a control for panning and tilting in the direction opposite to the intruding direction of the intruding object. Also, the control range for at least one of panning, tilting, and zooming may be set in advance, or may be automatically set so that an intruding object is out of the shooting range.
Further, for example, when the object detection unit 503 instructs the camera control unit 505 to restore the PTZ control that has been performed once, the camera control unit 505 issues a PTZ command for restoring the PTZ settings in response to the instruction. It can also be generated and transmitted to the camera 102 via the communication control unit 501 .

Note that the camera control unit 505 may be configured to control the camera 102 based on input from the user via the user input/output I/F 405 .

[Process flow]
FIG. 6 shows a flowchart of processing executed by the client device 103 according to this embodiment. Note that the client device 103 can repeatedly perform the processing of this flowchart.

In S601, the object detection unit 503 detects (determines) whether an object has entered the detection area. In this embodiment, the detection area is defined as an area from the outer frame of the captured image/capturing range (angle of view) of the camera 102 to the inner side of a certain width. If the presence (intrusion) of an object in the detection area is not detected within the predetermined time (No in S601), the process ends. The fixed time can be any time. On the other hand, if it is determined that an object exists within the detection area (Yes in S601), the process proceeds to S602.

In S602, the object detection unit 503 sets a flag (object intrusion flag) indicating that an intruding object exists in the detection area (flag=1). In response to this, the object detection unit 503 activates the timer unit 504 to start measuring time. In S603, the object detection unit 503 checks the timer unit 504 and determines whether a predetermined time has elapsed since the start of time measurement. If the predetermined time has not elapsed (No in S603), the timer unit 504 continues timekeeping. If the predetermined time has passed (Yes in S603), the process proceeds to S604.

In S604, the object detection unit 503 determines whether the object (intruding object) detected in S601 still exists within the detection area after a predetermined time has elapsed. If it is determined that an intruding object does not exist within the detection area (No in S604), the process proceeds to S605, the object detection unit 503 clears the object intrusion flag (flag=0), and cancels the timer unit 504. End the process.

On the other hand, if it is determined that the intruding object still exists within the detection area after the predetermined time has elapsed (Yes in S604), the object detection unit 503 determines that the intruding object is an obstacle (object unrelated to shooting). do. In this case, the object detection unit 503 instructs the camera 102 to move the intruding object out of the shooting range, and the process proceeds to S606. In S606, the camera control unit 505 performs PTZ control (angle of view control) on the camera 102 so that the intruding object is out of the shooting range. For example, the camera control unit 505 transmits to the camera 102 via the communication control unit 501 a PTZ command indicating control of at least one of panning, tilting, and zooming to remove the detection area from the shooting range. Upon receiving the command, the PTZ control unit 303 of the camera 102 performs digital PTZ control according to the command, and adjusts the angle of view of the camera 102 so that the intruding object is out of the shooting range.
Furthermore, in S606, the object detection unit 503 stops the operation of the timer unit 504 (releases the timer), and the process proceeds to S607.

In S607, the object detection unit 503 again determines (detects) whether or not the object detected in S601 exists within the detection area. If no intruding object is detected within the detection area within the predetermined time (No in S607), the intruding object flag is cleared (flag=0) in S608, and the process proceeds to S609. The fixed time can be any time. In S609, the camera control unit 505 controls the camera 102 so as to restore the PTZ control in S606. For example, the camera control unit 505 transmits a PTZ command indicating returning PTZ control to the camera 102 via the communication control unit 501 . Upon receiving the command, the PTZ control unit 303 of the camera 102 performs PTZ control so as to restore the original shooting range according to the command. After that, the process is terminated. On the other hand, if it is determined that an intruding object exists within the detection area within the predetermined time (Yes in S607), the determination is continued.

In the above description, in S606, the camera 102 performs digital PTZ control according to the PTZ command transmitted by the object detection unit 503. However, the present invention is not limited to this, and optical PTZ control may be performed, for example. In this case, the object detection unit 503 cannot determine from the captured image whether an intruding object exists in the detection area (S607). may be sent to the camera 102 to restore the original settings.

In the above description, in S601, the object detection unit 503 detects (determines) whether an object has entered the detection area. For example, when an object enters from outside the detection area, the object detection unit 503 determines that the object is to be detected. You may not judge. That is, in this case, the object detection unit 503 performs the processing of S601 for an object that has entered from the outside of the detection area, and performs the processing of S601 for an object that has entered from the inside of the detection area toward the outside. can not.
Also, instead of the processing of S601 to S604, when the object detection unit 503 detects that an intruding object has entered from the outside to the inside of the detection area, the object is determined to be an obstacle (an object unrelated to shooting). may be configured to proceed to S606.

In the above description, in steps S602 to S604, the object detection unit 503 determines whether the intruding object is an obstacle (an object unrelated to shooting) based on whether the intruding object still exists within the detection area after the predetermined time has elapsed. However, the determination method is not limited to this. For example, the object detection unit 503 may determine whether or not an intruding object is an obstacle based on preliminarily registered performer face information, costume colors, and the like. That is, the object detection unit 503 determines whether or not the intruding object is a pre-registered object, and if it determines that the intruding object is not the registered object, determines that the intruding object is an obstacle. , the client device 103 can perform the processing from S606 onwards. Also, at this time, if the intruding object is not the registered object and a predetermined time has passed since the intrusion, the object detection unit 503 determines that the intruding object is an obstacle, and the client device 103 determines that the intruding object is an obstacle after step S606. may be processed.

In addition, in the present embodiment, the object detection unit 503 determines whether or not the object is the object to be detected by analyzing the captured image, but the present invention is not limited to this. For example, a person (performer, etc.) involved in shooting wears (attaches) a sensor or the like as a registered object, and the object detection unit 503 receives (detects) a signal (sensor signal) from the sensor. Thus, an intruding object may be determined to be an obstacle. That is, when the object detection unit 503 receives a signal from a sensor attached to an intruding object, it can determine that the intruding object is not an obstacle even if a predetermined time has passed since the signal was received. Conversely, when an intruding object enters the detection area and a predetermined time elapses without receiving a sensor signal, the object detection unit 503 determines that the intruding object is an obstacle, and the client device 103 determines that the intruding object is an obstacle in step S606. Subsequent processing can be performed.

Next, with reference to FIG. 7, the photographing range before and after PTZ control according to the present embodiment will be described. FIG. 7 is a diagram for explaining the imaging range (angle of view) according to this embodiment. In FIG. 7, a shooting range 701 indicates a shooting range (angle of view) captured by the camera 102 . A shooting range 701 can be a shooting range at the start of shooting. When an object 704 as an intruding object enters the detection area (shaded area) 702 (Yes in S601), the object detection unit 503 determines whether the object 704 stays in the detection area 702 for a predetermined time (S602). , S603). After that, if a predetermined time has passed while the object 704 remains in the detection area 702 (Yes in S603), the camera control unit 505 determines that the object 701 is an obstacle, and removes the detection area 702 from the shooting range 701. As shown, the PTZ command is transmitted to the camera 102 (S606). In response to this, the camera 102 performs PTZ control, and the imaging range 701 is changed to the imaging range 703 after PTZ control.

As described above, according to the present embodiment, the client apparatus 103 determines that an object detected as entering the detection area is an object (obstacle) unrelated to photographing when the object satisfies a predetermined condition, and photographs the object. Control the camera 102 out of range. This enables automatic detection of obstacles and automatic camera control by the client device 103 . Further, as shown in FIG. 1, the image processing system includes a plurality of cameras 102-1 to 102-n, and the client device 103 can perform the above control on each camera. Note that the client device 103 may be configured to perform the above processing only on images used for broadcasting.

In this embodiment, the control by the client device 103 has been described as an example, but the same description can be applied to the control by the camera 102 as well. For example, when an object detected to enter the detection area remains in the detection area for a predetermined time, the camera 102 determines that the object is an obstacle, and removes the object from the imaging range. can control itself.

<Second embodiment>
Next, a second embodiment will be described. In this embodiment, an object that enters the detection area, which is an area outside the imaging range cut out by the electronic PTZ and within the imaging range of the optical PTZ, is determined as an obstacle. This is an embodiment for notifying the camera operator of the presence of the intruding object later. A camera operator may be a user/operator of the client device 103 . Differences from the first embodiment will be described below, and descriptions of configurations/features common to the first embodiment will be omitted.

FIG. 8 shows a flowchart of processing executed by the client device 103 according to this embodiment. Note that the client device 103 can repeatedly perform the processing of this flowchart. In this embodiment, the detection area is outside the imaging range (captured image) cut out by the electronic PTZ from the imaging range (captured image) of the optical PTZ by the camera 102 and within the imaging range (captured image) of the optical PTZ. area. In other words, in a photographed image, an area from the end of the image to the inside of the image by a certain width is set as the detection area, and a partial image excluding the detection area is cut out from the image, and the partial image is delivered from the client device 103 to the external device. be done. Then, when an object determined as an obstacle is detected in the detection area, the client device 102 notifies the camera operator of the existence of the intruding object. Note that in the present embodiment, when a partial image is cut out by the electronic PTZ from the captured image of the optical PTZ imaging range in the camera 102, the captured image of the optical PTZ imaging range and the partial image are generated by the client device 103. sent to. Then, the client device 103 receives (acquires) the captured image and the partial image, and sets an area other than the partial image in the captured image as a detection area. Subsequently, when the client device 103 detects an object determined as an obstacle in the detection area of the captured image (the captured image within the imaging range of the optical PTZ), the client device 103 notifies the camera operator of the existence of the intruding object. Here, the processing of the client device 103 in this embodiment will be described with reference to the processing of the flow of FIG. The same reference numerals are given to the same processes as in the flowchart shown in FIG. 6, and the description thereof is omitted.

In FIG. 8, when it is determined that the object still exists in the detection area after a predetermined time has elapsed since the detection of the object's entry into the detection area (S601) (Yes in S604), the process proceeds to S801. In S<b>801 , the object detection unit 503 notifies the camera operator via the user input/output I/F 405 of the presence of an intruding object, that is, that the intruding object exists as an obstacle in the detection area. The notification can be made via a display (for example, displaying a warning screen) or a speaker (for example, issuing a warning sound) as the user input/output I/F 405 .
Furthermore, in S801, the object detection unit 503 simultaneously stops the operation of the timer unit 504 (releases the timer), and the process proceeds to S607.

As described above, according to the present embodiment, in a state where PTZ control is performed in advance, the client apparatus 103 detects the presence of an intruding object by the camera operator when an object that has entered the detection area remains for a predetermined period of time. to notify. This enables the camera operator to recognize the presence of an intruding object existing in the detection area before it enters the shooting range.

<Third Embodiment>
Next, a third embodiment will be described. In this embodiment, as in the first embodiment, two detection areas are provided for detecting an intruding object after PTZ control is performed when it is determined that the intruding object is an obstacle. detection area and inner detection area). When an object enters the outer detection area, the presence of the intruding object is notified to the camera operator, and when the object enters the inner detection area, the object is determined as an obstacle. Implement digital PTZ control.

The photographing range before and after PTZ control according to the present embodiment will be described with reference to FIGS. FIGS. 9A and 9B are diagrams for explaining the imaging range (angle of view) according to this embodiment.

In FIG. 9, a shooting range 901 indicates the shooting range (angle of view) captured by the camera 102 . A shooting range 901 can be a shooting range at the start of shooting. A first detection area (shaded area) 902, which is an outer detection area, and a second detection area (hatched area) 905, which is an inner detection area, are detection areas for detecting an intruding object. In this example, in response to an object 904 as an intruding object entering the first detection area 902, the camera 102 performs PTZ control (S606 in FIG. 6), and the shooting range of the camera 102 changes to the shooting range after the PTZ control. It is changed to range 703 .

FIG. 9( a ) shows that the object 904 remains in the first detection area 902 . Here, when the object detection unit 503 of the client device 103 determines that the object 904 remains in the first detection area 902 for a predetermined period of time, it performs processing according to the second embodiment (see FIG. 8). That is, when it is determined that the object 904 remains in the first detection area 902 for a predetermined time (Yes in S604 of FIG. 8), the client device 103 notifies the camera operator of the presence of the intruding object ( S801 in FIG. 8).

Also, FIG. 9B shows that an object 904 has entered the second detection area 905 . At this time, the client device 103 performs processing according to the first embodiment (see FIG. 6). That is, when a predetermined period of time has elapsed since the object 904 entered the second detection area 905 (Yes in S603 of FIG. 6), the object detection unit 503 of the client device 103 removes the object 904 from the shooting range. A PTZ command is transmitted to the camera 102 (S606). In response to this, the camera 102 performs PTZ control, and the shooting range is changed to the shooting range 906 after PTZ control.

As described above, according to the present embodiment, two detection areas, an outer detection area and an inner detection area, are provided. Then, in a state in which PTZ control is performed in advance, the client device 103 notifies the camera operator of the presence of the intruding object when an object that has entered the outer detection area remains for a predetermined time. Also, when an object enters the inner detection area and satisfies a predetermined condition, the object is determined as an object (obstacle) unrelated to photographing, and the camera 102 is controlled to remove the object from the photographing range. This allows the client device 103 to automatically detect/notify an obstacle and to automatically control the camera.

Further, the present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device executes the program. It can also be realized by a process of reading and executing. It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

101: network, 102: camera, 103: client device

Claims (13)

Acquisition means for acquiring an image captured by the imaging means;
setting means for setting a detection area, which is an area within a certain width from an outer frame of the image in the image;
detection means for detecting an object that has entered the detection area;
Determination means for determining whether the object is an obstacle unrelated to imaging by the imaging means;
a control means for controlling an angle of view of the imaging means so as to remove the object from an imaging range of the imaging means when the determination means determines that the object is the obstacle;
An image processing device comprising: The judging means starts timing when the object enters the detection area, and judges the object as the obstacle when detecting that the object stays in the detection area for a predetermined period of time. 2. An image processing apparatus according to claim 1. The determination means detects a direction in which the object enters the detection area, and determines the object as the obstacle when detecting that the object has entered from the outside to the inside of the detection area. 2. The image processing apparatus according to claim 1, characterized by: The determining means determines whether or not the object is a pre-registered object, and determines that the object is the obstacle if it is determined that the object is not the registered object. 2. The image processing apparatus according to claim 1. A sensor is attached to the registered object, and the determination means determines whether or not the object is the registered object based on whether or not a signal is received from the sensor. 5. The image processing apparatus according to claim 4, wherein 6. The image processing apparatus according to claim 1, wherein the control means controls the angle of view of the imaging means by digital PTZ (pan-tilt-zoom) or optical PTZ. 7. The image processing apparatus according to any one of claims 1 to 6, wherein the photographing range is a photographing range of the image pickup means after the angle of view is controlled by the control means. acquisition means for acquiring an image captured by an imaging means and a partial image cut out from the image;
setting means for setting an area other than the partial image cut out from the image as a detection area;
detection means for detecting an object that has entered the detection area in the image;
Determination means for determining whether the object is an obstacle unrelated to imaging by the imaging means;
notification means for notifying the presence of the object when the determination means determines that the object is the obstacle;
An image processing device comprising: 9. The image processing apparatus according to claim 1, further comprising said imaging means. An image processing system comprising a plurality of imaging devices each having the imaging means, and the image processing device according to any one of claims 1 to 8. an acquisition step of acquiring an image captured by the imaging means;
a setting step of setting a detection area, which is an area within a certain width from an outer frame of the image in the image;
a detection step of detecting an object that has entered the detection area;
a determination step of determining whether the object is an obstacle unrelated to imaging by the imaging means;
a control step of controlling the angle of view of the imaging means so as to exclude the object from the imaging range of the imaging means when the object is determined to be the obstacle in the determination step;
An image processing method characterized by comprising: an acquisition step of acquiring an image captured by an imaging means and a partial image cut out from the image;
a setting step of setting an area other than the partial image cut out from the image as a detection area;
a detection step of detecting an object that has entered the detection area in the image;
a determination step of determining whether the object is an obstacle unrelated to imaging by the imaging means;
a notification step of notifying the presence of the object when the object is determined to be the obstacle in the determination step;
An image processing method characterized by comprising: A program for causing a computer to function as each means of the image processing apparatus according to any one of claims 1 to 9.

Images