JP2014042160A - Display terminal, setting method of target area of moving body detection and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce operation load for permitting a user to precisely designate a target area of moving body detection when a moving object is detected by using a photographed image including image distortion.SOLUTION: A display terminal acquires photographed data showing a photographed image including image distortion due to a photographic lens of a fish-eye camera. The display terminal corrects the distortion so that the image distortion becomes less as magnification of the partial image is higher when partial images 110 and 120 obtained by enlarging a part of the photographed image are displayed ((a), (b)). When the partial image 120 in which the image distortion is made the minimum by distortion correction is displayed, the display terminal divides the partial image 120 into mesh shapes and displays a partial image 120a (c). The display terminal receives a selection operation of a target area of moving body detection from a mesh area M of the partial image 120a, and displays partial images 120b and 120c showing the selected target areas ((d), (e)). The display terminal detects the moving body of the target area, which is selected by above procedures in accordance with the photographed data.

Description

  The present invention relates to a technique for setting a target area for moving object detection.

  For example, Patent Documents 1 to 3 disclose a monitoring system including an omnidirectional camera using a fisheye lens or the like as a photographing lens. Patent Document 1 discloses that a moving object detection process is performed by comparing distorted circular images photographed using a fisheye lens with each other in chronological order, and the detected moving object is cut out and displayed. Patent Document 2 stores omnidirectional images obtained using a fisheye lens in a moving image memory in chronological order, and cuts out an image at a specified recording time and specified coordinates from the omnidirectional image recorded in the moving image memory. Thus, it is disclosed that the display is developed in plan view. Patent Document 3 discloses that an omnidirectional image obtained by an omnidirectional camera is divided concentrically and radially, each divided area is clearly indicated, and the zoom camera is zoom-driven in units of areas.

JP 2011-61511 A JP 2005-286820 A JP 2003-259350 A

In the inventions described in Patent Documents 1 to 3, the presence or absence of a moving object is monitored for the entire imaging range of the omnidirectional camera, that is, the entire image area of the omnidirectional image. In this type of monitoring system, there is a case where it is desired to monitor a part of the photographing range (for example, a door or a window) depending on a monitoring place or a monitoring purpose. However, even if the user visually recognizes a captured image including image distortion caused by the imaging lens such as an omnidirectional image and performs an operation of designating a monitoring target area, how the monitored place in the captured image looks And the way the actual monitoring location is viewed are different, and it is difficult for the user to accurately specify the desired monitoring target area.
Accordingly, an object of the present invention is to reduce an operation burden for a user to accurately specify a target area for moving object detection when moving object detection is performed using a captured image including image distortion.

  In order to solve the above-described problem, a display terminal of the present invention includes a display unit that displays an image on a display surface, and shooting data indicating a shot image including an image distortion caused by a shooting lens of the movie camera from the movie camera. An acquisition unit to be acquired, a correction unit that performs distortion correction that reduces the image distortion based on shooting data acquired by the acquisition unit, and an image that has been subjected to the distortion correction by the correction unit are displayed on the display surface A first control unit that receives a first operation for designating a target region for motion detection using the display control unit that is displayed, the image that is displayed on the display surface and that is subjected to the distortion correction, and the first reception unit. A setting unit configured to set the moving object detection so as to perform the moving object detection of the target area specified by the received operation.

In the display terminal of the present invention, the display terminal includes a second reception unit that receives a second operation that instructs display of a partial image obtained by enlarging a part of the captured image, and the display control unit receives the second operation. The partial image after the distortion correction is divided into a plurality of areas and displayed on the display surface, and the first receiving unit receives an operation of selecting the target area from the plurality of areas as the first operation. Also good.
In this display terminal, the correction unit performs the distortion correction so that the image distortion is reduced as the enlargement ratio of the partial image is higher, and the display control unit has the enlargement ratio or the correction degree of the distortion correction. The partial image may be divided into the plurality of regions when a threshold level is exceeded.

Moreover, the display terminal of this invention WHEREIN: The said setting part may be made to perform the said moving body detection according to the detection sensitivity set for every said target area | region.
In this display terminal, the setting unit may set the detection sensitivity of the target region according to the position of the target region in the captured image or the magnitude of the image distortion.

  In the display terminal of the present invention, the display control unit may display the captured image in which the target area is clearly specified after the first reception unit receives the first operation.

  In the display terminal according to the present invention, the display control unit may display the captured image that clearly shows the target area so that the target area is displayed at a predetermined position on the display surface. Also good.

  In the display terminal of the present invention, the first reception unit receives the first operation that specifies the target region and a time at which moving object detection of the target region is performed, and the setting unit includes the first operation. It may be set so that the moving object detection of the target area designated by the above is performed at the time designated by the first operation.

  The moving object detection target region setting method of the present invention includes an acquisition step of acquiring shooting data indicating a shot image including image distortion caused by a shooting lens of the moving image camera from the moving image camera, and the shooting acquired in the acquisition step. Based on the data, a correction step for performing distortion correction to reduce the image distortion, a display control step for displaying the image subjected to the distortion correction in the correction step on a display surface for displaying an image, and a display surface for displaying Using the displayed image subjected to distortion correction, an accepting step for accepting an operation for designating a target area for motion detection, and an operation accepted in the accepting step based on the shooting data obtained in the obtaining step A setting step for setting the moving object detection so as to perform the moving object detection of the target area.

  The program of the present invention obtains shooting data indicating a shot image including image distortion caused by a shooting lens of the moving image camera from a moving image camera to a computer of a display terminal that displays an image on a display surface; Based on the shooting data acquired in the acquisition step, a correction step for performing distortion correction for reducing the image distortion, a display control step for displaying the image subjected to the distortion correction in the correction step on the display surface, An accepting step for accepting an operation for designating a target area for motion detection using the image subjected to the distortion correction displayed on the display surface, and accepting the accepting step based on the imaging data obtained in the obtaining step. A setting step setting step for setting the moving object detection so that the moving object is detected in the target area specified by the operation. Is a program for executing and.

  ADVANTAGE OF THE INVENTION According to this invention, when performing a moving body detection using the picked-up image containing an image distortion, the operation burden for a user to specify the object area | region of a moving body detection exactly can be reduced.

The figure which shows the structure of the external appearance of a display terminal and a fisheye camera. The block diagram which shows the hardware constitutions of a fisheye camera. The figure which illustrates the installation aspect of a fisheye camera. The figure which illustrates the picked-up image of a fisheye camera. The block diagram which shows the hardware constitutions of a display terminal. The figure which shows the data structure of a reference table. The functional block diagram which shows the function structure of the control part of a display terminal. The flowchart which shows the operation | movement procedure when a display terminal sets an object area | region. The figure explaining the designation | designated procedure of an object area | region (range V11). The figure explaining the designation | designated procedure of an object area | region (range V12). The figure which illustrates the picked-up image which specified the object area | region. The figure which shows the data structure of a reference table (modification 1). The figure which illustrates the picked-up image which specified the object area (modification 1). The figure explaining the detection sensitivity for every object field (modification 2). The figure which illustrates the picked-up image which specified the object area (modification 4).

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram illustrating the external configuration of the display terminal 10 and the fisheye camera 20.
As shown in FIG. 1, the display terminal 10 includes a display surface 141 for displaying an image, a touch sensor for detecting a contact operation performed by a user touching the display surface 141 with an indicator such as a finger or a stylus pen, and the like. It is a display terminal provided with. The display terminal 10 is a smartphone here, but may be another display terminal such as a mobile phone terminal, a tablet computer, a personal computer, or a PDA (Personal Digital Assistant). The fisheye camera 20 is a photographing device that photographs a subject using a fisheye lens 241. The fisheye camera 20 is called an omnidirectional camera or the like whose viewing angle of the shooting range is approximately 360 degrees, and is a moving image camera that takes a moving image with the direction of the fisheye lens 241 as the shooting direction. In addition, the fisheye camera 20 performs moving object detection that detects the presence or absence of a moving object based on a captured image captured using the fisheye lens 241.

A wireless communication path is established between the display terminal 10 and the fisheye camera 20, and the display terminal 10 and the fisheye camera 20 can transmit / receive data to / from each other using this communication path. In addition to this, it is also possible to establish a wired communication path between the display terminal 10 and the fisheye camera 20 using a communication cable or the like, and to transmit / receive data to / from each other.
Note that the display terminal 10 and the fisheye camera 20 are each of a size and weight that can be easily carried by the user. Further, the display terminal 10 and the fisheye camera 20 may exchange data with each other via a predetermined server.

FIG. 2 is a block diagram illustrating a hardware configuration of the fisheye camera 20. As shown in FIG. 2, the fisheye camera 20 includes a control unit 21, an operation unit 22, a communication unit 23, a photographing unit 24, and a storage unit 25.
The control unit 21 includes a microcomputer having a central processing unit (CPU), a read only memory (ROM), and a random access memory (RAM). The CPU controls each unit of the fisheye camera 20 by reading the program stored in the ROM or the storage unit 25 into the RAM and executing it. The operation unit 22 has operation buttons operated by the user, and outputs an operation signal indicating an operation performed by the user to the control unit 21. The communication unit 23 is an interface for communicating with the display terminal 10 using a wireless communication path or a wired communication path. For example, the communication unit 23 transmits the shooting data generated by the shooting unit 24 to the display terminal 10.

The imaging unit 24 includes a fisheye lens 241, an imaging element, and a signal processing circuit, generates imaging data indicating a captured image, and outputs the imaging data to the control unit 21. Specifically, the photographing unit 24 forms an image at the position of the image sensor using the fisheye lens 241. The imaging element has, for example, a CCD (Charge Coupled Device), receives the imaged light, and generates an image signal corresponding to the light. The image sensor is an image sensor that images, for example, three colors of R (red), G (green), and B (blue), and an analog image that represents a subject image with the three colors of R, G, and B. Generate a signal. The signal processing circuit performs signal processing such as A (analog) / D (digital) conversion on the image signal generated by the image sensor to generate shooting data.
In the present embodiment, the color components included in the shooting data generated by the shooting unit 24, the number of gradations of each color component, and the like may be arbitrary.

  The storage unit 25 includes an internal storage device exemplified by an EEPROM (Electronically Erasable and Programmable ROM), and stores a program executed by the control unit 21, shooting data indicating a shot image shot by the shooting unit 24, and the like. The storage unit 25 stores a program for performing moving body detection based on the shooting data generated by the shooting unit 24 as one of the programs executed by the control unit 21. The storage unit 25 may include an external storage device such as an SD card that can be attached to and detached from the fisheye camera 20, and data such as shooting data can be stored in the external storage device.

FIG. 3 is a diagram for explaining an installation mode when the fisheye camera 20 is installed in a room R of a house. FIG. 3A is a diagram illustrating a state in which the room R is viewed from above. As shown in FIG. 3A, a table T is installed near the center of the room R, and sofas SF1 to SF3 are installed so as to surround the table T from different directions. A plant P is placed near the sofas SF1 and SF2 in the room R. The walls of the room R are provided with windows W1 to W3 and a door D. As shown in FIG. 3B, the fisheye camera 20 is installed on the ceiling C of the room R with the fisheye lens 241 facing vertically downward. For example, the fisheye camera 20 can shoot the entire room space including the windows W1 to W3 and the door D of the room R within the shooting range. The fisheye camera 20 may be fixed in its posture and installation position by a fixture provided in the fisheye camera 20, or may be fixed by a separately prepared fixture.
When the fisheye camera 20 captures an image using the fisheye lens 241, the imaging range can be widened as compared with a case where a standard lens or the like is used as the imaging lens. On the other hand, in the captured image of the fisheye camera 20, image distortion caused by using the fisheye lens 241 is included in the imaging range. As a result, the appearance of the room R is greatly different between when the user looks directly at the room R and when the user looks at the room R in the captured image captured by the fisheye camera 20.

FIG. 4 is a diagram illustrating a state in which the captured image 100 of the fisheye camera 20 is displayed on the display surface 141 of the display terminal 10. The symbols shown in FIG. 4 and the ellipses indicating the ranges V11 and V12 are not displayed on the display surface 141 as images. The same applies to other drawings.
As can be seen from FIG. 4, the fish-eye camera 20 shoots in an elliptical shooting range and generates shooting data indicating an elliptical shot image 100. As shown in FIG. 4, the photographed image 100 shows an elliptical photographed image representing a state where the room R is viewed from above. In the photographed image 100, the image distortion caused by the fisheye lens 241 is relatively small at the center, but the image distortion caused by the fisheye lens 241 tends to increase as it approaches the outer edge. For example, the subject located closer to the center in the captured image 100 has less image distortion, and the ratio of the dimension of the subject in the major axis direction and the minor axis direction of the captured image 100 is close to the actual ratio of the subject. On the other hand, a subject located closer to the outer edge in the photographed image 100 has a higher degree of image distortion. For example, the photographed image 100 is photographed in a large size in the major axis direction due to image distortion, and is photographed in a small size in the minor axis direction due to image distortion. Therefore, in such a subject, the ratio of the dimension of the major axis direction and the minor axis direction of the captured image 100 is greatly different from the actual ratio.

FIG. 5 is a block diagram illustrating a hardware configuration of the display terminal 10. As shown in FIG. 5, the display terminal 10 includes a control unit 11, an operation unit 12, a communication unit 13, a display unit 14, and a storage unit 15.
The control unit 11 includes a microcomputer having a CPU, a ROM, and a RAM. The CPU controls each unit of the display terminal 10 by reading the program stored in the ROM or the storage unit 15 into the RAM and executing it. The operation unit 12 includes a touch sensor and a hard button provided to overlap the display surface 141 of the display unit 14, and outputs an operation signal indicating an operation performed by the user of the display terminal 10 to the control unit 11. The communication unit 13 is an interface for communicating with the fisheye camera 20 using a wireless communication path or a wired communication path. The communication part 13 receives the imaging | photography data transmitted by the fisheye camera 20, for example. The display unit 14 has a display surface 141 in which a plurality of pixels are arranged, and displays an image on the display surface 141. The storage unit 15 includes a storage device such as an EEPROM (Electronically Erasable and Programmable ROM) and a flash memory, and stores a program executed by the control unit 11. In addition to this, the storage unit 15 stores shooting data 151 and a reference table 152.

  The shooting data 151 corresponds to the recording data of the shot image of the fisheye camera 20 and is data indicating a shot image (moving image) shot by the fisheye camera 20. The shooting data 151 indicates a shot image including image distortion caused by the fisheye lens 241 (see FIG. 4). The reference table 152 is a data table in which setting details of moving object detection that the display terminal 10 causes the fisheye camera 20 to perform are written.

  FIG. 6 is a diagram illustrating the data structure of the reference table 152. As illustrated in FIG. 6, the reference table 152 is a data table that associates each information of “target region” and “detection sensitivity”. “Target area” is data indicating an area designated as a target area for moving object detection in the captured image 100. The data of the target area includes, for example, information indicating the position of the target area in the captured image 100 (information indicating a mesh area described later). The detection sensitivity is data indicating the sensitivity of moving object detection. In the moving object detection performed by the display terminal 10, the higher the detection sensitivity, the more the moving object (for example, the intruder) is detected even when there is a slight movement of the subject (that is, the movement of the image in the captured image). The

In the example illustrated in FIG. 6, in the reference table 152, “ST1” is associated with the target region and “high” is associated with the detection sensitivity. In the reference table 152, “ST2” is associated with the target area and “low” is associated with the detection sensitivity. The operation of the display terminal 10 relating to the writing of data to the reference table 152 will be described later, but before starting the motion detection, each field of the reference table 152 is blank.
In this embodiment, the detection sensitivity of the moving object detection performed by the display terminal 10 has two levels, “high” and “low”. However, the detection sensitivity may be three or more levels, or may be constant. Good. When the display terminal 10 and the fisheye camera 20 exchange data via a server, the shooting data 151 and the reference table 152 may be stored in this server. In this case, the display terminal 10 may acquire the shooting data 151 from this server or refer to the reference table 152 stored in this server.

FIG. 7 is a functional block diagram illustrating a functional configuration of the control unit 11 of the display terminal 10. As illustrated in FIG. 7, the control unit 11 of the display terminal 10 includes an acquisition unit 111, a correction unit 112, a display control unit 113, a first reception unit 114, a second reception unit 115, and a setting unit 116. A function equivalent to is realized.
The acquisition unit 111 communicates with the fisheye camera 20 through the communication unit 13 and acquires imaging data indicating a captured image of the fisheye camera 20. The acquisition unit 111 acquires, for example, shooting data indicating a shot image that is a live moving image and supplies the shooting data to the correction unit 112 or stores shooting data 151 as recording data in the storage unit 15.

  The correction unit 112 performs distortion correction that reduces image distortion based on the shooting data acquired by the acquisition unit 111. Here, the correction unit 112 performs distortion correction based on the shooting data in real time, and supplies the display control unit 113 with the shooting data after the distortion correction.

  The display control unit 113 controls display of images on the display unit 14. For example, the display control unit 113 displays the captured image acquired by the acquisition unit 111 or the captured image indicated by the captured data 151 stored in the storage unit 15 or the image subjected to distortion correction by the correction unit 112 on the display surface 141. Let Further, the display control unit 113 displays an image on the display surface 141 in accordance with the operation received by the first reception unit 114 or the second reception unit 115.

  The second reception unit 115 receives an operation (second operation) for instructing display of a partial image obtained by enlarging a part of the captured image. The second receiving unit 115 receives an operation signal (for example, a contact operation) performed by the user on the operation unit 12 by acquiring an operation signal from the operation unit 12. When the second reception unit 115 receives an operation for instructing display of a partial image, the display control unit 113 displays the partial image after distortion correction on the display surface 141. At this time, the correction unit 112 performs distortion correction so that the image distortion decreases as the enlargement ratio of the partial image increases, and supplies the image data after the distortion correction to the display control unit 113. When the display control unit 113 displays a partial image in which the enlargement ratio or the correction degree of distortion correction exceeds the threshold level, the display control unit 113 divides the partial image into a mesh shape and displays the partial image on the display surface 141.

  The first accepting unit 114 accepts an operation (first operation) for designating a target area for moving object detection using the image after distortion correction displayed on the display surface 141. For example, the first reception unit 114 selects an area for moving object detection from a partial image area (hereinafter referred to as a “mesh area”) divided into meshes, and detects the detection sensitivity for the target area. Each specified operation is accepted. The first reception unit 114 receives an operation signal from the operation unit 12 to receive an operation (for example, a contact operation) performed on the operation unit 12 by the user. When the target region is specified by the operation received by the first receiving unit 114, the display control unit 113 causes the display surface 141 to display a captured image that clearly indicates the specified target region and detection sensitivity.

  The setting unit 116 sets the moving object detection performed by the fisheye camera 20 by updating the reference table 152 according to the target region and the detection sensitivity specified by the operation received by the first reception unit 114. In addition, the setting unit 116 instructs the fisheye camera 20 to perform the moving object detection based on the target region and the detection sensitivity associated with each other in the reference table 152. The fisheye camera 20 performs moving object detection according to an instruction from the setting unit 116. When the fish-eye camera 20 detects the presence of a moving object from the target area in the captured image, the fish-eye camera 20 executes a specific process according to the detection result of the moving object.

FIG. 8 is a flowchart illustrating a procedure when the display terminal 10 sets the moving object detection. It is assumed that the reference table 152 is blank before the operation shown in FIG.
A user who wants to detect a moving object using the fisheye camera 20 installs the fisheye camera 20 in advance so that the shooting range includes a space where the moving object is to be detected. Here, as shown in FIG. 3B, it is assumed that the user has installed the fisheye camera 20 on the ceiling C of the room R with the shooting direction facing downward so that the entire room R is within the shooting range.

When the control unit 11 of the display terminal 10 is instructed to set the moving object detection by operating the operation unit 12, first, a captured image (shown by the captured data acquired by the communication unit 13 from the fisheye camera 20) is displayed. Here, an entire image showing the entire photographing range is displayed on the display surface 141 (step S1). Here, the control unit 11 displays the captured image 100 shown in FIG. 4 on the display surface 141.
The control unit 11 may display a moving image based on the shooting data, or may generate a still image data by performing frame processing based on the shooting data and display a shooting image that is a still image. Good.

  Next, the control unit 11 determines whether or not a display change operation for the captured image 100 has been received by the operation of the operation unit 12 (step S2). Here, the control unit 11 determines whether or not an operation for changing an image area to be displayed on the display surface 141 in the captured image 100 is received by a touch operation of paying a finger on the display surface 141, or so-called pinch-in. It is determined whether or not an operation for changing the display magnification of the photographed image 100 is accepted by an operation or a pinch-out operation.

  The control unit 11 stands by until a display change operation is accepted (step S2; NO). When the display change operation is accepted (step S2; YES), a captured image is displayed on the display surface 141 according to the received display change operation. Let When the control unit 11 receives a display change operation for displaying a partial image obtained by enlarging a part of the captured image, the control unit 11 performs distortion correction based on the captured data to reduce image distortion as the enlargement ratio increases. A partial image is displayed on the surface 141 (step S3). The control unit 11 performs image processing for reducing image distortion of the captured image in accordance with a known conversion algorithm for reducing image distortion. When the display change operation for enlarging and displaying the range V11 near the door D illustrated in FIG. 4 is performed by the user, the control unit 11 displays the partial image 110 illustrated in FIG. 9A on the display surface 141. The partial image 110 is a partial image with less image distortion than the captured image 100.

Next, the control unit 11 determines whether or not the partial image on the display surface 141 is displayed at the highest magnification (step S4). Here, the control unit 11 determines “NO” in the process of step S4, and returns to the process of step S2.
Next, in step S2, when the control unit 11 receives a display change operation for further enlarging and displaying the partial image 110 (step S2; YES), the partial image 120 shown in FIG. 9B is displayed on the display surface 141. It is displayed (step S3). The partial image 120 is an image on which distortion correction for further reducing the image distortion included in the partial image 110 is performed. The partial image 120 is a partial image with the highest magnification.
As can be seen from the above processing steps S2 to S4, the control unit 11 enlarges the partial image and gradually converts the partial image into a planar image when a display change operation for gradually enlarging the captured image is performed. Distortion correction is performed so as to be close.

Then, the control unit 11 determines that the partial image on the display surface 141 is displayed at the highest magnification in the process of step S4 (step S4; YES), and proceeds to the process of step S5. Next, the control unit 11 divides the partial image being displayed on the display surface 141 into a plurality of areas in a mesh shape and displays the partial image on the display surface 141 (step S5). Here, as shown in FIG. 9C, the control unit 11 divides the partial image 120 into 10 × 10 (total 100) square mesh regions M and displays the partial image 120a on the display surface 141. Let The mesh area M is an area that the user can select by an operation (for example, a contact operation) on the operation unit 12 in order for the user to specify a target area for moving object detection. That is, the user can designate a target area for moving object detection in units of mesh area M. In FIG. 9 and other drawings, the boundary between the mesh areas M is indicated by a solid line. However, the control unit 11 causes the display surface 141 to display a line indicating the boundary, so that the user can change the position in the mesh area M. The range can be grasped.
Note that the number of mesh regions into which the partial image is divided is merely an example, and the control unit 11 may set the number of mesh regions to a number other than 10 × 10.

Next, the control unit 11 determines whether a selection operation for selecting a target region from the mesh region M has been received by the operation unit 12 (step S6). Here, as illustrated in FIG. 9D, when the control unit 11 receives a selection operation of any mesh region M by touching the mesh region M with a finger F, the control unit 11 sets the mesh region M as the target region ST1. To do. And the control part 11 displays the partial image 120b which specified this object area | region ST1 on the display surface 141 (step S7). Here, the control unit 11 applies a mask of a predetermined color (for example, light blue) to the original region ST1 with respect to the target region ST1, so that the user can distinguish from the other mesh region M. Is specified. And the control part 11 sets a moving body detection by updating the reference table 152 according to object area | region ST1 designated by the user (step S8). In this case, it is assumed that the user has designated the detection sensitivity in advance and has selected the target region from the mesh region M, and here, “high” has been designated as the detection sensitivity. In this case, the control unit 11 writes “ST1” as the target region and “High” as the detection sensitivity in association with each other and writes them in the reference table 152.
Note that, in FIG. 9 and other drawings, the target area having a detection sensitivity of “high” is represented by being filled with a lattice pattern.

  Next, the control unit 11 determines whether or not to finish specifying the target area (step S9). Here, the control unit 11 determines whether or not the user has performed an operation for confirming that moving object detection is to be performed with the contents set in the current reference table 152. When the control unit 11 determines that the designation of the target area has not ended (step S9; NO), the control unit 11 determines whether the display change operation described in step S2 has been received by the operation unit 12 (step S10). ). Here, if the control part 11 judges that display change operation is not received (step S10; NO), it will return to the process of step S6. Then, the control unit 11 determines whether a selection operation for selecting the mesh region M as the target region has been received by the operation unit 12.

  Thereafter, the control unit 11 executes the processing steps of steps S6 to S10, accepts a selection operation of a target area for moving object detection (step S6), specifies the target area according to the accepted operation, The reference table 152 is updated to set the moving object detection (steps S7 and S8). Here, when the user designates the door D and the vicinity of the door D as the target area, the control unit 11 causes the display surface 141 to display a partial image 120c in which the target area ST1 is clearly shown as illustrated in FIG. .

  Next, when the control unit 11 determines that the target area has not been designated (step S9; NO) and determines that the display change operation has been received by the operation unit 12 (step S10; YES), the process of step S3 is performed. Return to. Then, similarly to the processing in step S3, when the display unit 11 receives a display change operation for displaying a partial image, the control unit 11 performs distortion correction to reduce image distortion as the enlargement ratio increases, and the display surface 141. Display a partial image (steps S2 and S3). Here, it is assumed that the user is instructed to display the range V12 near the window W3 shown in FIG. In this case, the control unit 11 displays the partial image 130 illustrated in FIG. The partial image 130 is a partial image with the highest magnification, and is a partial image with the least image distortion.

  In this case, the control unit 11 determines “YES” in the process of step S4, and executes the process steps of steps S5 to S10 as described above. Here, it is assumed that the control unit 11 receives an operation for designating a part of the window W <b> 3 as a target region using the partial image 130 by the operation unit 12. In this case, the control unit 11 displays the partial image 130a illustrated in FIG. 10B, clearly indicates the target region ST2, and updates the reference table 152. Here, the entire window W3 does not fit in the partial image 130 (130a). In this case, the user performs a display change operation for changing the display area on the display surface 141 in the captured image 100 by operating the operation unit 12. The control unit 11 receives a selection operation for selecting a target region from the mesh region M by the operation unit 12 while changing the display region according to the display change operation. Here, the control unit 11 displays the partial image 130b illustrated in FIG. 10C or the partial image 130c illustrated in FIG. 10D according to the display change operation of the user and the selection operation of the target area ST2. 141 is displayed.

It is assumed that the detection sensitivity of the target area ST2 is designated as “low” by the user. In FIG. 10 and other drawings, the target area with low detection sensitivity is represented by being filled with a diagonal line rising to the right.
In addition, even if there is no abnormality in the window W3, the user is likely to change the image depending on the outdoor brightness. Therefore, it is assumed that the detection sensitivity is designated as “low” in order to suppress erroneous detection. On the other hand, since there is little change in the image due to the brightness of the door D, it is assumed that the detection sensitivity is designated as “high”.
As described above, when the display terminal 10 sets the moving object detection, the reference table 152 has the contents shown in FIG. In the present embodiment, the display terminal 10 manages data using the reference table 152 as a single target area that is configured by continuous mesh areas M and designated with the same detection sensitivity. However, the display terminal 10 may manage data as a different target area for each mesh area M, and various modifications can be made to the data management method in the reference table 152.

  If the control unit 11 determines in the process of step S9 that the designation of the target area has been completed (step S9; YES), the control unit 11 proceeds to the process of step S11. And the control part 11 displays the picked-up image 100a which specified the object area | region designated by the user on the display surface 141 based on picked-up data. Here, as shown in FIG. 11, the control unit 11 displays on the display surface 141 a photographed image 100a in which the target areas ST1 and ST2 are clearly shown by applying a predetermined mask to the photographed image displayed in step S1. Display. The user who has visually recognized the captured image 100a can immediately know which position in the imaging range the positions of the target areas ST1 and ST2 designated by the user are from the captured image having an overview of the entire imaging range.

  When the operation unit 12 instructs the control unit 11 to start moving object detection, the control unit 11 instructs the fisheye camera 20 to perform moving object detection according to the data written in the reference table 152. Then, the fish-eye camera 20 starts moving object detection (step S12). In response to the instruction to start moving object detection, the control unit 21 of the fisheye camera 20 performs moving object detection according to the setting by the display terminal 10. Here, based on the imaging data acquired from the imaging unit 24, the control unit 21 detects a moving object with “high” as the detection sensitivity for “ST1” as the target region, and detects “ST2” as the target region. Motion detection is performed with the sensitivity set to “low”. The control unit 21 performs moving object detection according to a predetermined moving object detection algorithm such as an inter-frame difference method or a background difference method. Specifically, for example, the control unit 21 performs moving object detection by comparing captured images obtained by performing frame processing on captured image data in chronological order. At that time, according to the control unit 21 and the detection sensitivity, for example, as the detection sensitivity is higher, the number of images obtained by frame processing is increased, or the threshold of image change (for example, luminance change) determined to be a moving object is reduced. Then, it is good to perform the moving body detection according to the detection sensitivity. And the control part 21 notifies the detection result of a moving body detection to the display terminal 10 by the communication part 23. FIG. When the communication unit 13 receives the detection result of the moving object detection from the fisheye camera 20, the control unit 11 of the display terminal 10 notifies the user of the detection result by display or sound on the display surface 141 or data indicating the detection result. Or the like in the storage unit 15.

As described above, when the display terminal 10 causes the user to perform an operation of designating a target area from a captured image of the fisheye camera 20, the display terminal 10 displays a partial image subjected to distortion correction that reduces image distortion. To display. Thus, for the user, the difference between how the monitoring location (in this case, room R) appears in the captured image and how the actual monitoring location looks is reduced, so that the desired target area is specified accurately. This reduces the operational burden.
In addition, when the display terminal 10 displays a partial image, the higher the magnification ratio is, the distortion correction is performed so that the partial image is closer to a planar image, and when the partial image is displayed at the highest magnification ratio, The partial image is divided into a plurality of mesh regions M. As a result, the user can specify the target area in detail by looking at the partial image enlarged to the maximum magnification. In addition, since the user can designate the target area on the display terminal 10 by selecting the mesh area M, the operation to be performed by the user is simple. Therefore, according to the display terminal 10, the operation burden for designating the desired target region accurately is further reduced.
Moreover, according to the display terminal 10, since the user can designate detection sensitivity for every object area | region (mesh area | region M), the operation burden of the user for designating the detection sensitivity suitable for a monitoring place or the monitoring objective is also carried out. Few.

[Modification]
The present invention can be implemented in a form different from the above-described embodiment. The present invention can also be implemented in the following forms, for example. Further, the following modifications may be combined as appropriate.
(Modification 1)
The display terminal 10 may receive an operation for designating a time zone for performing motion detection for the target region together with the target region, set the designated time zone, and cause the fisheye camera 20 to perform motion detection. In this case, the display terminal 10 refers to the reference table 152a instead of the reference table 152 described in the above-described embodiment, and causes the fisheye camera 20 to perform moving object detection.

  FIG. 12 shows the data structure of the reference table 152a. As shown in FIG. 12, the reference table 152a is a data table in which pieces of information of “target region”, “detection sensitivity”, and “detection time zone” are associated with each other. The “target region” and “detection sensitivity” are the same information as in the above-described embodiment, and the description thereof is omitted here. The “detection time zone” is information indicating the time at which moving object detection is performed, and is specified by the user. In the example illustrated in FIG. 12, for the target region ST1 located near the door D, “9:00:00 to 20:00”, which is a time zone in which the user is absent from the room R, is designated as the detection time zone. On the other hand, for the target area ST1 located in the window W3, the detection time zone is not specified and is blank in the reference table 152a.

The operation of the display terminal 10 according to this modification may basically operate according to the same procedure as in FIG. 8, and the difference from the embodiment will be described here.
In the process of step S6, the display terminal 10 also accepts an operation for designating the time (detection time zone) for conducting the conductor detection in the target area, together with the target area and the detection sensitivity.
In the process of step S <b> 11, when displaying the captured image specifying the target area, the display terminal 10 displays the captured image specifying the target area to be detected at each time. For example, when the user instructs to specify the target area by specifying the time of 9:00 to 20:00, the display terminal 10 displays the target areas ST1 and ST2 as shown in FIG. The specified captured image 100a is displayed on the display surface 141. On the other hand, when the user designates a time other than 9:00 to 20:00 to indicate the target area, the display terminal 10 displays the target area ST2 as shown in FIG. The specified captured image 100b is displayed on the display surface 141.

  In the process of step S12, when the detection time zone is specified based on the reference table 152a, the display terminal 10 performs the fish-eye camera so as to detect the moving object only during the period in which the current time is included in the detection time zone. 20 Here, the display terminal 10 instructs to perform moving body detection for the target areas ST1 and ST2 at a time between 9:00 and 20:00. On the other hand, at a time other than 9:00 to 20:00, an instruction is given not to perform moving object detection for the target area ST1 but to perform moving object detection for the target area ST2.

  According to the display terminal 10 of this modified example, not only the user can easily specify the target area for moving object detection, but also the user can specify the time for performing the moving object detection for each target area. Therefore, even when there are a plurality of target areas with different detection time zones at the same monitoring location, the display terminal 10 can set the detection time zone for each target area. For example, in offices and conference rooms, it may be possible not to detect moving objects during working hours, but to detect moving objects outside working hours. If so, the display terminal 10 can be set to perform moving object detection at an appropriate time for each target area.

(Modification 2)
In the embodiment described above, the display terminal 10 allows the user to specify the detection sensitivity. However, the display terminal 10 may automatically set the detection sensitivity according to the position of the target region in the captured image. As described above, in the photographed image of the fisheye camera 20, the subject located closer to the center has less image distortion, and the photographed image 100 is photographed with a relatively large size in each of the major axis direction and the minor axis direction. On the other hand, a subject located closer to the outer edge of the captured image 100 is photographed with a large size due to distortion in the major axis direction of the photographing range, while being photographed with a small size due to distortion in the minor axis direction.

  Therefore, as shown in FIG. 14A, the display terminal 10 sets the detection sensitivity to “high” in an elliptical range A1 located near the center where the degree of occurrence of image distortion is small. In the ranges A2 and A3 located closer to the outer edge, the subject is photographed with a large size due to image distortion in the major axis direction, so that even a slight image movement can be easily detected. Set to Low. In the ranges A4 and A5 located closer to the outer edge, the size of the subject is reduced in the major axis direction, but the subject size is secured to some extent in the minor axis direction. Is set to “medium” which is between “high” and “low”. In addition to this, as shown in FIG. 14B, the display terminal 10 sets the detection sensitivity of the elliptical range A6 located near the center where the degree of occurrence of image distortion is small to “high”. The detection sensitivity of the doughnut-shaped range A7 may be set to “low”.

  As described above, the display terminal 10 lowers the detection sensitivity in the original captured image indicated by the captured data acquired from the fisheye camera 20 in a region where the size of the subject increases due to image distortion. On the contrary, the display terminal 10 increases the detection sensitivity in a region where the degree of enlargement of the subject size due to image distortion is small in the original captured image. As an example, in the captured image indicated by the captured data acquired from the fisheye camera 20, the display terminal 10 has a size in the original captured image (for example, an area or a length in a predetermined direction) rather than a size in the planar image. The detection sensitivity is set to be lower in the larger region, and the detection sensitivity is set to be higher in the region where the size in the planar image is larger than the size in the original captured image.

Further, the display terminal 10 may automatically set the detection sensitivity according to the degree of occurrence of image distortion in the original captured image when the user specifies the detection sensitivity in advance. For example, when the detection sensitivity can be designated from a plurality of stages, the display terminal 10 corrects the detection sensitivity designated by the user to be lowered in a region where the degree of size expansion of the subject due to image distortion is large. Further, for example, the display terminal 10 corrects the detection sensitivity specified by the user in a direction to increase in a region where the degree of size expansion of the subject due to the occurrence of image distortion is small.
According to the display terminal 10 of this modification, it is possible to set an appropriate detection sensitivity according to the degree of occurrence of image distortion in a captured image without depending on how the detection sensitivity is designated by the user. Further, the above-described automatic setting method of the detection sensitivity in the display terminal 10 is merely an example, and may be appropriately set according to the type of photographing lens and the degree of occurrence of image distortion included in the photographed image.

(Modification 3)
In the above-described embodiment, the fisheye camera 20 has photographed an elliptical photographing range using the fisheye lens 241. However, the photographing range may be other shapes such as a circle. Thereby, the captured image of the fisheye camera 20 may also have another shape such as a circle instead of an ellipse.

(Modification 4)
In the process of step S10, the display terminal 10 may display a captured image that clearly indicates the target area so that the target area is displayed at a predetermined position on the display surface 141. This is because when the user wants to confirm the position of the target area, if the target area is displayed at a specific position on the display surface 141, the user can easily grasp the position of the target area.
For example, as shown in FIG. 15A, the target area ST is specified in the upper area of the photographed image 100a, whereas the display terminal 10 displays the target area in the lower area of the photographed image. Suppose that it was specified. In this case, the display terminal 10 may display the captured image 100c obtained by rotating the captured image 100a by 180 degrees on the display surface 141 and display the target region ST in the lower region of the captured image. Even when the target area is displayed at another position, the display terminal 10 may display the target area at a specific position on the display surface 141 by using a rotation process of the captured image. Further, when the fisheye camera having a circular shooting range is used as described in the third modification example, when the shot image 200 is circular as shown in FIG. The photographed image 200b may be displayed by rotating the photographed image 200 by a predetermined angle so that the photographed image 200 is displayed at a specific position (here, the lower region of the photographed image).

(Modification 5)
In the embodiment described above, the display terminal 10 displays the partial image with the least image distortion when displaying the partial image at the highest magnification. On the other hand, if the display terminal 10 can reduce the image distortion to below a certain level even if it is less than the maximum enlargement ratio, the partial image displayed when the display terminal 10 is less than the maximum enlargement ratio is displayed in the mesh region M. The target area may be specified by dividing. Further, the display terminal 10 does not divide the captured image having a larger image distortion into a plurality of mesh areas M, instead of dividing the image into a mesh area M at the time of displaying the partial image with the least image distortion. May be specified. That is, when the display terminal 10 displays a partial image with less image distortion than the original captured image, the display terminal 10 meshes the partial image on the condition that the enlargement ratio or the correction degree of distortion correction exceeds the threshold level. What is necessary is just to divide and display on the area | region M. According to the display terminal 10, it is possible to reduce an operation burden related to the designation of the target area of the user as compared with the case where the user designates the target area using the original captured image as it is.

(Modification 6)
The display terminal 10 may attach a specific character string such as “target area” to the position of the target area instead of clearly indicating the target area with a mask of a predetermined color, or may display a predetermined image such as a mark. May be added to the position of the target area. As long as the user can grasp the target area, the method of specifying the target area performed by the display terminal 10 is not particularly limited.

(Modification 7)
In the embodiment described above, the display terminal 10 divides the partial image so that the mesh region M is square, but the method of dividing the partial image is not limited to this. For example, the display terminal 10 may divide the mesh region into other shapes such as a rectangle, a circle, and other polygons.
Further, the display terminal 10 may not have a configuration for dividing the partial image into the mesh region M. For example, the display terminal 10 may perform image analysis on image data that has been subjected to distortion correction so as to reduce image distortion, extract objects such as windows and doors, and allow a user to specify a target area in units of extracted objects. . With this configuration, it is considered that the display terminal 10 can reduce the operation burden on the user regarding the designation of the target area without displaying the partial image.

(Modification 8)
Further, the operation received by the display terminal 10 may not be a contact operation that can be detected by the touch sensor. For example, the display terminal 10 may accept a proximity operation that can be detected by a proximity operation, or may accept a cursor operation.

(Modification 9)
In the above-described embodiment, the fisheye camera 20 is used as the photographing device. However, the display terminal 10 may use photographing data of a photographing device (moving image camera) in which image distortion caused by the photographing lens occurs. The type of lens is not particularly limited. In addition, the projection method of the fisheye lens 241 included in the fisheye camera 20 may be any projection method such as an equidistant projection method or an equal solid angle projection method.
Instead of the moving eye detection of the fisheye camera 20 in real time, the moving body detection may be performed based on the recorded data. Further, the display terminal 10 may perform moving object detection using the imaging data 151 according to the reference table 152 or 152a.
Moreover, the place where the fisheye camera 20 is installed is not limited to a room in a house, but may be a room space of a commercial facility such as an office, a conference room, or a store, or the fisheye camera 20 is installed outdoors. May be. In this way, the fisheye camera 20 may take any place as a shooting range.

(Modification 10)
In the embodiment described above, each function realized by the control unit 11 of the display terminal 10 can be realized by a combination of a plurality of programs, or can be realized by cooperation of a plurality of hardware resources.
The display terminal of the present invention can also be understood as a program executed by the computer or the control unit 11 or a method for setting a target area for moving object detection.

DESCRIPTION OF SYMBOLS 10 ... Display terminal, 11 ... Control part, 111 ... Acquisition part, 112 ... Correction | amendment part, 113 ... Display control part, 114 ... 1st reception part, 115 ... 2nd reception part, 116 ... Setting part, 12 ... Operation part, DESCRIPTION OF SYMBOLS 13 ... Communication part, 14 ... Display part, 141 ... Display surface, 15 ... Memory | storage part, 151 ... Shooting data, 152,152a ... Reference table, 20 ... Fisheye camera, 21 ... Control part, 22 ... Operation part, 23 ... Communication unit, 24: photographing unit, 241 ... fisheye lens, 25 ... storage unit

Claims (10)

A display unit for displaying an image on the display surface;
An acquisition unit that acquires shooting data indicating a shot image including image distortion caused by a shooting lens of the movie camera from the movie camera;
A correction unit that performs distortion correction to reduce the image distortion based on the shooting data acquired by the acquisition unit;
A display control unit for displaying the image on which the distortion correction has been performed by the correction unit on the display surface;
A first accepting unit that accepts a first operation for designating a target area for motion detection using the distortion-corrected image displayed on the display surface;
A display terminal comprising: a setting unit configured to set the moving object detection so as to perform the moving object detection of the target area specified by the operation received by the first receiving unit. A second receiving unit that receives a second operation for instructing display of a partial image obtained by enlarging a part of the captured image;
The display control unit
When the second operation is accepted, the partial image after the distortion correction is divided into a plurality of regions and displayed on the display surface,
The first reception unit
The display terminal according to claim 1, wherein an operation of selecting the target region from the plurality of regions is received as the first operation. The correction unit is
The distortion correction is performed so as to reduce the image distortion as the magnification of the partial image is higher,
The display control unit
The display terminal according to claim 2, wherein the partial image is divided into the plurality of regions when the enlargement ratio or the correction degree of the distortion correction exceeds a threshold level. The setting unit
The display terminal according to any one of claims 1 to 3, wherein the moving object detection is performed according to a detection sensitivity set for each target region. The setting unit
The display terminal according to claim 4, wherein the detection sensitivity of the target area is set according to a position of the target area in the captured image or a magnitude of the image distortion. The display control unit
The display terminal according to any one of claims 1 to 5, wherein after the first reception unit receives the first operation, the captured image in which the target area is clearly displayed is displayed. The display control unit
The display terminal according to claim 6, wherein the captured image in which the target area is clearly displayed is displayed so that the target area is displayed at a predetermined position on the display surface. The first reception unit
Receiving the first operation for designating the target area and a time at which moving object detection of the target area is performed;
The setting unit
The display terminal according to claim 1, wherein the moving object detection of the target area specified by the first operation is set to be performed at a time specified by the first operation. An acquisition step of acquiring shooting data indicating a shot image including image distortion caused by a shooting lens of the movie camera from the movie camera;
A correction step for performing distortion correction to reduce the image distortion based on the shooting data acquired in the acquisition step;
A display control step for displaying the image subjected to the distortion correction in the correction step on a display surface for displaying an image;
An accepting step of accepting an operation for designating a target area for moving object detection using the distortion-corrected image displayed on the display surface;
A setting step for setting the moving object detection so as to perform the moving object detection of the target area specified by the operation received in the receiving step based on the imaging data acquired in the acquiring step. Setting method. In the computer of the display terminal that displays the image on the display surface,
An acquisition step of acquiring shooting data indicating a shot image including image distortion caused by a shooting lens of the movie camera from the movie camera;
A correction step for performing distortion correction to reduce the image distortion based on the shooting data acquired in the acquisition step;
A display control step for displaying the image subjected to the distortion correction in the correction step on the display surface;
An accepting step of accepting an operation for designating a target area for moving object detection using the distortion-corrected image displayed on the display surface;
A program for executing a setting step for setting the moving object detection so as to perform the moving object detection of the target area specified by the operation received in the receiving step based on the imaging data acquired in the acquiring step.

Images