JP5105190B2 - Remote monitoring system and server - Google Patents

Description

  The present invention relates to a remote monitoring system capable of transmitting and displaying an image of a camera installed in a remote place to a mobile terminal device via a communication network and a server thereof, and more specifically, improving the usability of the mobile terminal device. The present invention relates to a remote monitoring system and a server thereof.

  There is a system in which an image of a camera installed at a remote place can be viewed by a mobile terminal device via a communication network (see Patent Document 1). In this system, the camera server and the mobile terminal device are configured to be communicable via a communication network, the camera is connected to the camera server, and a real-time image captured by the camera can be displayed on the mobile terminal device. The camera can be remotely controlled by the device.

  In this system, the entire landscape image captured by the camera is displayed at the center of the portrait display screen of the mobile terminal device. When the user of the mobile terminal device pans, tilts, or zooms the camera, the user presses a preset key (for example, left and right cursor keys for panning and up and down cursor keys for tilting). Thus, remote control of the camera can be performed.

However, since the conventional system has the following disadvantages (1) to (3), there is a problem that the usability of the mobile terminal device is poor.
(1) On a small display screen of a mobile terminal device, the monitoring target becomes small and difficult to see.
(2) It is difficult to quickly and accurately specify a desired location by specifying the display location with the cursor keys (pan, tilt).
(3) When the monitoring target is moving, even if the display location is specified, it may move immediately outside the screen, so it is difficult to see the desired location.
JP 2005-200999 A

  The present invention has been made to solve such problems, and its purpose is to control a camera installed at a remote place and to acquire and display an image photographed by the camera via a communication network. In a remote monitoring system that is possible with a mobile terminal device, the convenience of the mobile terminal device is improved.

The server of the present invention is capable of controlling a camera and transmitting an image photographed by the camera to a mobile terminal device via a network, and controlling the camera according to a key operation of the mobile terminal device And when the image captured by the camera is displayed on the mobile terminal device, the image is displayed on the mobile terminal device in response to an operation of a cursor key in a predetermined direction of the mobile terminal device. A moving body detecting means for detecting a moving body located in a predetermined direction from the center of the image, and a location including the moving body so that the detected moving body is displayed at the center of the display screen of the portable terminal device. It is a server having a moving body image cutout means to cut out.
Here, it is preferable that the moving body image cutout means cuts out according to the size of the display screen of the mobile terminal device so that the image is displayed on the entire display screen of the mobile terminal device.
Further, face detection means for detecting a face in an image displayed on the mobile terminal device in response to an operation of a key of the mobile terminal device set for zooming, and the detected face is the mobile terminal You may provide the face image clipping means which zooms out and cuts out the location containing the face so that it may be displayed on the center of the display screen of an apparatus.
The remote monitoring system of the present invention includes a remote monitoring system comprising the server of the present invention, a camera controlled by the server and capable of transmitting captured images to the server, and a mobile terminal device connected to the server via a network. System.

  According to the present invention, in a remote monitoring system in which a mobile terminal device can control a camera installed in a remote place and acquire and display an image captured by the camera via a network, the usability of the mobile terminal device Will improve.

The best mode for carrying out the present invention will be described below with reference to the drawings.
FIG. 1 is a diagram showing a configuration of a remote monitoring system according to an embodiment of the present invention. This remote monitoring system includes a server 2 and a plurality of portable terminals 3 each connected to a communication network 1 such as the Internet, and a camera 4 connected to the server 2 by a cable or the like.

  The server 2 has a function of controlling the camera 4 and a function of receiving image data of an image taken by the camera 4 and performing processing described later, and then transmitting the data to the portable terminal 3 via the communication network 1. ing. Details of the configuration and functions of the server 2 will be described later with reference to FIG.

  The portable terminal 3 can perform remote control of the camera 4 by operating a predetermined key and transmitting a control signal to the server 2 via the communication network 1. Further, an image photographed by the camera 4 and processed by the server 2 can be received via the communication network 1 and displayed. Details of the configuration and functions of the portable terminal 3 will be described later with reference to FIG.

  The camera 4 includes an image sensor such as a CCD or MOS, a camera signal processing unit that performs predetermined analog signal processing and A / D conversion on an image signal generated by the image sensor, and an external interface that is an interface to the server 2. It is possible to pan, tilt, and zoom in / out by a control signal from the server 2, and to transmit the captured image to the server 2. The camera 4 is a camera called a PTZ camera.

  FIG. 2 is a block diagram of the inside of the server 2. The server 2 includes a bus 21, a communication interface unit 22, an external interface unit 23, and an image processing unit 24 each connected to the bus 21. The image processing unit 24 includes a moving body detection unit 25, a tracking unit 26, a face detection unit 27, and an image size adjustment unit 28. Here, each unit in the image processing unit 24 is a functional block realized by a CPU (not shown) and a program stored in a memory. However, a part of these functional blocks may be configured by hardware. it can.

  The communication interface unit 22 is a unit that communicates with the mobile terminal 3 via the communication network 1, and the external interface unit 23 is a unit that communicates with the camera 4.

  The moving body detection unit 25 is a means for detecting a moving body in an image captured by the camera 4 using a frame difference of image data sent from the camera 4. When there are a plurality of moving objects, one moving object set by the user of the mobile terminal 3, for example, exists in a predetermined direction (eg, left) with respect to the center of the image of the current frame, and is closest to the center It has a function of outputting only a moving object as a detection result.

  The tracking unit 26 is means for tracking the moving body by storing the position data of the moving body detected by the moving body detecting unit 25 for each frame. The face detection unit 27 is a means for detecting the face of a person in the image based on the image data sent from the camera 4.

  The image size adjustment unit 28 is a unit that adjusts the size (number of vertical and horizontal pixels) of an image to be transmitted to the mobile terminal 3 so as to fit the display unit 33 of the mobile terminal 3.

  FIG. 3 is a block diagram of the interior of the mobile terminal 3. The mobile terminal 3 includes a bus 31, a communication interface unit 32, an image display unit 33, an operation input unit 34, a control unit 35, and a storage unit 36 that are connected to the bus 31.

  The communication interface unit 32 is means for communicating with the server 2 and other portable terminals 3 via the communication network 1. The display unit 33 includes an LCD and the like, and is a unit that displays an operation state of the mobile terminal 3, information input by the user from the operation input unit 34, an image transmitted from the server 2, and the like.

  The operation input unit 34 is an input unit that includes various keys (numeric keys, cursor keys, a determination key, an end key, and the like) used when the user uses the mobile terminal 3. The control unit 35 includes a CPU and is means for controlling the operation of the entire mobile terminal 3. The storage unit 36 includes a ROM that stores a program to be executed by the control unit 35, a RAM that serves as a work area during the operation of the control unit 36, and the like.

  Regarding the operation of the remote monitoring system having the above configuration, the operation flowchart of the server 2 shown in FIGS. 4 and 5 and the display unit corresponding to the operation content of the operation input unit 33 of the portable terminal 3 shown in FIGS. A description will be given with reference to FIG.

  When the user of the portable terminal 3 operates the operation input unit 34 and accesses the server 2 via the communication network 1, the flow of FIG. In this flow, the operations of the server 2 and the camera 4 are remotely controlled according to the key operation of the mobile terminal 3, and an image taken by the camera 4 is displayed on the display unit 33 of the mobile terminal 3. The program necessary for this is downloaded in advance from the server 2 to the mobile terminal 3 via the communication network 1 and stored in the storage unit 36.

  As shown in FIG. 4, first, the reference point of the image taken by the camera 4 is reset (step S1), and the process proceeds to a subroutine for initial image display processing (step S2, FIG. 5). In the initial image display process, first, the center point of the monitoring image being photographed by the camera 4 is calculated (step S31). Next, the entire monitoring image is reduced to the size of the display unit 33 of the portable terminal 3 (step S32), and transmitted to the portable terminal 3 together with the position data of the center point (step S33). The portable terminal 3 displays the received initial image and center point on the display unit 33.

  FIG. 6A shows an example of the initial image 331. Here, a horizontally long monitoring image is arranged at the center of the vertically long display unit 33, and the upper and lower areas are areas without an image. Note that a symbol composed of a combination of ◯ and x displayed at the center of the initial image 331 represents the center point of the monitoring image.

  After executing step S33, the server 2 returns to the main routine, sets the center point of the image currently being transmitted to the terminal device 3 as a reference point, and detects a moving object (steps S3 and S4). In step S4, a moving object in the monitoring image is detected by taking a frame difference of the monitoring image of the camera 4 or the like. The detected position data in the screen of the moving body is stored in a storage unit (not shown) in the server 2.

  Next, the server 2 checks whether the initial screen is displayed on the mobile terminal 3 (step S5). Here, the initial image is transmitted in step S33, and since then the transmitted image is not changed (S5: YES), the process proceeds to step S6.

  In step S6, it is checked whether the right cursor key or the left cursor key of the operation input unit 34 is pressed on the portable terminal 3. Here, as shown in FIG. 6B, it is assumed that the right cursor key 341 is pressed. Accordingly, the process proceeds to step S7, the barycentric positions of all the moving objects in the monitoring image are obtained, and then the process proceeds to step S8 to check whether there is a moving object in the designated direction (here, right) from the reference point.

  If there is a moving object (S8: YES), after selecting the moving object having the closest distance between the reference point and the center of gravity, the center of the selected moving object is set as the reference point of the monitoring image (step S9, S10). Next, a monitoring image is cut out in accordance with the size of the display unit 33 of the mobile terminal 3 around the reference point, and transmitted to the mobile terminal 3 (steps S11 and S12).

  In the case of FIG. 6A, a monitoring image centered on a child riding a tricycle is cut out on the right side of the reference point, and is displayed on the entire vertically long display unit 33 as shown in FIG. 6C. Here, the cutout range changes according to the display screen size of the mobile terminal 3.

  If it is determined in step S6 that the right button or the left button is not pressed, or if it is determined in step S8 that there is no moving body in the specified direction from the reference point, both return to step S1.

  After executing step S12 as described above, the process proceeds to step S4, where the moving body is detected again, and then in step S5, it is checked whether the initial screen is displayed on the portable terminal 3. In this time, as shown in FIG. 6C, since the moving body cut-out screen is displayed (S5: NO), the process proceeds to step S13, and the position information of the moving body of the current frame is acquired. This position information is acquired from the tracking result of the moving body in the previous frame. Here, as shown in FIG. 6C, since the tricycle and the child riding on the tricycle are detected and stored as the moving body, the position information of the current frame of the moving body is acquired.

  In step S14, the center of gravity of the moving body is set as a new reference point. Here, the center of gravity of the current frame of the tricycle and the child riding on it is taken as the new reference point.

  Next, the mobile terminal 3 checks whether the up cursor key or the down cursor key is pressed (step S15). If it is pressed (S15: YES), it is checked whether it is the up cursor key (step S16). Here, it is assumed that the up cursor key 342 is pressed as shown in FIG. 7A.

  Accordingly, the process proceeds to step S17 to perform face detection, and then proceeds to step S18 to check whether a face has been detected. If detected (S18: YES), the face is zoomed in (step S19), and the zoomed-in image is cut out in accordance with the size of the display unit 33 of the mobile terminal 3 (step S20) and transmitted to the mobile terminal 3. (Step S12). As a result, the face zoom image 334 shown in FIG. 7B is displayed on the display unit 33 of the portable terminal 3.

  After executing step 12, the process proceeds to step S4 to detect a moving object. As in the previous time, the process moves from step S13 to S14 to S15. Here, as shown in FIG. 7C, if the down cursor key 343 is pressed, the process moves from S15: YES → S16: NO → S21 to check whether a face zoom image is displayed.

  Here, as shown in FIG. 7B, since the face zoom image is displayed (S21: YES), the process proceeds to step S22, and the reference point (here, the center of gravity of the moving body including the tricycle and the child riding on it) Is cut out in accordance with the size of the display unit 33 of the mobile terminal 3 (step S22) and transmitted to the mobile terminal 3 (step S12). As a result, the image shown in FIG. 7B is zoomed out on the display unit 33 of the mobile terminal 3, and the image 332 shown in FIG. 7D is displayed as in FIG. 6C.

  If it is determined in step S15 that the up cursor key or the down cursor key is not pressed (S15: NO), the process proceeds from step S22 to S12. Also in this case, the image 332 shown in FIG. 6C or FIG. 7D is displayed on the display unit 3 of the mobile terminal 3.

  After executing step 12, the process proceeds from step S4 to S13 to S14 to S15. Here, as shown in FIG. 7E, if the down button 343 is pressed, the process moves from S15: YES → S16: NO → S21 to check whether a face zoom image is displayed.

  This time, as shown in FIG. 7D, since the zoomed out image is displayed (S21: NO), the process proceeds to step S1. And the initial screen 331 shown in FIG. 7F is displayed on the display part 33 of the portable terminal 3 by performing next step S2.

  In the flow of FIG. 4, when S15 is NO, the process proceeds to S22. However, a step for checking whether or not the right cursor key or the left cursor key is pressed is added. If the key is pressed, the process proceeds to step S7. It may be configured to proceed. If comprised in this way, as shown in FIG. 8A in the state where the image 332 centering on the tricycle and the child riding on it is displayed on the display unit 33 of the portable terminal 3 as shown in FIG. When the right cursor key 341 is pressed, the process proceeds as S7 → S8: YES → S9 → S10 → S11 → S12, and as shown in FIG. Is displayed. Further, when the right cursor key 341 is pressed from this state as shown in FIG. 8C, the initial image is displayed as shown in FIG. 8D by proceeding in the order of S7 → S8: NO → S1 → S2.

  Although not shown in the drawing, the flow in this figure ends when the server 2 detects that the end button of the operation input unit 34 of the mobile terminal 3 is pressed.

  As described above, according to the remote monitoring system of the present embodiment, when the image captured by the camera 4 and cut out by the server 2 is displayed on the display unit 33 of the mobile terminal 3, the mobile terminal 3. When the right cursor key or the left cursor key of the operation input unit 33 is pressed, a new image is cut out centering on the right or left moving body of the center of the currently displayed image, and the mobile terminal 3 is displayed. Since it can be displayed on the part 33, the place to be seen can be seen quickly and surely only by pressing the cursor key once. At this time, since all image processing is performed on the server 2 side, the load on the terminal device 3 side does not increase.

  In addition, since the moving body is tracked for each frame, it is possible to prevent a situation where a portion detected and displayed by the right cursor key or the left cursor key moves outside the screen. Furthermore, since an image is cut out in accordance with the size of the display unit 33 of the mobile terminal 3, even a small display unit of the mobile terminal 3 can display the monitoring target in an easily viewable state.

  In addition, by operating the up cursor key, face detection is performed, the magnification is set to a preset zoom magnification, and the periphery of the face is cut out in accordance with the size of the display unit 33 of the mobile terminal 3, so that You can see your face quickly and accurately clearly.

  Next, with respect to an editing process in which the user of the mobile terminal 3 extracts and saves an image of a desired mobile object from images stored in a storage device (such as a hard disk) in the server 2. This will be described with reference to the flowchart of FIG. In the flow of this figure, when the user of the mobile terminal 3 accesses the server 2 via the communication network 1, selects an image to be edited from the images stored in the server 2, and inputs an execution instruction. Be started.

  The server 2 reads the image selected by the user from the storage device, and detects the moving object in the same procedure as in step S4 in FIG. 4 (step S41). When a moving body is detected (step S42: YES), it is set as an editing target (step S43), and an image with a rectangular frame around the moving body is transmitted to the terminal device 3 (step S44).

  When the user of the mobile terminal 3 operates the cursor key while viewing the image displayed on the display unit 33 to select a frame around the moving body and presses the enter button (step S45), the server 2 This is detected, and an image of the moving body is cut out from the image of the frame in which the determined moving body is shown, and the process of saving the edited image in the storage device in the server 2 is started (step S46). Editing is continued until the user of the portable terminal 3 instructs the end of editing (step S47: YES).

  By executing the flow shown in FIG. 9, it is possible to cut out, edit, and save only a desired person and object from long-time recorded images. In addition to observing images in time series, images with no movement are not excluded from editing targets, and only a desired person's part can be cut out and edited from a wide range of images. Therefore, for example, in the case of a system in which a camera is installed in a nursery school or a pet hotel, it is possible to cut out and save only the portion of the recorded image showing the child or pet.

  In addition, by storing an image obtained by cutting out a moving person as a search index, it is possible to easily search for an image captured over a wide range. Furthermore, when the police search for a criminal from long-time recorded images, it is possible to greatly reduce the labor of searching for the criminal by using a function that can automatically extract a person and enlarge it with a simple operation. it can.

  In the above embodiment, a PZT camera is used as the camera 4, but a wide-angle fixed camera may be used. A wide-angle fixed camera and a PTZ camera may be used in combination. In the case of a fixed camera, panning and tilting cannot be performed. The zoom can be handled by electronic zoom (digital zoom). Further, compared with the PTZ camera, there is an advantage that it is not necessary to control the control right of the camera between a plurality of portable terminals.

  In the above embodiment, the camera and the server are separate devices. However, the camera and the server may be configured as a single device.

  Furthermore, in the above embodiment, the processes and operations shown in FIGS. 4 to 8 are for live images, but these processes and operations are also possible for recorded images stored in the server 2.

It is a figure which shows the structure of the remote monitoring system of embodiment of this invention. It is a block diagram inside the server of FIG. It is a block diagram inside the terminal device of FIG. It is a flowchart which shows operation | movement of the remote monitoring system of the implementation mobile of this invention. 5 is a flowchart of an initial image processing subroutine in the flowchart of FIG. 4. It is a figure which shows the change of the image of a display part according to the operation content of the operation input part of a portable terminal. It is a figure which shows the change of the image of a display part according to the operation content of the operation input part of a portable terminal. It is a figure which shows the change of the image of a display part according to the operation content of the operation input part of a portable terminal. It is a flowchart which shows the edit process which extracts the image by which the user of the portable terminal 3 image | photographed the desired mobile body from the image preserve | saved at the server 2, and preserve | saves it.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Communication network, 2 ... Server, 3 ... Portable terminal, 4 ... Camera, 24 ... Image processing part, 25 ... Moving body detection part, 26 ... Tracking part, 27... Face detection unit, 28... Image size adjustment unit.

Claims (6)

A server capable of controlling a camera and transmitting an image captured by the camera to a mobile terminal device via a communication network and having a function of controlling the camera in accordance with a key operation of the mobile terminal device. And
When an image photographed by the camera is displayed on the mobile terminal device, a predetermined direction from the center of the image displayed on the mobile terminal device in response to an operation of a cursor key in the predetermined direction of the mobile terminal device Mobile object detection means for detecting a mobile object located in the mobile terminal, and mobile object image clipping means for cutting out a location including the mobile object so that the detected mobile object is displayed at the center of the display screen of the mobile terminal device. The server characterized by having. In the server of claim 1,
The server is characterized in that the moving body image cutout means cuts out according to the size of the display screen of the mobile terminal device so that the image is displayed on the entire display screen of the mobile terminal device. In the server of claim 1,
The moving body detecting means, when detecting a plurality of moving bodies, selects the moving body closest to the center of the image displayed on the portable terminal device. In the server of claim 1,
A face detection unit configured to detect a face in an image displayed on the mobile terminal device in response to an operation of a key of the mobile terminal device set for zooming, and the detected face of the mobile terminal device A server comprising face image cutout means for zooming out and cutting out a portion including the face so as to be displayed in the center of the display screen. In the server according to claim 4,
A server, comprising: an initial setting image cutout unit that cuts out an image of an initially set monitoring target area to be displayed on the display screen of the mobile terminal device when a face is not detected by the face detection unit.   A server according to any one of claims 1 to 5, a camera controlled by the server and capable of transmitting captured images to the server, and a mobile terminal device connected to the server via a communication network Remote monitoring system consisting of

Images