JP5896438B2 - Surveillance camera system and monitoring method - Google Patents

Description

  The present invention relates to a surveillance camera system and a surveillance method for taking an image for crime prevention, management, and the like and monitoring it with a monitor via an electric communication line.

  Conventionally, for the purpose of crime prevention, investigation or management, the monitoring target points such as the entrances and exits of buildings such as apartment buildings, stores or streets, factories and distribution centers are monitored by the monitoring camera in a timely manner. A camera system for sending and monitoring to a monitor of a user terminal via a communication line has been introduced. In general, a system is known in which a moving image captured by a camera is recorded, transmitted to a user side by wire or wirelessly, and appropriately monitored by simultaneous monitoring or subsequent reproduction (for example, see Patent Document 1).

JP 2002-77882 A

  When moving image data is acquired by the UDP method as in the conventional surveillance camera system, there is a problem that packet loss is likely to occur when data is transmitted from the camera to the server. Therefore, there is a problem that data cannot be stored in the cloud server over the Internet line (WAN).

  In addition, in the simultaneous monitoring, a moving image is always transmitted from the camera to the user terminal, so that there is a problem that the amount of data to be communicated increases, the load on the communication network through which the traffic passes is large, and traffic is congested. Therefore, if there are a large number of cameras, they cannot be installed on all the same networks, and there are problems that a number of servers are required, and data on cloud servers over the Internet (WAN) There was a problem that it could not be accumulated.

  In addition, there is a problem that the data capacity becomes enormous because video data is stored in the server, and that long-term storage is not possible. In addition, it is necessary to prepare a large number of large-capacity servers, and there is a problem that costs increase. Since the amount of data is limited, there is a problem that the resolution cannot be increased and a clear image cannot be obtained during reproduction.

  In addition, the recording conditions on the camera side are the same as the storage conditions on the recording server side, and in order to reduce the moving image data in the mainstream H.264 format, the network camera side In addition, it is necessary to manufacture a camera having a function of recording in the H.264 format, which is expensive.

  In addition, when a moving image is played back on a terminal such as a smartphone, there is a problem that it takes a long time to execute the operation after performing an operation such as playback due to a large amount of data. There was a problem that time lag and freeze occurred due to the time required for playback, and trouble discovery was delayed. To monitor multiple cameras, it cannot be played back smoothly on a smartphone, it must be monitored where there is a large-capacity computer such as a monitor room, and it is difficult to check alongside work while moving between places There was a problem that the burden was large. Therefore, the current situation is that the recorded images are not used even if they are installed at a high cost, and are only taken.

  In addition, the camera system described in the above document has a problem that when viewing a real-time image during shooting, it is not possible to rewind from the current image, play back after rewinding from the current time, or fast forward to the current time. .

  Further, in shooting, there may be a trouble that a time when shooting could not be performed for some reason may occur. However, in that case, there is a problem that an image skips during playback of a moving image.

  Therefore, the first object of the present invention is to provide a surveillance camera system that solves the above-described problems, makes it easy to reproduce high-quality images and rewind live images, and can reduce the amount of data and reduce costs. There is to do.

In order to achieve the above object, according to a first aspect of the present invention, there is provided a server connected to a camera group for taking a still image via a network, and a terminal connected to the server via a network and displaying an image. Equipped,
The server
Camera image acquisition means for capturing a still image in the camera group and inputting the captured camera image to the server at regular time intervals,
Primary data creation means for compressing the camera image to create and store primary data;
A thumbnail image receiving means for receiving a thumbnail image request for one or a plurality of cameras having a monitoring authority from the terminal;
Thumbnail image creation means for creating a thumbnail image from the camera image for a camera that has received a thumbnail image request;
Thumbnail image transmitting means for transmitting the thumbnail image created by the thumbnail image creating means to the terminal;
Live image selection accepting means for accepting selection of a camera for displaying a live image from the terminal,
Live image transmission means for transmitting the camera image as a live image to the terminal instead of a thumbnail image for the camera selected by the live image selection receiving means,
Live image rewind receiving means for receiving a rewind request from a live image from the terminal,
For each rewind request, a rewind live image transmission means for extracting primary data at a certain point in time from a previously transmitted image to the terminal and transmitting it to the terminal as a rewind live image;
Rewind live image fast forward accepting means for accepting a fast forward request from the rewind live image from the terminal,
Fast-forward live image transmission means for extracting primary data at a certain time in the future from the previously transmitted image to the terminal and transmitting it to the terminal as a fast-forward live image until the current image is reached for each fast-forward request Have
The terminal
Image display means for displaying an image transmitted from the server;
During thumbnail image display, thumbnail image request means for requesting a thumbnail image from the server at regular intervals;
Live image requesting means for requesting a live image from the server at every camera image acquisition interval from the camera to the server during live image display,
Rewind start request means for accepting an input of a rewind request during live image display and requesting a rewind live image from the server,
During rewind live image display, rewind continuation requesting means for requesting the server for a rewind live image every fixed time shorter than the request interval of the live image,
Rewind live image fast forward start request means for accepting an input of a fast forward request and displaying a fast forward live image from the server while displaying the rewind live image.
Rewind live image fast forward continuation requesting means for requesting the server for a fast forward live image every fixed time shorter than the live image request interval during fast forward live image display,
There is provided a surveillance camera system characterized by comprising: The “thumbnail image” is an image reduced to display a list of a large number of images. The “live image” indicates a real-time image.

  According to the first aspect of the present invention, since data acquisition is performed with a still image, the amount of data that flows simultaneously on the network is small, and a large number of cameras can be installed on the same network. In addition, since the data transfer amount is small, data can be stored in the cloud server via the Internet line or the like. When a live image is viewed, if an oversight or the like is noticed, the user can rewind and review the image as it is, and can check the image from the past to the current time while confirming the image by fast-forwarding.

  It is preferable that the server further includes secondary data creation means for converting the primary data into secondary data that is compressed at a predetermined time, combined with a time stamp, and compressed in a moving image format. The compression format for moving images is H.264, which is currently highly compressed. More preferably, it is H.264. It is lightweight and easy to implement regardless of hardware or software.

The server further includes:
List accepting means for accepting a camera list request from the terminal;
Authority confirmation means for extracting a list of cameras having authority to monitor the terminal from the camera group;
List sending means for sending a list of cameras extracted by the authority checking means to the terminal;
Thumbnail image selection accepting means for accepting from the list one or more selections of cameras for displaying thumbnail images from the terminal;
And the terminal
List display means for requesting a camera list to the server and displaying a camera list acquired from the server,
Camera selection means for selecting a camera that displays a thumbnail image or a camera that displays a live image,
It is preferable that it has. It is preferable that the thumbnail image accepting unit accepts a thumbnail image request for a camera selected from the list.

The server further includes:
Relive image receiving means for receiving a relive image request from a rewind live image from the terminal,
Relive image transmission means for extracting primary data at a certain time in the future for a predetermined time from the image previously transmitted to the terminal and transmitting it to the terminal as a relive image for each relive image request;
And the terminal
Relive image start request means for accepting an input of a relive image request during rewind live image display and requesting the relive image from the server,
Relive image continuation request means for requesting the relive image from the server at every camera image acquisition interval from the camera to the server during the relive image display,
It is preferable to have. In the present application, the “live image” indicates a real-time image, whereas the “relive image” indicates a reproduced image that is rewound from the live image to the past.

According to a second aspect of the present invention, in order to achieve the above object, a server connected to a camera group that captures a still image via a network transmits a still image to the camera group at regular intervals. is captured, the TCP / IP system from the camera unit, acquires the camera image in a JPEG still picture, within a certain period to save a still image, save Tadashi converted into compressed video file after a certain period of time And a means for receiving a rewind live image from the terminal and displaying it on the terminal during the display of the live image within the predetermined period of time stored as a still image, and displaying the current image during the rewind live image display. A surveillance camera system is provided that includes means for accepting display of a fast-forward live image until reaching . Since communication is performed after confirming the communication partners with each other, unlike the UDP method, data transmission / reception is not lost on the network, data integrity is high, and reliability is high. It is not necessary for the server to be directly connected to the camera. If the server is connected on the network, an image can be reliably provided even with a service based on cloud computing. Further, it is not necessary to provide a server device or a server function on the camera side, and space and cost can be reduced. In the present invention, the “camera” may have a moving image shooting function as long as it has a still image shooting function and an output function. The camera is preferably an IP camera, but can also be an analog camera. In addition, since it is a still image for a certain period and then a compressed moving image, the data capacity is small, so that long-term storage is possible. Since it is stored as a still image within a certain period, it is possible to quickly respond to operations such as real-time image transmission and rewind / fast-forward. A system that combines input / temporary storage for still images and subsequent storage for compressed movies, which significantly reduces the amount of data compared to input / storage for movies, enables connection to many cameras, and images The quality of the can be kept high.

According to a third aspect of the present invention, a server connected to a camera group that captures a still image via a network captures a camera image obtained by capturing the still image in the camera group at regular intervals. Camera image acquisition step to be input to,
A primary data creation step in which the server compresses the camera image to create and store primary data;
A camera selection step in which a terminal connected to the server via a network and displaying an image selects a camera that displays a live image;
A live image transmitting step in which the server transmits a camera image of the selected camera as a live image to the terminal;
A live image display step in which the terminal displays a live image transmitted from the server;
The terminal receives a rewind request input during live image display, and requests a rewind start request step to request a rewind live image from the server.
A rewind live image transmission step in which the server extracts primary data at a time point past a certain time from the image transmitted last time to the terminal and transmits it to the terminal as a rewind live image;
Rewind live image display step in which the terminal displays a rewind live image transmitted from the server,
A rewind continuation request step for requesting a rewind live image to the server at a certain time shorter than the request interval of the live image while the terminal is displaying the rewind live image.
The server repeats the rewind live image transmission step and the rewind live image display step for each rewind request,
The terminal receives a fast-forward request input while displaying a rewind live image, and requests a fast-forward start request requesting a server for a fast-forward live image,
A fast-forward live image transmission step in which the server extracts primary data at a certain time in the future from the image previously transmitted to the terminal and transmits it to the terminal as a fast-forward live image. A fast-forward live image display step for displaying a live image,
While the fast-forward live image is displayed, the terminal requests a fast-forward live image from the server every predetermined time shorter than the live image request interval.
There is provided a monitoring method characterized by including repeating the fast-forward live image transmission step and the fast-forward live image display for each fast-forward live image request until the server reaches the current image.

  According to the present invention, high-quality playback and live image rewinding can be easily performed, and the amount of data can be reduced and the cost can be reduced.

It is a figure which shows an example of the network structure of Example 1 of the surveillance camera system of this invention. It is a block diagram of the server of Example 1 of the surveillance camera system of this invention. It is a block diagram of the terminal (mobile) of Example 1 of the surveillance camera system of the present invention. It is a block diagram of the terminal (PC) of Example 1 of the surveillance camera system of the present invention. It is a flowchart which shows the procedure outline | summary from the connection of the camera and server in Example 1 of the surveillance camera system of this invention to image editing. It is an image data collection flowchart in Example 1 of the surveillance camera system of the present invention. It is a data compression flow figure of Example 1 of the surveillance camera system of the present invention. It is a flowchart which shows the procedure outline | summary from the connection of the terminal and server in Example 1 of the surveillance camera system of this invention to an image display. It is a live image selection flowchart of Example 1 of the surveillance camera system of the present invention. It is explanatory drawing which shows the various requests | requirements from the terminal in Example 1 of the surveillance camera system of this invention, and the response from a server.

  EXAMPLES Hereinafter, although this invention is concretely demonstrated using an Example about this invention, this invention is not limited to these.

  In this embodiment, a monitoring camera is operated on a server via a network, and the server collects, edits, and stores images, and the server monitors the user's terminal via the network in response to a request from the terminal. , And playback / rewind / fast-forward of past images as well as playback / rewind / fast-forward and re-live playback of live images. For example, it can be used to manage images transmitted from a surveillance camera attached to a production line in a local factory connected via a network to a smartphone of a person in charge at a management department at the head office or a person in charge at each factory.

{Constitution}
FIG. 1 is a diagram illustrating an example of a network configuration of a monitoring camera system according to a first embodiment of the present invention. On a network 500 including a communication network such as a LAN, the Internet, a Wi-Fi line, or a 3G line, (1) a plurality of monitoring cameras 400A to 400F as cameras for taking images, and (2) a network 500. Connected to the monitoring camera 400 via the network, the recording server 100 as a server that acquires, stores and transmits image data taken by the camera to the terminal, and (3) connected to the recording server 100 via the network 500, As terminals for receiving and displaying image data from 100, mobile terminals 200A and B such as smartphones, and viewing PCs 300A to 300C such as desktop PCs (personal computers) and notebook PCs are connected. Hereinafter, when the mobile terminal and the viewing PC are combined, they are referred to as “terminal” or “Viewer”.

  The recording server 100 is not limited to the case where the monitoring camera 400 exists in the same LAN, and may be connected via the Internet as long as they are connected to the network 500 via the router 600. In this embodiment, the communication between the monitoring camera 400 and the recording server 100 is a TCP / IP system, and even when there is no camera and server in the same LAN, image data can be reliably transmitted and received, and the reliability is high. . The monitoring camera 400 is an IP camera in this embodiment, but may be an analog camera. The viewing PC 300, which is a terminal for image display, is not limited to being in the same LAN as the recording server 100. If the viewing PC 300 is connected to the network 500 via the router 600, it may be connected via the Internet. Good. The mobile terminal 200, which is an image display terminal, may also be connected via a plurality of networks such as a LAN, the Internet, and a cellular phone network (Wi-Fi line, 3G line, etc.).

  One recording server 100 can acquire images from a plurality of monitoring cameras 400. In the conventional IP camera system, the number of cameras that can be managed by one server with the same capacity is about 20 IP cameras. However, in this embodiment, even when recording once per second, in the same network environment, 100 IP cameras can be connected. Therefore, cost reduction and large-scale monitoring can be performed simultaneously. In addition, since the server side automatically distributes network traffic, network design is unnecessary, and it is only necessary to connect the camera to the port.

  FIG. 2 is a configuration diagram of a server according to the first embodiment of the surveillance camera system of the present invention. The recording server 100 includes a CPU 101 with a memory 102 that is a cache memory, a control / arithmetic unit having a device driver, a storage device 110 having a main storage device such as a DRAM and an auxiliary storage device such as a hard disk, a network interface 104, and the like. And an input / output device including a display 103 as a display device, a keyboard 105, a mouse 106, and the like. In the storage device 110, an operating system 114 is stored in addition to a primary image folder 111, a secondary image folder 112, a photographing / editing / transmission program 113, an authentication database, an environment setting folder, and the like. The shooting / editing / transmission program 113 is normally stored in the auxiliary storage device of the storage device 110, and is loaded into the main storage device at the time of execution. In the primary image folder 111, a compressed file of the camera image is stored as primary data, and in the secondary image folder 112, the primary data stored in the primary image folder 111 is compressed and converted into a moving image. Accumulate as next data. In the authentication database, ID, password, port number and IP address of each monitoring camera 400, mobile terminal 200, and viewing PC 300, and individual identification information (UID) are stored in a terminal without an IP address. In this embodiment, since the recording server 100 is integrated with the server and the terminal, and also serves as a terminal for displaying the image itself, and for maintenance and management, the display 103 and the keyboard 105 as the input means. And a mouse 106. When the recording server does not need to reproduce the camera image, the terminal function as the image display device may not be provided. The environment setting folder stores the image acquisition timing of each monitoring camera, the creation interval of primary image data and secondary image data, compression conditions, and the like.

  The recording server 100 realizes the function of a computer capable of acquiring a camera image and sending an image to a terminal according to the present invention by causing the CPU 101 to load and execute the shooting / editing / transmission program 113 in the memory 102. The CPU 101 is an arithmetic processing device mounted on a normal computer, executes various programs, and performs various controls.

  The recording server 100 may be a server group consisting of a plurality of servers in addition to a single server. For example, for the secondary image folder, for secondary data after a certain period (for example, 24 hours), the storage destination may be a secondary image folder provided in a server separate from the server that acquires the camera image. By separating past stored data that is not frequently played back, a larger number of cameras can be monitored on the same network.

  The shooting / editing / transmission program is connected to the computer with a camera connection function for connecting to the surveillance camera, a terminal connection function for connecting to the terminal, and stationary at the connected surveillance camera every fixed time (for example, 1 second). A camera image acquisition function that captures images and inputs the captured camera images to a recording server, a primary data creation function that compresses the camera images to create and accumulate primary data, and a list reception function that accepts a camera list request from a terminal , An authority confirmation function for extracting a list of cameras having authority to monitor the terminal, a list transmission function for transmitting the extracted camera list to the terminal, and one or more selections of cameras for displaying thumbnail images from the terminal. A thumbnail image selection accepting function for accepting a thumbnail image request for one or a plurality of cameras with authority to monitor, Thumbnail image creation function that creates thumbnail images from camera images for cameras that have received a thumbnail image request, thumbnail image transmission function that transmits thumbnail images to the terminal, and live image selection reception that accepts selection of cameras that display live images from the terminal Function, live image transmission function that transmits camera image data to the terminal as a live image instead of thumbnail images for the selected camera, live image rewind acceptance function that accepts a rewind request from the live image from the terminal, each rewind request In addition, a rewind live image transmission function that extracts primary data for a certain time (for example, 1 second) in the past from the image previously transmitted to the terminal and transmits it to the terminal as a rewind live image, and a rewind live image from the terminal. Rewind live image fast forward accepting machine that accepts fast forward requests from For each fast-forwarding request, until the current image is reached, the fast-forwarding live that extracts primary data at a future time from the previously transmitted image to the terminal and transmits it to the terminal as a fast-forwarding live image. This is a program for realizing the image transmission function. “Current image” means the latest image acquired from the monitoring camera. Therefore, the current image changes to a new image each time a camera image is input to the server.

  In the present embodiment, the shooting / editing / transmission program further combines the primary data with the computer every second time (for example, one hour), and attaches a time stamp to the compressed secondary data. This is a program for realizing the function of creating secondary data to be converted.

  In the present embodiment, the shooting / editing / transmission program further has a re-live image reception function that accepts a re-live image request from a rewind live image from the terminal to the computer, and is constant from the previous image transmitted to the terminal for each re-live image request. A relive image transmission function that extracts primary data at a future time and transmits it to the terminal as a fast-forward live image for the time, in detail, for the camera image acquisition interval from the setting monitoring camera 400 to the recording server 100 in the camera. It is a program to realize.

  In this embodiment, the connection between the surveillance camera and the recording server is performed by the TCP / IP method, and the surveillance camera is specified. Therefore, after authenticating with the user ID and password set on the recording server side, Make a request. The connection between the terminal and the recording server is also performed by the TCP / IP method, authentication is performed with a user ID and a password, and image transmission is performed after confirming that the terminal is a terminal registered in the recording server. The authentication is preferably performed by an authentication database on the recording server.

  If the terminals and cameras are not on the same LAN but are connected via a network via a router, the IP address of the router to be connected and the port number assigned to each are used. When connected via the Internet, set the port number of the router between the two to “valid”. A mobile terminal without an IP address uses a UID.

  FIG. 3 is a configuration diagram of a terminal (mobile) according to the first embodiment of the surveillance camera system of the present invention. The mobile terminal 200 includes a control / arithmetic apparatus having a CPU 201 with a memory 202, a device driver, and the like, a storage device 210, a communication control device that transmits and receives data, a display 204 as a display device, operation buttons, a touch panel, and the like. An output device is provided. The storage device 210 stores an image display program 213 and an operating system 214. The mobile terminal 200 is, for example, a mobile phone such as a smartphone, and the CPU 201 loads the image display program 213 into the memory 202 and executes it, thereby realizing a computer function capable of displaying an image according to the present invention. The CPU 201 is an arithmetic processing device mounted on a normal mobile terminal, executes various programs, and performs various controls.

  FIG. 4 is a configuration diagram of a terminal (PC) in the first embodiment of the surveillance camera system of the present invention. The viewing PC 300 includes a control / arithmetic unit having a CPU 301 and a device driver with a memory 302, a storage device 310 having a main storage device such as a DRAM and an auxiliary storage device such as a hard disk, a communication control device such as a network interface 304, And an input / output device including a display 303 as a display device, a keyboard 305, a mouse 306, and the like. The storage device 310 stores an image display program 313 and an operating system 314. The browsing PC 300 is, for example, a desktop PC, a notebook PC, a tablet terminal, or the like. The CPU 301 loads the image display program 313 into the memory 302 and executes it, thereby realizing a computer function capable of displaying an image according to the present invention. The CPU 301 is an arithmetic processing device mounted on a normal PC, executes various programs, and performs various controls.

  The image display program includes a terminal connection function for connecting to a recording server to a computer, a list display function for requesting a camera list to the recording server and displaying a camera list acquired from the server, and a monitoring camera for displaying thumbnail images. Or a camera selection function for selecting a monitoring camera for displaying a live image, an image display function for displaying an image transmitted from a recording server, and a thumbnail at regular intervals (for example, 1 second) during thumbnail image display. A thumbnail image request function for requesting an image from the recording server, a live image display function during live image display, and a live image request function for requesting a live image from the recording server at every camera image acquisition interval from the monitoring camera to the recording server. The rewind start request function that accepts the input of the rewind request and requests the rewind live image from the recording server, During live image display, a rewind continuation request function for requesting a rewind live image to the recording server every fixed time (for example, 0.2 seconds) shorter than the live image request interval, A fast-forward live image fast-forward start request function that accepts a fast-forward request input from the user and requests the fast-forward live image from the recording server, while displaying the fast-forward live image, a certain time shorter than the request interval of the live image (for example, 0.2 seconds) ) A rewind live image fast forward continuation request function that requests a fast-forward live image to the recording server every time, and a relive image start request function that accepts a relive image request and displays a relive image from the server while the rewind live image is displayed. During the live image display, the live image that requests the live image from the server at every camera image acquisition interval from the camera to the server Is a program for realizing the connection request function.

  The recording server 100 can check the IP address, port number, and UID of the monitoring camera and terminal every time connection is started, and can identify the connection partner by connection using the IP address, port number, and UID. Authenticate with If the router port number is set to “valid”, the router on the terminal side can be set from the recording server 100, and a global IP address or a private IP address can be set. Therefore, even when the address of the PC in the LAN has been converted, since the surveillance camera is recognized, the camera image can be acquired. It is not necessary to set global IP address and private IP address on the monitoring camera side, and it is possible to send and receive camera images even when a broadband router is used and the PC address in the LAN is converted. In addition, even if the user's PC operation capability is at a level where various settings cannot be made, image display is possible. When the monitoring camera and the recording server are limited to the same LAN, recognition may be performed using a local IP address and a port number.

  In this embodiment, the recording server 100 and the viewing PC 300 are both configured as personal computers, and have a clock function or the like that a normal personal computer has. The mobile terminal 200 also has a clock function and the like.

  In the present embodiment, the recording server 100 connected to the monitoring camera 400 for capturing a still image via the network 500, and a terminal (mobile terminal 200 or viewing PC 300) connected to the recording server 100 via the 500 for displaying an image. The recording server 100 includes (1) camera image acquisition means that causes the monitoring server 400 to capture a still image and input the captured camera image to the recording server 100 at certain time intervals, (2) ) Primary data creation means for creating and storing primary data by compressing the camera image; (3) list acceptance means for accepting a camera list request from the terminal; and (4) monitoring camera 400 monitoring the terminal. Authority confirmation means for extracting a list of authorized cameras, (5) List transmission for transmitting a list of cameras extracted by the authority confirmation means to the terminal (6) Accepting from the list one or more selections of the monitoring camera 400 for displaying thumbnail images from the list, that is, accepting thumbnail image requests for one or more cameras with monitoring authority. Means, (7) a thumbnail image creation means for creating a thumbnail image from the camera image for the monitoring camera 400 that has received the thumbnail image request, and (8) transmitting the thumbnail image created by the thumbnail image creation means to the terminal. Thumbnail image transmitting means, (9) Live image selection receiving means for receiving selection of a monitoring camera 400 for displaying a live image from the terminal, and (10) Thumbnail for the monitoring camera 400 selected by the live image selection receiving means. The camera image data is replaced with the image instead of the image. Live image transmitting means for transmitting to the terminal as a live image, (11) live image rewind receiving means for receiving a rewind request from the live image from the terminal, and (12) for each rewind request, Rewind live image transmission means for extracting primary data at a point in time past the transmitted image and transmitting it to the terminal as a rewind live image. (13) A fast forward request from the rewind live image is sent from the terminal. Rewind live image fast-forward accepting means for accepting, (14) For each fast-forward request, until the current image is reached, primary data is extracted from the previously transmitted image to the terminal for a predetermined time and the fast-forward live image is extracted. (15) The primary data is combined at regular time intervals and attached with a time stamp. Secondary data creating means for converting to secondary data compressed in a picture format; (16) relive image accepting means for accepting a relive image request from a rewind live image from the terminal; and (17) for each relive image request. Relive image transmission means is provided for extracting primary data at a time point that is a certain time later than the image transmitted to the terminal last time and transmitting it to the terminal as a relive image. The recording server 100 functions as means (1) to (17) by the hardware configuration and the shooting / editing / transmission program 112 described above.

  In addition, the terminal (mobile terminal 200 and viewing PC 300), (1) a list display means for requesting the camera list to the recording server 100 and displaying the camera list acquired from the recording server 100, and (2) displaying thumbnail images. Camera selection means for selecting the monitoring camera 400 or selecting the monitoring camera 400 for displaying a live image, (3) Image display means for displaying an image transmitted from the recording server 100, and (4) During thumbnail image display. (5) A thumbnail image requesting means for requesting a thumbnail image from the recording server 100 at regular time intervals. (5) During live image display, a live image is recorded at every camera image acquisition interval from the monitoring camera 400 to the recording server 100. Live image request means for requesting the server 100, (6) Receiving an input of a rewind request during live image display, Rewind start request means for requesting the recording server 100 for the live image, and (7) requesting the recording server 100 for the rewinding live image at regular intervals shorter than the request interval of the live image during displaying the rewinding live image. Rewind continuation request means, (8) Rewind live image fast-forward start request means for accepting fast-forward request input and requesting fast-forward live image from recording server 100 during rewind live image display, (9) Fast-forward live image display And a rewind live image fast forward continuation requesting means for requesting the recording server 100 for a fast forward live image every fixed time shorter than the live image request interval, and (10) inputting a relive image request while displaying the rewind live image. Replay image start requesting means for requesting the recording server 100 for a relive image, and (11) while the relive image is being displayed, For each camera image acquisition interval of the recording server 100, Riraibu image continuation request means for requesting Riraibu image server is provided with a city. The terminal functions as means (1) to (11) by the hardware configuration and the image display program 213 described above. In this embodiment, since the primary data created by compressing the camera image data is converted into secondary data by the recording server at regular time intervals, the terminal can rewind or rewind the live image. In addition to fast-forwarding, past camera image playback, rewinding, and fast-forwarding that are possible with conventional recording and playback devices are also possible. For example, when playback is performed on the next day of recording, etc., it is selected what time of day recording is desired to be played, and a file converted into secondary data for one hour is called and played back. Rewind and fast forward during playback are also possible. The live image is a high-quality still image because it uses the camera image input from the camera without compressing it, so the amount of data is small, and the thumbnail image is reduced and compressed to the specified size. The rewind live image and fast forward live image use the primary data compressed for primary still image storage, so the data amount is small while maintaining high image quality. . Since the secondary data, which is a moving image file that has been compressed and converted, is used for playback and fast forward rewinding more than the secondary data creation unit that is not frequently used, the amount of data required for storage can be reduced while moving images. In addition, with the configuration of the present embodiment, it is possible to reproduce the live image after rewinding.

  In the surveillance camera system of this embodiment, a server connected to a camera group that captures a still image via a network receives a camera image from the camera group in a TCP / IP system at regular time intervals in a JPEG format. And is stored as a still image for a certain period, and after a certain period of time, a compression format into a moving image file, for example H.P. Convert to a highly compressed video file in H.264 format and save again. In this embodiment, H. Compressed to H.264 format and converted to a video file. By requesting still image data from the camera group by server-driven type in TCP / IP communication, there is no data loss on communication, and high data integrity can be realized. Conventionally, when data is acquired from a network camera with moving image quality (for example, about 30 fps), a huge amount of data flows on the network through which it was not possible to install a large number of cameras on the same network. In this embodiment, data acquisition is not performed with a moving image, but, for example, one image per second and the image acquisition interval are lengthened, and data is acquired as a still image instead of a moving image. Therefore, the amount of data that flows simultaneously on the network is reduced, and the network camera information can be flowed to a general Internet line or the like. Furthermore, it is possible to store data on a cloud server over the Internet line (WAN), which is impossible due to problems such as packet loss and data transfer amount in the conventional moving image transfer method. In the conventional network camera recording system, the recording conditions on the network camera side are the storage conditions on the recording server side as they are, and recording is performed in the currently mainstream H.264 video format to reduce the video data weight. In order to do this, it is necessary to manufacture a camera having a function for recording in the H.264 format on the network camera side. In this embodiment, however, the server side converts to the H.264 format, It is not necessary to have the function of the video format you want to save. In the case of the conventional system, even in the H.264 format in which the file size is reduced, the amount of data is still large for long-term storage, and the resolution is lowered at the stage of acquiring moving image data (at the stage of recording with a network camera). In addition, there is a problem that the image quality of moving image data becomes coarse by lowering the resolution in order to extend the storage period. Is stored as primary data, that is, high-resolution temporary storage data (data that may be frequently viewed), and after a certain period of time, secondary data, that is, resolution is reduced or image quality is reduced. The storage period can be further extended by converting to temporary storage data (long-term storage data) by reducing the weight. In the present embodiment, for the conversion from data to secondary data, conversion to secondary data is performed while adding image and time stamp information. For example, if the setting is 1 frame per second (1 fps), 3600 images and time stamps are stored in the secondary data. If an image is missing due to some abnormality, the same image as the previous image is used again to adjust the time axis.

{procedure}
Next, a monitoring procedure according to the first embodiment of the surveillance camera system of the present invention will be described. In this procedure, the above-described monitoring camera system is used, but the recording server 100 in FIG. 1 will be described as a server, the monitoring camera 400 as a camera, the mobile terminal 200 and the viewing PC 300 as terminals. Each terminal can simultaneously display images taken by a plurality of cameras via the server.

  In the monitoring method of this embodiment, (1) a server connected to a camera group that captures a still image via a network captures the camera image data obtained by capturing a still image in the camera group at regular intervals. A camera image acquisition step to input to the server, (2) a primary data creation step in which the server creates and stores primary data by compressing the camera image data, and (4) an image connected to the server via a network. A camera selection step in which a terminal to display selects a camera to display a live image; (5) a live image transmission step in which the server transmits camera image data of the selected camera to the terminal as a live image; (6) a live image display step in which the terminal displays a live image transmitted from the server; (7) the terminal A rewind start request step for accepting an input of a rewind request during image display and requesting a rewind live image from the server; (8) a time point past a predetermined time from the image that the server transmitted to the terminal last time; A rewind live image transmission step of extracting primary data and transmitting it to the terminal as a rewind live image; (9) a rewind live image display step in which the terminal displays the rewind live image transmitted from the server; (10) During the rewind live image display, the terminal requests a rewind live image to the server every predetermined time shorter than the live image request interval. (11) The server includes the server For each rewind request, the rewind live image transmission step and (12) rewind live image display step are repeated. (13) The terminal rewinds the live image. A fast-forward start request step for accepting a fast-forward request input and requesting a fast-forward live image from the server during the display, (14) The server extracts primary data at a time point that is a certain time in the future from the image previously transmitted to the terminal A fast-forward live image transmission step for transmitting to the terminal as a fast-forward live image, (15) a fast-forward live image display step for displaying the fast-forward live image transmitted from the server, and (16) the terminal for fast-forwarding. During the live image display, a fast-forward continuation requesting step for requesting the server for a fast-forward live image at regular intervals shorter than the request interval of the live image. (17) The fast-forward live image until the server reaches the current image. For each request, the fast-forward live image transmission step (18) The fast-forward live image display is repeated, (19) A secondary data creation step of converting the primary data into secondary data that is compressed at a predetermined time, combined with a time stamp, and compressed in a moving image format.

  In the monitoring method of the present embodiment, (20) the terminal accepts an input of a relive image request during rewind live image display, and requests a relive image from the server, (21) the server A relive image accepting step for accepting a relive image request from the rewind live image from the terminal; (22) requesting the relive image from the server at every camera image acquisition interval from the camera to the server during the relive image display. A relive image continuation request step; (23) for each relive image request, a relive image transmission step of extracting primary data at a time point that is a certain time in the future from the image previously transmitted to the terminal and transmitting it to the terminal as a relive image. Including.

  A surveillance camera system program having a photographing / editing / transmission program and an image display program causes a computer to execute these steps. This will be described more specifically with reference to the drawings.

(From camera to server, from shooting to image editing)
FIG. 5 is a flowchart showing an outline of a procedure from connection between the camera and the server to image editing in the monitoring camera system according to the first embodiment of the present invention. First, the server requests the camera to connect using the TCP / IP method. A plurality of cameras are configured, and each camera and server authenticates with a user ID and a password. If authentication is successful by referring to the server authentication database, the connection state is established. The server requests the camera to take a still image and send the taken camera image to the server in JPEG format. The camera receives a request for an image, captures a snapshot (still image), and transmits the captured still image as a camera image in the JPEG format to the server using the TCP / IP method.

  In this embodiment, since the camera captures an image with a still image in JPEG format and transmits it to the server, the camera can be used even if it does not have a function of recording in the H.264 format. Further, as will be described later, in this embodiment, since the moving image compression conversion is performed on the server side to the secondary data every predetermined time, it is not necessary for the camera to have a moving image format function. Therefore, the camera may be an analog camera in addition to the IP camera, and an old camera can be used, thereby reducing the cost.

  When the server receives the still camera image, it expands the camera image in the memory. Further, the camera image is compressed to create still data primary data. After receiving the camera image, it sleeps for a certain period of time, and requests the connected camera to send a still image to the server. The sleep time is arbitrarily set in advance, such as 1/10 second, 1 second, 3 seconds, or 60 seconds. In this embodiment, it is 1 second.

  FIG. 6 is a flowchart of image data collection in the first embodiment of the surveillance camera system of the present invention. A more detailed flow of camera image acquisition in the server is shown. The server establishes a connection with the camera using the TCP / IP method, and when the connection is established, obtains a camera image of a still image from the camera at regular time intervals (1 second in this embodiment). If it cannot be obtained, establish a connection again. If it can be obtained, save the camera image to a file. Camera images are acquired until an end signal is received.

  FIG. 7 is a data compression flowchart of Embodiment 1 of the surveillance camera system of the present invention. This is a step of performing moving image compression conversion from primary data to secondary data. When a certain period of time (1 hour in this embodiment) has elapsed since the creation of the primary data, the primary data file is opened in chronological order from the primary image folder in which the primary data has been accumulated, and each of them is for one frame of the moving image. Acquired as a frame image. When a camera image is acquired once per second, 60 × 60 = 3600 files exist in the primary image folder. Each time a frame image is acquired, it is determined whether the weight can be reduced. If the resolution or image quality is higher than a predetermined resolution, the resolution or image quality is reduced. For example, a file having a resolution of 640 × 480 dpi is reduced in weight to 320 × 240 dpi, or the image quality is reduced from 90% to 60%. If the resolution or image quality is less than or equal to a predetermined value, it is left as it is. Next, when a certain period of time, 1 hour in this embodiment, has passed, 1 hour of frame images, ie 3600 in this embodiment, are collected and compressed and converted into secondary data of a moving image. When the end of the file, that is, when 3600 completions are reached, the end of the conversion to the secondary data is waited and the primary data file is closed.

(Until connection of terminal and server, image display)
FIG. 8 is a flowchart showing an outline of the procedure from the connection between the terminal and the server to the image display in the first embodiment of the surveillance camera system of the present invention. First, the terminal receives a connection request to the server from the user, and when the connection request is input, requests the server to connect using the TCP / IP method. In a mobile terminal, a part of the mobile terminal may be connected using a protocol unique to the mobile phone. One or a plurality of terminals are configured, and each terminal and server authenticate with a user ID and a password. If the authentication database of the terminal is not registered by referring to the server authentication database, the encryption key is issued for registration only for the first time, and the user ID and password are issued at the time of registration. When authentication is successful, the connection state is established.

  The terminal receives the camera list request from the user, and transmits the camera list request to the server when the camera list request is input. When the server receives the camera list request, the server confirms authority information about cameras that can be monitored by the requested terminal. The server transmits authorized camera list data to the terminal. The terminal displays the received camera list data on the display, and accepts selection of a camera for displaying a thumbnail image from the user.

  When the selection of the camera to display the thumbnail image is input, the terminal calculates the size of the thumbnail image from the number of selected cameras and the screen size of the display, and specifies the thumbnail image of the selected camera on the server. Request. Upon receiving a request for thumbnail images, the server reduces and compresses the latest camera image acquired from each camera and developed on the memory to a specified size, and transmits the thumbnail image to the terminal as a thumbnail image. The terminal displays the transmitted thumbnail image on the display. Until the end signal is confirmed, the process is repeated at regular intervals, for example, every second.

  FIG. 9 is a live image selection flowchart of the surveillance camera system according to the first embodiment of the present invention. The terminal accepts a selection from the user for a camera that wants to view a live image from the displayed thumbnail images. When the selection of the camera for displaying the reproduction of the live image is input, the terminal requests the live image of the selected camera from the server. When the server receives the request for the live image, the server transmits the latest camera image acquired from the selected camera and developed on the memory as a live image to the terminal. The terminal displays the transmitted live image on the display. The terminal repeats the live image request and subsequent operations at every image acquisition timing of the camera until the user cancels the selection of the camera, or while other than playback is selected.

  FIG. 10 is an explanatory diagram showing various requests from the terminals and responses from the server in the first embodiment of the surveillance camera system of the present invention. The terminal accepts an input of a rewind request from the user while displaying the live image. When a rewind request is input, the terminal requests a rewind image from the server. When the server receives the rewind image request, the server extracts primary data at a certain point in time from a previously transmitted image to the terminal and transmits it to the terminal as a rewind live image. In this embodiment, the time point that is a certain time in the past is set as the time point in the past for 1 second, and the live image is an image acquired every second. Data is sent.

  While the rewind is selected, that is, while the rewind live image is being displayed, the terminal requests the rewind live image from the server at regular intervals. The fixed time at that time is a fixed time shorter than the request interval of the live image, and is 0.2 seconds in this embodiment. Accordingly, the rewind live image created from the primary data of the previous camera image is displayed every 0.2 seconds on the terminal.

  Further, the terminal accepts an input of a fast-forward request from the user during rewind live image display. When a fast forward request is input, the terminal requests a fast forward live image from the server. When the server receives the fast-forward live image request, the server extracts primary data of a certain time in the future from the image previously transmitted to the terminal and transmits it to the terminal as a fast-forward live image. In the present embodiment, since a future time of a certain time is a future time of 1 second and a live image is an image acquired every second, when a fast-forward request is input for the first time, a rewind live image is obtained. From the primary data that is currently displayed, the primary data that is one sheet in the future is transmitted.

  While fast forward is selected, that is, while a fast forward live image is being displayed, the terminal requests a rewind image from the server at regular intervals. The fixed time at that time is a fixed time shorter than the request interval of the live image, and is 0.2 seconds in this embodiment. Therefore, the previous camera image is displayed on the terminal every 0.2 seconds by 1 second. When the fast-forward live image reaches the current image, the live image display is switched to, and the fast-forward is finished.

  Furthermore, the terminal accepts an input of a relive image request from the user while displaying the rewind live image. When the relive request is input, the terminal requests a relive image from the server. When the server receives the request for the relive image, the server extracts primary data at a future time point from the image previously transmitted to the terminal, and transmits it to the terminal as a relive image. In the present embodiment, since a future time of a certain time is a future time of 1 second and a live image is an image acquired every second, when a fast-forward request is input for the first time, a rewind live image is obtained. From the primary data that is currently displayed, the primary data that is one sheet in the future is transmitted.

  While relive is selected, that is, while the relive image is being displayed, the terminal requests the relive image from the server at regular intervals. The fixed time at that time is the same fixed time as the request interval of the live image, and is every second in this embodiment. Therefore, the previous camera image is displayed every second by 1 second on the terminal. If a selection other than relive is made, relive ends.

  In the server, the primary data created by compressing the camera image is converted into secondary data by the recording server every fixed time, in this embodiment every hour. Since it is converted into a moving image with a time stamp and compressed, the capacity required for data storage can be small. For example, when a Quad-VGA image is recorded and saved at 1280 × 960 dpi, the data capacity of one IP camera is 10 MB per hour when acquiring one image per second with a camera suitable for night photography. Since 7.2 GB is required for one month and 86.4 GB is required for one year, even if the data of 40 IP cameras are stored for one year, only 3.456 TB is required. If it is a VGA image, the data capacity of one IP camera when recording and saving at 640 x 480 dpi is 8 MB per hour for 1 month when acquiring one image per second with a camera suitable for night photography. Therefore, even if the data of 40 IP cameras are stored for one year, it is only 2.7648 TB. Therefore, the storage data capacity can be greatly reduced.

{effect}
Since the present embodiment has such a configuration, high-quality image reproduction and live image rewinding can be easily performed, the amount of data is small, and the cost can be reduced.

  Specifically, in the present embodiment, since the still image is transmitted, during playback of the live image being shot, rewind and relive playback from the current image, and fast forward to the current image are immediately performed. Can do. Even if there is a point in time when shooting was not possible for some reason, in this embodiment, the still image at the time of loss is supplemented with a copy of the still image at the previous point in time. Make it possible. In addition, when a problem is found in a live image, it can be immediately rewound and reviewed by relive playback.

  In this embodiment, a still image, not a moving image, is sent from the camera to the recording server, and shooting / still image storage / replayed image transmission is performed under the initiative of the server. Therefore, no load is applied to the communication network, and traffic congestion hardly occurs. Many cameras can be managed by one server so that traffic is always stable. According to the present embodiment, since a camera image is acquired and compressed into a primary data within a certain period of time, the data capacity is small, and the required capacity is small even if there are many cameras and users. The number is small. For example, 50 or more cameras can be connected to one server. In addition, since the primary data is compressed and converted into a moving image and stored together after a predetermined time in the server, the data capacity is small, so that long-term storage is possible. Compared to moving images, the amount of data is much smaller, and the data can be sent out at a higher resolution, and can be reproduced with clear images. By converting to H.264, the storage capacity can be reduced by, for example, 80% or more compared to Motion JPEG and 50% or more compared to MPEG-4. Further, not only an IP camera but also an analog camera can be used by using a converter.

  When playing back on a device such as a smartphone, it takes no time to perform playback and other operations, so it can be played back without stress, and trouble can be found immediately. Even when monitoring multiple cameras, it can be played back easily and smoothly with a smartphone, making it easy to check alongside work while moving from place to place. In this embodiment, the captured still images are sequentially transmitted in real time, and at the time of operations such as rewinding, the still images obtained by compressing the captured still images are sequentially transmitted. Is combined with the use of a moving image that has been created by compressing and converting the data to make the data lighter.

  According to the present embodiment, the terminal that reproduces the image can be carried and real-time reproduction and rewinding can be performed smoothly on the spot, so that verification at the site can be facilitated. Furthermore, according to the present embodiment, both high functionality and low cost can be realized, and power can be suppressed, which is ecological. For example, when shooting with 100 cameras, both the power consumption and the carbon dioxide emission are about one-third of the conventional one. In rewinding, the display interval is shorter than playback and frame skipping is not performed, so there is no omission in the image being rewinded, so check omission can be prevented, and the image is skipped compared to live image playback. There is no sense of incongruity.

  In this embodiment, the images of the surveillance cameras attached to the production lines in the local factories connected by the network are sent to the managers of the headquarters' management department and the smartphones of the managers of the factories for management. It can be used not only for monitoring the operation status but also for cooperation between production lines and factories. There are a wide range of scenes available, which can be used for security and delivery status management in a vast site like a delivery center, check for lost or forgotten deliveries, and entrances and entrances in condominiums such as condominiums Can be checked even if the manager or resident is not in a fixed room such as a management room, and can also be used for security purposes such as private houses, shops, street locations, etc., and for survey purposes such as the number of people passing by . According to this embodiment, when a trouble is detected while checking a monitor remotely on a mobile terminal, it is possible to respond accurately and promptly while rewinding the mobile terminal as soon as it arrives on the spot and arriving at the site and watching the relive image. And

  Even if there are many cameras, they can be installed on all the same networks, and a large number of cameras can be controlled by one server.

  When transmitting data from the camera to the terminal, packet loss is unlikely to occur and the amount of data transfer is small. Therefore, data can be stored in the cloud server over the Internet line (WAN).

  According to this embodiment, a still image is acquired from the camera side and converted into a high-compression video file (converted in H.264 format) on the recording server side. Even if the function is not provided, it is possible to convert on the recording server side. Furthermore, the data at the time of image collection is first saved as high-resolution temporarily stored data as data that may be frequently viewed, and after a certain period of time, the resolution is reduced or the image quality is reduced. By converting to long-term storage data, the storage period can be further extended.

  The present invention is not limited to the above embodiment, and various modifications can be made without departing from the spirit of the invention. In addition, the constituent elements of the above embodiments can be arbitrarily combined without departing from the spirit of the invention.

100 Recording server 101 CPU
102 Memory 103 Display 104 Network Interface 105 Keyboard 106 Mouse 110 Storage Device 111 Primary Image Folder 112 Secondary Image Folder 113 Shooting / Editing / Transmission Program 114 Operating System 200 Mobile Terminal 201 CPU
202 Memory 203 Display 210 Storage device 213 Image display program 214 Operating system 300 Viewing PC
301 CPU
302 Memory 303 Display 305 Keyboard 306 Mouse 310 Magnetic disk 313 Image display program 314 Operating system 400 Surveillance camera 500 Network 600 Router

Claims (1)

A server connected to a camera group that captures a still image via a network causes the server to cause the camera group to capture a still image at regular time intervals, and captures a camera image from the camera group using the TCP / IP method. obtained in still image JPEG format, a predetermined period and stored as a still image, and straight and save into a compressed video file after a certain period of time, said predetermined period are stored in the still image of the live image It has means for accepting the display of the rewind live image from the terminal during display and displaying it on the terminal, and means for accepting the display of the fast-forward live image until the current image is reached during the rewind live image display. Surveillance camera system.

Images