WO2018123078A1 - Monitoring camera system - Google Patents

Abstract

A monitoring camera system comprises a monitoring camera and a server connected to the monitoring camera via a network. The monitoring camera comprises: an image capture unit that captures light-receiving images to generate unprocessed image data; an encode unit that encodes the unprocessed image data into moving image data including bidirectional prediction interframes produced by referring to both past and future frames and then transmits the moving image data to the server; and a live image generation unit that produces real-time image data from the unprocessed image data and transmits the real-time image data to the server.

Description

Surveillance camera system

The present invention relates to a surveillance camera system.

For the purpose of crime prevention, investigation, or management, points to be monitored such as entrances and exits of buildings such as apartment buildings, storefronts, streets, factories, distribution centers, etc. are timely imaged by a surveillance camera, and the camera image is connected to an electric communication line such as the Internet. There is known a surveillance camera system that sends and monitors to a monitor of a user terminal (see, for example, JP-A-2002-77882). In the surveillance camera industry, since it is an essential condition to display real-time video, the analog configuration is still dominant even when the technology of the IP camera (network camera) has been developed.

When using an IP camera in a conventional surveillance camera system, the stream output from the IP camera is H.264. Limited to H.264 baseline profile. This is because the surveillance camera system does not allow a high-profile stream with a delay due to the nature of viewing in real time. In the present specification, “high profile” means moving image data including bi-directional prediction inter frames (also expressed as future prediction frames, B frames, etc.). The high profile is optimized by referring to both the past frame and the future frame.

By the way, in general, when moving image data is stored in a server, there are problems that the data capacity becomes enormous and long-term storage is impossible. In addition, it is necessary to prepare a large number of large-capacity servers, and there is a problem that costs increase.

In order to reduce the volume of video data, conventional surveillance camera systems optimize the baseline profile and reduce the capacity at a variable rate. Further, the applicant of the present application has already proposed a surveillance camera system that converts and saves the video data of the baseline profile into a high profile (Japanese Patent Application No. 2016-077940).

A surveillance camera system according to one aspect of the present disclosure includes:
A surveillance camera,
A server connected to the surveillance camera via a network;
With
The surveillance camera is
An imaging unit that captures the received video and generates raw video data;
An encoding unit that encodes the raw video data into video data including a bi-directional prediction interframe created by referring to both past and future frames, and transmits the encoded video data to the server;
Creating a real-time video data from the raw video data and transmitting it to the server;
Have

FIG. 1 is a diagram illustrating a schematic configuration of a surveillance camera system according to an embodiment. FIG. 2 is a diagram showing a schematic configuration of the surveillance camera in the surveillance camera system shown in FIG. FIG. 3 is a block diagram showing a schematic internal configuration of the encoding unit of the surveillance camera shown in FIG. FIG. 4 is a diagram for explaining output frame delay and rearrangement in the encoding unit shown in FIG. 3. FIG. 5 is a diagram for explaining the principle operation of the encoding unit shown in FIG. FIG. 6 is a block diagram illustrating an example of a hardware configuration of the encoding unit illustrated in FIG. 3. FIG. 7 is a block diagram showing a modification of the hardware configuration of the encoding unit shown in FIG. FIG. 8 is a diagram showing a schematic configuration of a server in the monitoring camera system shown in FIG. FIG. 9 is a diagram showing a schematic configuration of a terminal device (mobile terminal) in the surveillance camera system shown in FIG. FIG. 10 is a diagram showing a schematic configuration of a terminal device (browsing PC) in the surveillance camera system shown in FIG. FIG. 11 is a schematic diagram showing an example of the operation of the surveillance camera system shown in FIG. FIG. 12 is a diagram illustrating a schematic configuration of a monitoring camera according to a first modification of the embodiment. FIG. 13 is a diagram illustrating a schematic configuration of a monitoring camera according to a second modification of the embodiment. FIG. 14 is a diagram illustrating a schematic configuration of a surveillance camera according to a third modification of the embodiment.

In conventional surveillance camera systems, video data is stored in a baseline profile, so the video itself deteriorates due to capacity compression, or the video is distorted by instantaneous movement in the case of variable rate. The limit is coming.

On the other hand, in the configuration in which the video data of the baseline profile is converted to a high profile and saved, the optimized data can be saved, but the conversion is performed using the CPU resource of the server serving as a recorder. Such costs tend to increase.

Also, since the stream output from the camera is limited to the baseline profile, connecting multiple cameras to the network increases the traffic, causing a load on the network design and equipment.

[Embodiment]
The embodiment described below has been made in consideration of the above points, and the purpose thereof is a surveillance camera system capable of acquiring real-time video, and is optimal without imposing a load on the server. Another object of the present invention is to provide a surveillance camera system that can store data in an organized manner.

The surveillance camera system according to the first aspect of the embodiment includes:
A surveillance camera,
A server connected to the surveillance camera via a network;
With
The surveillance camera is
An imaging unit that captures the received video and generates raw video data;
An encoding unit that encodes the raw video data into video data including a bi-directional prediction interframe created by referring to both past and future frames, and transmits the encoded video data to the server;
Creating a real-time video data from the raw video data and transmitting it to the server;
Have

According to such an aspect, the encoding unit of the surveillance camera encodes the raw video data into moving image data including a bi-directional prediction inter frame (B frame) and transmits the encoded video data to the server, which places a load on the server. And optimized data storage is possible. In addition, the amount of data flowing on the network is significantly reduced, and network traffic can be reduced. As a result, the load on the network design and equipment is reduced, and the life of the network equipment can be extended. In addition, a plurality of cameras can be connected to the network.

Further, according to such an aspect, encoding into moving image data including a bi-predictive inter frame (B frame) is not performed using the CPU resource of the server, but is performed inside the surveillance camera. Therefore, the load on the server is reduced and the cost for the system can be reduced. Moreover, since the load on the server is reduced, the number of surveillance cameras that can be accommodated per server can be increased.

In addition, according to such an aspect, the moving image data transmitted from the surveillance camera includes a bidirectional prediction inter frame (B frame), and thus is inevitably delayed, but the live video generation unit performs the delay. Since real-time video data is created from raw video data without data and transmitted to the server, the server can acquire real-time video, and the essential requirement of the surveillance camera system for real-time browsing is lost. It will not be.

A surveillance camera system according to a second aspect of the embodiment is the surveillance camera system according to the first aspect,
The encoding unit transmits the moving image data to the server by a stream method.

A surveillance camera system according to a third aspect of the embodiment is the surveillance camera system according to the first aspect,
The encoding unit creates divided data in order of shooting time from the moving image data in GOP units, stores the divided data in a memory, and transfers the divided data from the memory to the server one by one each time a divided data request is received from the server. And
The server creates moving image data by combining divided data received from the monitoring camera in order of photographing time.

A surveillance camera system according to a fourth aspect of the embodiment is the surveillance camera system according to any one of the first to third aspects,
Each time the live video generation unit receives a live image request from the server, the live video generation unit cuts out real-time still image data from the raw video data and transmits it to the server.

A surveillance camera system according to a fifth aspect of the embodiment is the surveillance camera system according to any one of the first to third aspects,
The live video generation unit encodes the raw video data into real-time video data not including the bidirectional prediction interframe, and transmits the encoded video data to the server in a stream format.

A surveillance camera system according to a sixth aspect of the embodiment is the surveillance camera system according to any one of the first to third aspects,
The live video generation unit repeats cutting out real-time still image data from the raw video data and transmitting it to the server at a predetermined snapshot time interval.

A surveillance camera system according to a seventh aspect of the embodiment is the surveillance camera system according to the fourth or sixth aspect,
The server
A primary video storage unit for storing still image data received from the surveillance camera in a storage device;
A live image transmission unit that transmits the still image data as a live image to a terminal device connected via a network;
When receiving a rewind request from a live image from the terminal device, the terminal device acquires still image data at a time point that is a certain time past from the previously transmitted still image data from the storage device, and the terminal device as a rewind live image. A rewind live image transmitter to transmit to
When a fast-forward request from a rewind live image is received from the terminal device, still image data at a time point that is a certain time later than the previously transmitted still image data is acquired from the storage device until the current still image data is reached. A fast-forward live image transmission unit for transmitting to the terminal device as a fast-forward live image;
Have

According to such an aspect, when the user notices an oversight or the like while displaying a live image on the terminal device, the terminal device transmits a rewind request from the terminal device to the server. It is possible to check back in the past as it is, and by sending a fast-forward request from the terminal device to the server, it is possible to check from the past to the current time by fast-forward.

A surveillance camera system according to an eighth aspect of the embodiment is the surveillance camera system according to any one of the first to seventh aspects,
The server includes a secondary video storage unit that stores the moving image data received from the monitoring camera in a storage device by adjusting the time according to the delay associated with the encoding by the encoding unit.

According to such an aspect, the moving image data transmitted from the surveillance camera is inevitably delayed by encoding in the encoding unit, but is stored in the storage device in a state adjusted in time according to the delay time. Therefore, when the server specifies the time from the terminal device and receives a reproduction request for past moving image data, the server can accurately extract the moving image data at the requested time from the storage device and transmit it to the terminal device. it can.

The surveillance camera according to the ninth aspect of the embodiment is
A surveillance camera connected to a server via a network,
An imaging unit that captures the received video and generates raw video data;
An encoding unit that encodes the raw video data into video data including a bi-directional prediction interframe created by referring to both past and future frames, and transmits the encoded video data to the server;
Creating a real-time video data from the raw video data and transmitting it to the server;
Have

The server according to the tenth aspect of the embodiment is
An imaging unit that captures the received video and generates raw video data;
An encoding unit that encodes the raw video data into video data including a bi-directional prediction interframe created by referring to both past and future frames, and transmits the encoded video data to the server;
Creating a real-time video data from the raw video data and transmitting it to the server;
A server connected to a surveillance camera via a network,
A primary video storage unit that stores real-time video data received from the surveillance camera in a storage device;
A secondary video storage unit that saves the video data received from the monitoring camera in the storage device by adjusting the time according to the delay related to the encoding in the encoding unit;
Is provided.

Hereinafter, specific examples of the embodiment will be described in detail with reference to the accompanying drawings. In addition, in each figure, the component which has an equivalent function is attached | subjected the same code | symbol, and detailed description of the component of the same code | symbol is not repeated.

FIG. 1 is a diagram illustrating a schematic configuration of a surveillance camera system 10 according to an embodiment.

As shown in FIG. 1, the surveillance camera system 10 includes surveillance cameras 400A and 400B, a server 100, and terminal devices 200A, 200B, and 300A to 300C. The monitoring cameras 400A and 400B, the server 100, and the terminal devices 200A, 200B, and 300A to 300C are connected to each other via the network 500.

The network 500 may be either a wired line or a wireless line, and the type and form of the line are not limited. In the present embodiment, the network 500 includes a communication network such as a LAN, the Internet, a Wi-Fi line, or a 3G / LTE line.

In this embodiment, every time the monitoring cameras 400A and 400B accept a live image request from the server 100, the monitoring cameras 400A and 400B create a real-time still image from the camera video and transmit it to the server 100. The server 100 acquires and accumulates still images from the monitoring camera 400, and each time a browsing request is received from the terminal devices 200A, 200B, 300A to 300C, the still image for browsing is stored in the terminal devices 200A, 200B, 300A to 300C. Send. In the illustrated example, the monitoring cameras 400A and 400B exist in the same LAN as the server 100. The communication between the monitoring cameras 400A and 400B and the server 100 is a TCP / IP system, can reliably transmit and receive camera images, and has high reliability.

The terminal devices 200A, 200B, 300A to 300C are used by the user, and are mobile terminals such as smartphones and tablets, notebook computers or desktop computers. The terminal devices 200A, 200B, and 300A to 300C may exist in the same LAN as the server 100. If the terminal devices 200A, 200B, and 300A to 300C are connected to the network 500 via the router 600, they are connected to the server 100 via the Internet. May be.

In the following description, the monitoring cameras 400A and 400B may be collectively expressed by adding a reference numeral 400. In some cases, the terminal devices 200A and 200B as mobile terminals are denoted by reference numeral 200, and the terminal devices 300A to 300C as viewing PCs are denoted by reference numeral 300 and collectively represented.

Next, the configuration of the monitoring camera 400 will be described. FIG. 2 is a diagram illustrating a schematic configuration of the monitoring camera 400.

As shown in FIG. 2, the monitoring camera 400 includes an imaging unit 401, an encoding unit 402, a live video generation unit 403, and a communication control unit such as a network interface 404. Each of these parts 401 to 404 is housed in the same housing 405.

In the illustrated example, the imaging unit 401 includes a lens 4011, a video data receiving unit 4012 that captures an image received through the lens 4011, and raw video data (also expressed as RAW data) based on the imaging result. And a RAW data generation unit 4013 for generating. The configuration of the imaging unit 401 is the same as that of a conventional surveillance camera, and detailed description thereof is omitted.

The encoding unit 402 is connected to the imaging unit 401, and both of the raw video data (RAW data) generated by the imaging unit 401 are created with reference to both past frames and future frames. It encodes into the moving image data containing a direction prediction inter frame (B frame), and transmits to the server 100 via the network interface 404. In the present embodiment, the encoding unit 402 transmits moving image data to the server 100 by a stream method (also referred to as a streaming method).

FIG. 3 is a block diagram showing a schematic internal configuration of the encoding unit 402.

As shown in FIG. 3, the encoding unit 402 includes a control circuit 4021, an intra-frame compression block 4022, an inter-frame compression block 4023, a variable length code compression / frame rearrangement block 4024, and a memory 4025. ing.

The control circuit 4021 controls each block such as setting compression parameters. The intra-frame compression block 4022 performs information compression (data compression within one screen) based on the spatial correlation within the frame. The inter-frame compression block 4023 performs information compression based on temporal correlation with previous and subsequent frames. The variable-length code compression / frame rearrangement block 4024 compresses information by assigning a short code to frequently occurring data (entropy coding) and rearranges the output order of the encoded frame data. The memory 4025 accumulates a plurality of (5 frames in the illustrated example) frame data in order to generate a bidirectional prediction inter frame (B frame).

FIG. 4 is a diagram for explaining output frame delay and rearrangement in the encoding unit 402.

As shown in FIG. 4, the encoding unit 402 includes a plurality of pieces of RAW data between an intra frame (I frame) that can be decoded independently and created at regular intervals, and an intra frame with reference to a past frame. It is encoded into moving image data including a prediction inter frame (P frame) to be created and a bidirectional prediction inter frame (B frame) created by referring to both the past frame and the future frame. Here, a bidirectional prediction inter frame (B frame) cannot be generated unless a prediction inter frame (P frame) is determined. Therefore, a high profile output including a bi-predictive interframe (B frame) has a longer delay time to output and a rearrangement of frames than a baseline output not including a bi-predictive interframe (B frame). is required. Since the delay time is long, the encoding unit 402 is provided with a memory 4025 that holds a frame for that period.

FIG. 5 is a diagram for explaining the principle operation of the encoding unit 402.

As shown in FIG. 5, the encoding unit 402 performs intra-frame compression (spatial information compression) for the first frame data among the sequentially input RAW data (step S41). On the other hand, for the second and subsequent frame data, the encoding unit 402 first decompresses the frames before and after the frame (step S42), and performs inter-frame compression (temporal information compression) with reference to them. After that (step S43), intra-frame compression (spatial information compression) is performed (step S41). Then, the encoding unit 402 performs compression (entropy encoding) and rearrangement on the frame data compressed in the frame based on the appearance frequency of the code and outputs the result (step S44).

FIG. 6 is a block diagram illustrating an example of a hardware configuration of the encoding unit 402.

In the example shown in FIG. 6, the encoding unit 402 includes an encoding LSI 410 and interface blocks 411A and 411B. The encoding LSI 410 is hardware that performs high-profile encoding, and includes a control circuit 4021, an intra-frame compression block 4022, an inter-frame compression block 4023, and a variable-length code compression / frame rearrangement block 4024 (see FIG. 3). Is included. In the illustrated example, the encoding LSI 410 includes a memory 4025 inside. The interface blocks 411A and 411B perform conversion when data input / output and LSI input / output are different.

As the encode LSI 410, specifically, for example, H.S. H.264 translation compatible memory built-in transcoder LSI MB86M0x is used. Conventionally, such an encoding LSI 410 generates a B frame, and therefore inevitably generates a delay. Therefore, in terms of its nature, a stationary device equipped with a digital broadcast recording function such as a television, a hard disk recorder, or a personal computer, or It is used for mobile-related devices for smartphones and tablet products, and as far as the inventors know, there is no example used for surveillance cameras.

In the example shown in FIG. 6, the memory 4025 is built in the encode LSI 410, but the present invention is not limited to this, and the memory 4025 may be externally attached to the encode LSI 410 as shown in FIG.

Further, the hardware configuration of the encoding unit 402 is not limited to an aspect implemented using the encoding LSI 410 as shown in FIGS. 6 and 7, and may be an aspect implemented using a microcomputer and software.

Returning to FIG. 2, the live video generation unit 403 is connected to the imaging unit 401, creates real-time video data from raw video data (RAW data) without delay, and transmits the real-time video data to the server 100. In the present embodiment, every time a live image request is received from the server 100, the live video generation unit 403 creates real-time still image data from raw video data (RAW data) without delay (for example, as it is). The live image is cut out and transmitted to the server 100 via the network interface 404. In this specification, “live image” means one still image representing a real-time camera image.

Next, the configuration of the server 100 will be described. FIG. 8 is a diagram illustrating a schematic configuration of the server 100.

As shown in FIG. 8, the server 100 includes a control / arithmetic unit having a CPU 101 and a device driver with a memory 102 as a cache memory, and a storage device 110 having a main storage device such as a DRAM and an auxiliary storage device such as a hard disk. And a communication control device such as a network interface 104, a display 103 as a display device, and an input device including a keyboard 105, a mouse 106, and the like.

The storage device 110 stores a primary video folder 111, a secondary video folder 112, a program 113, an operating system 114, an authentication database, an environment setting folder (not shown), and the like.

In the primary video folder 111, real-time video data received from the monitoring camera 400 is stored. In the present embodiment, real-time still image data received from the monitoring camera 400 is sequentially stored in the primary video folder 111. In the secondary video folder 112, moving image data received from the monitoring camera 400 is stored as one file for each predetermined storage unit (for example, 10 minutes).

The program 113 is normally stored in the auxiliary storage device of the storage device 110, and is loaded into the main storage device at the time of execution. The authentication database stores IDs, passwords, port numbers and IP addresses of the respective monitoring cameras 400 and terminal devices 200 and 300, and individual identification numbers (UIDs) in the mobile terminal 200 having no IP address. In the environment setting folder, video acquisition timing of each monitoring camera 400, delay time related to encoding by the encoding unit 402 of each monitoring camera 400, and the like are stored in advance.

In this embodiment, since the server and the terminal device are integrated with each other in the present embodiment, the server 100 also serves as a terminal device that displays the camera video by itself, and for maintenance and management, the display 103 as a display device. And a keyboard 105 and a mouse 106 as input devices. If the server 109 does not require display of camera video, the terminal function as a display device may not be provided.

The server 100 implements a computer function that allows the CPU 101 to load the program 113 into the memory 102 and execute it, thereby acquiring the camera video from the monitoring camera 400 and sending the camera video to the terminal devices 200 and 300. To do. The CPU 101 is an arithmetic processing device mounted on a normal computer, executes various programs, and performs various controls.

The server 100 may be a single server or a server group including a plurality of servers. For example, for the secondary video folder 112, the secondary video folder 112 provided in a server different from the server that acquires the video data from the monitoring camera 400 as the storage destination of the video data after a certain period (for example, 24 hours) has elapsed. It is good. By separating the storage destination of past video data that is not frequently played back, the load on the server 100 can be reduced, and a larger number of cameras can be monitored on the same network.

The program 113 causes the computer to realize a secondary video storage function that saves the video data received from the monitoring camera 400 in the secondary video folder 112 by adjusting the time according to the delay related to the encoding in the encoding unit 402. It is a program. In the server 100, the CPU 101 loads the program 113 to the memory 102 and executes it so that the moving image data received from the monitoring camera 400 is time-adjusted according to the delay related to the encoding in the encoding unit 402 and the secondary video. It functions as a secondary video storage unit stored in the folder 112.

The program 113 is a program for causing a computer to realize a primary video storage function for storing still image data received from the surveillance camera 400 in the primary video folder 111, for example, for a predetermined browsing period (for example, 24 hours). is there. In the server 100, a primary video storage unit that stores still image data received from the monitoring camera 400 in the primary video folder 111, for example, for a predetermined viewing period, by the CPU 101 loading and executing the program 113 in the memory 102. Function as. Note that the still image data browsing period (storage period in the primary video folder 111) is not limited to a predetermined mode, and is set as appropriate according to the free capacity of the storage device 110 at that time. May be.

Further, the program 113 has a camera video playback request receiving function for receiving camera video playback requests from the terminal devices 200 and 300 in the computer, and the latest still image data stored in the primary video folder 111 as a live image as a terminal. This is a program for realizing a live image transmission function to be transmitted to the devices 200 and 300. When the CPU 101 receives a camera video playback request from the terminal devices 200 and 300 by the CPU 101 loading and executing the program 113 in the memory 102, the latest still image data stored in the primary video folder 111 is It functions as a live image transmission unit that transmits to the terminal devices 200 and 300 as a live image.

In addition, when the program 113 receives a rewind request receiving function from the terminal devices 200 and 300 for receiving a rewind request from a live image, or when receiving a rewind request from a live image, the program 113 transmits the still image data transmitted last time. A rewind live image transmission function for acquiring still image data of a certain time in the past from the primary video folder 111 and transmitting the still image data to the terminal devices 200 and 300 as a rewind live image. A fast-forwarding request acceptance function that accepts a fast-forward request from a return live image, and when a fast-forward request from a rewind live image is accepted, the still image at a future point in time from the transmitted still image data until the current still image data is reached Acquire image data from the primary video folder 111 as a fast-forward live image Is a program for realizing the fast-forward live image transmission function of transmitting to the terminal device 200 and 300. When the CPU 101 loads the program 113 to the memory 102 and executes it, the server 100 accepts a rewind request from the live image from the terminal devices 200 and 300, and then the past time from the previously transmitted still image data. Still image data from the primary video folder 111 is transmitted from the primary video folder 111 and transmitted to the terminal devices 200 and 300 as a rewind live image. From the terminal devices 200 and 300, the rewind live image is transmitted from the rewind live image. When the fast-forwarding request is received, still image data at a time point that is a certain time later than the previously transmitted still image data is acquired from the primary video folder 111 until the current still image data is reached, and the terminal device 200 as a fast-forwarding live image is acquired. It functions as a fast-forward live image transmission unit that transmits to 300.

In addition, the program 113 extracts a past video reproduction reception function for accepting reproduction of past video data from the terminal devices 200 and 300 to the computer, extracts the received past video data from the secondary video folder 112, and extracts the extracted video data. Is a program for realizing a past moving image transmission function for transmitting to the terminal devices 200 and 300.

Further, the program 113 causes the computer to realize a monitoring camera connection function for connecting to one or a plurality of monitoring cameras 400, and a terminal device connection function for connecting to one or a plurality of terminal devices 200 and 300. It is a program.

In the present embodiment, the connection between the monitoring camera 400 and the server 100 is performed using the TCP / IP method, and authentication is performed using the user ID and password set on the server 100 side in order to identify the monitoring camera 400 to be connected. To request camera image acquisition. The connection between the terminal devices 200 and 300 and the server 100 is also performed by the TCP / IP method, and authentication is performed with a user ID and a password, and it is confirmed that the terminal devices 200 and 300 are terminal devices registered in the server 100. Then send the camera video. The authentication is preferably performed by an authentication database on the server 100.

Even if the terminal devices 200 and 300 and the monitoring camera 400 do not exist in the same LAN, as long as they are connected to the network 500 via the router 600, the IP address of the router 600 to be connected and the port number assigned to each are used. Identification is done. When connected via the Internet, the port number of the router 600 between the two is made “valid”. The mobile terminal 200 having no IP address uses a UID.

In the present embodiment, transmission of moving image data from the monitoring camera 400 to the server 100 is performed by TCP / IP-based RTSP (Real Streaming Protocol). In this embodiment, the moving image data is H.264. H.264 is compressed with high profile and transmitted. A high profile of a next generation codec such as H.265 may be used.

Next, the configuration of the terminal device 200 as a mobile terminal will be described. FIG. 9 is a diagram illustrating a schematic configuration of the terminal device 200.

As illustrated in FIG. 9, the terminal device 200 includes a control / arithmetic apparatus including a CPU 201 with a memory 202 and a device driver, a storage device 210, a communication control device (not shown) that transmits and receives data, a display, and the like. A display 204 as an apparatus and an output device (not shown) such as operation buttons or a touch panel are provided. The storage device 210 stores an image display program 213 and an operating system 214.

The terminal device 200 is, for example, a mobile phone such as a smartphone, and the CPU 201 loads the program 213 into the memory 202 and executes it, thereby realizing a computer function capable of displaying the camera video transmitted from the server 100. . The CPU 201 is an arithmetic processing device mounted on a normal mobile terminal, executes various programs, and performs various controls.

Next, the configuration of the terminal device 300 as a browsing PC will be described. FIG. 10 is a diagram illustrating a schematic configuration of the terminal device 300.

As shown in FIG. 10, a terminal device 300 includes a control / arithmetic unit having a CPU 301 and a device driver with a memory 302, a main storage device such as a DRAM, a storage device 310 having an auxiliary storage device such as a hard disk, and a network. A communication control device such as an interface 304, a display 303 as a display device, and output devices such as a keyboard 305 and a mouse 306 are included. The storage device 310 stores an image display program 313 and an operating system 314.

The terminal device 300 is, for example, a desktop computer, a notebook computer, a tablet terminal, or the like. The CPU 301 loads a program 313 into the memory 302 and executes it, so that the camera image transmitted from the server 100 can be displayed. Realize the function. The CPU 301 is an arithmetic processing device mounted on a normal PC, executes various programs, and performs various controls.

The programs 213 and 313 for displaying images are a server connection function for connecting the server 100 to the computer, and a camera that requests the server 100 to play back the camera video related to one or a plurality of monitoring cameras 400 having the monitoring authority. This is a program for realizing a video playback request function and a camera video display function for displaying a camera video transmitted from the server 100.

The server 100 identifies the connection partner by connecting using the IP address and port number when starting the connection with the monitoring camera 400 in the local environment, and authenticates with the user ID and password. When connecting to the monitoring camera 400 in the remote environment, the server 100 uses the port forwarding (also expressed as port forwarding or the like) of the router 600 to identify the connection partner using the global IP address and the port number. Since the server 100 identifies the terminal device 200 based on the terminal-specific information using the UID at the start of connection with the terminal device 200, the camera image can be displayed by authenticating the user ID and the password and matching the terminal-specific information. And

In the present embodiment, the server 100 and the terminal device 300 as a viewing PC are both configured as personal computers and have a clock function or the like possessed by a normal personal computer. The terminal device 200 as a mobile terminal and the monitoring camera 400 also have a clock function and the like.

Next, an example of the operation of the monitoring camera system 10 having such a configuration will be described with reference to FIG. FIG. 11 is a schematic diagram illustrating an example of the operation of the monitoring camera system 10. In the example illustrated in FIG. 11, the terminal device 200 as a mobile terminal is illustrated, but the operation is the same even if the terminal device 300 is replaced with a browsing PC.

Referring to FIG. 11, first, the server 100 requests the monitoring camera 400 to connect using the TCP / IP method. The monitoring camera 400 includes one or more monitoring cameras, and the monitoring camera 400 and the server 100 authenticate with a user ID and a password. When the authentication database of the server 100 is referred to and the authentication is successful, the monitoring camera 400 and the server 100 are connected.

Next, the server 100 requests the monitoring camera 400 to capture and transmit the camera video. When the monitoring camera 400 receives a video request from the server 100, it continuously captures the camera video, and converts the raw video data (RAW data) obtained by the imaging to H.264. H.264 high profile video data is encoded. More specifically, the surveillance camera 400 creates a plurality of RAW data between an intra frame (I frame) that can be decoded independently and created at regular intervals, and an intra frame with reference to a past frame. Encoding is performed into moving image data including a prediction inter frame (P frame) and a bidirectional prediction inter frame (B frame) created by referring to both past and future frames. Then, the monitoring camera 400 transmits the encoded moving image data to the server 100 by a stream method.

In addition, the server 100 repeatedly transmits a live image request to the monitoring camera 400 at a predetermined snapshot time interval. The snapshot time interval is arbitrarily set in advance, and is, for example, 1/10 second, 1 second, 3 seconds or 60 seconds. In the embodiment described below, the snapshot time interval is 1 second. Each time the monitoring camera 400 receives a live image request from the server 100, the monitoring camera 400 creates (cuts out) real-time still image data from raw video data (RAW data) and transmits it to the server 100 as a live image.

When the server 100 acquires the moving image data from the monitoring camera 400 by the stream method, the server 100 stores it in the secondary video folder 112 as one file for each storage unit (for example, 10 minutes). Thereby, the reproduction of the past moving image data can be accepted after the moving image data is stored (for example, after 10 minutes).

In the present embodiment, the server 100 saves the moving image data received from the monitoring camera 400 in the secondary video folder 112 by adjusting the time according to the delay related to the encoding in the encoding unit 402. For example, when the moving image data transmitted from the monitoring camera 400 is delayed by one minute with respect to the real-time still image data transmitted simultaneously with the moving image data, the server 100, for example, 9: 51: 00-10 The moving image data received at 0:00:59 is saved in the secondary video folder 112 as one file with a time stamp of 9:55:00 to 9:59:59.

In addition, when the server 100 acquires still image data as a live image from the monitoring camera 400, the primary video folder 111 is displayed as high-resolution data that may be frequently viewed during a predetermined browsing period (for example, 24 hours). When the viewing period is over, the primary video folder 111 is deleted.

The terminal device 200 accepts a connection request to the server 100 from the user. When a connection request to the server 100 is input to the terminal device 200, the terminal device 200 requests the server 100 to connect using the TCP / IP method. In the mobile terminal 200, connection may be made partially using a protocol unique to the mobile phone. The terminal device 200 includes one or a plurality of terminal devices, and the terminal device 200 and the server 100 authenticate with a user ID and a password. When the authentication database of the server 100 is referred to and the terminal device 200 is not registered for authentication, registration is performed by issuing an encryption key only for the first time, and a user ID and password are issued during registration. If the authentication is successful, the terminal device 200 and the server 100 are connected.

The terminal device 200 receives a camera list request from the user. When the camera list request is input, the terminal device 200 transmits the camera list request to the server 100. When the server 100 receives the camera list request, the server 100 checks authority information about the monitoring camera 400 that can be monitored by the requested terminal device 200. The server 100 transmits authorized camera list data to the terminal device 200. The terminal device 200 displays the received camera list data on the display 203 and accepts a selection from the user regarding the monitoring camera to be played live.

When the terminal device 200 receives the selection of the monitoring camera from the user, the terminal device 200 requests the server 100 to perform live playback of the camera video for the selected monitoring camera. Upon receiving a live playback request, the server 100 transmits the latest still image data acquired from the selected monitoring camera 400 and stored in the primary video folder 111 to the terminal device 200 as a live image.

The terminal device 200 displays the still image data received from the server 100 on the display 203 as a live image. The terminal device 200 repeats the request for live playback at the same time interval as the snapshot time interval in the monitoring camera 400 until the user deselects the monitoring camera or while other than live playback is selected. .

The terminal device 200 receives an input of a rewind request from the user while the live image is displayed. When a rewind request is input from the user, the terminal device 200 requests a rewind image from the server 100.

When the server 100 receives the rewind image request, the server 100 extracts still image data at a point in time that is a predetermined time earlier than the previously transmitted still image data from the primary video folder 111 and transmits it to the terminal device 200 as a rewind live image. To do. In this embodiment, since the past time for a certain time is set to the past time for one second, and the live image is still image data acquired every second, when one rewind request is input for the first time, The previous still image data is transmitted to the terminal device 200.

While the rewind is selected, that is, while the rewind live image is being displayed, the terminal device 200 requests the rewind live image from the server 100 at regular intervals. The fixed time at that time is a fixed time shorter than the request interval of the live image, for example, every 0.2 seconds. Therefore. On the terminal device 200, a rewind live image created from still image data of the previous camera image by 1 second is displayed every 0.2 seconds.

Furthermore, the terminal device 200 accepts an input of a fast forward request from the user while displaying the rewind live image. When the fast forward request is input, the terminal device 200 requests the server 100 for a fast forward live image. When the server 100 receives the fast-forward live image request, the server 100 extracts still image data at a time point that is a certain time later than the previously transmitted still image data from the primary video folder 111 and transmits it to the terminal device 200 as a fast-forward live image. . In the present embodiment, the future time is set to the future time of 1 second, and the live image is an image acquired every second. Therefore, when a fast-forward request is input for the first time, the rewind live image From the still image data that is currently displayed, still image data that is one second in the future is transmitted.

While the fast forward is selected, that is, while the fast forward live image is being displayed, the terminal device 200 requests the server 100 for a fast forward live image at regular intervals. The fixed time at that time is a fixed time shorter than the request interval of the live image, for example, every 0.2 seconds. Accordingly, the fast-forward live image created from the still image data of the camera video after 1 second is displayed on the terminal device 200 every 0.2 seconds. When the fast-forward live image reaches the current image, the live image display is switched to, and the fast-forward is finished.

Instead of live playback, past video data can be played in the same way. The server 100 extracts the received past moving image data from the secondary video folder 112 instead of the still image data, and transmits the extracted moving image data to the terminal device 200 for display.

The still image data is an image that is light enough to be continuously transmitted to the terminal device 200, and the moving image data is encoded with a high profile and the amount of data is extremely small while being a moving image. Light enough to send to. In addition, both still image data and moving image data can be rewinded, played back after rewinding, and fast-forwarded, which is highly convenient.

By the way, as mentioned above, in the conventional surveillance camera system, the video data is stored in the baseline profile, so the video deteriorates due to the compression of the capacity, or in the case of the variable rate, the video is instantaneously moved. There was a limit to the mechanism itself, such as being disturbed.

In addition, in the configuration in which the video data of the baseline profile is converted to a high profile and saved, the optimized data can be saved, but the conversion is performed using the CPU resource of the server serving as a recorder. There was a tendency for such costs to increase.

Also, since the stream output from the camera is limited to the baseline profile, connecting multiple cameras to the network increased the traffic, causing a load on the network design and equipment.

On the other hand, according to the present embodiment, the encoding unit 402 of the monitoring camera 400 encodes raw video data into moving image data including a bidirectional prediction inter frame (B frame) and transmits the encoded video data to the server. Optimized data storage is possible without imposing a load on the server. In addition, the amount of data flowing on the network 500 is significantly reduced, and network traffic can be reduced. As a result, the load on the network design and equipment is reduced, and the life of the network equipment can be extended. In addition, a plurality of surveillance cameras 400 can be connected to the network 500.

Further, according to the present embodiment, encoding into moving image data including a bi-predictive inter frame (B frame) is not performed using the CPU resource of the server 100, but inside the surveillance camera 400. As a result, the load on the server 100 is reduced and the cost of the system can be reduced. Further, since the load on the server 100 is reduced, the number of monitoring cameras 400 per server can be increased.

In addition, according to the present embodiment, the moving image data transmitted from the monitoring camera 400 includes a bidirectional prediction inter frame (B frame), and thus the live video generation unit 403 inevitably delays. Since the real-time video data is generated from the raw video data without delay and transmitted to the server 100, the server 100 can acquire the real-time video, and is essential as a surveillance camera system for browsing in real time. There is no loss of requirements.

In addition, according to the present embodiment, the moving image data transmitted from the monitoring camera 400 is necessarily delayed due to encoding by the encoding unit 402, but the secondary video folder 112 is time-adjusted according to the delay. Therefore, when the server 100 specifies a time from the terminal device 200 and receives a playback request for past video data, the server 100 accurately stores the video data at the requested time from the secondary video folder 112. And transmitted to the terminal device 200.

In addition, according to the present embodiment, when a user notices an oversight or the like while displaying a live image on the terminal device 200, a rewind request is transmitted from the terminal device 200 to the server 100. Thus, the terminal device 200 can check back as it is, and by transmitting a fast-forward request from the terminal device 200 to the server 100, it is possible to confirm from the past to the present time by fast-forward.

Note that various modifications can be made to the above-described embodiment. Hereinafter, an example of modification will be described with reference to the drawings. In the following description and the drawings used in the following description, for parts that can be configured in the same manner as the above-described embodiment, the same reference numerals as those used for the corresponding parts in the above-described embodiment are used, and overlapping Description to be omitted is omitted.

(First modification)
A first modification will be described with reference to FIG. FIG. 12 is a diagram illustrating a schematic configuration of a monitoring camera 400 according to the first modification.

As shown in FIG. 12, in the first modified example, the encoding unit 402B of the monitoring camera 400 converts raw video data (RAW data) generated by the imaging unit 401 into moving image data including B frames (for example, H.264 high profile video data), and then, in the GOP unit (for example, 1 second) in which the I frame is the first image from the video data, divided data is created in the order of shooting time and stored in the memory 4025. save.

Note that GOP (Group Of Picture) is a frame group including one or more I frames (intra frames), a plurality of P frames (prediction interframes), and B frames (bidirectional prediction interframes). For example, when detecting an I frame from the moving image data, the encoding unit 402 sets one frame of data up to the frame immediately before the next I frame is detected. In this case, one divided data includes one I frame. One divided data may include two or more I frames.

Moreover, every time the encoding unit 402B of the monitoring camera 400 receives a divided data request from the server 100, the encoded data is transferred from the memory 4025 to the server 100 one by one.

On the other hand, the server 100 repeatedly transmits the divided data request to the monitoring camera 400 and receives the divided data. Then, the server 100 combines the divided data received from the monitoring camera 400 in the order of shooting time, creates one file of moving image data for each storage unit (for example, 10 minutes), and stores it in the secondary video folder 112.

According to the first modification as described above, when the server 100 is installed in a cloud environment and video data is transferred from the remote monitoring camera 400 to the server 100, the original video can be restored later. Since it can be divided into smaller units and transmitted while avoiding congestion of the network 500, it is possible to collect camera images stably in a cloud environment.

(Second modification)
A second modification will be described with reference to FIG. FIG. 13 is a diagram illustrating a schematic configuration of a monitoring camera 400 according to the second modification.

As shown in FIG. 13, in the second modified example, the live video generation unit 403B of the monitoring camera 400 converts the raw video data (RAW data) into real-time data that does not include a bidirectional prediction interframe (B frame). From moving image data, that is, an intra frame (I frame) that can be decoded independently and created at regular intervals, and a plurality of predicted inter frames (P frames) created between intra frames with reference to past frames The encoded video data of the baseline profile is encoded and transmitted to the server 100 in a stream format.

Also according to the second modified example, as in the above-described embodiment, the encoding unit 402 of the surveillance camera 400 encodes raw video data into moving image data including a bi-directional prediction interframe (B frame). Because the data is transmitted to 100, optimized data storage can be performed without imposing a load on the server 100, and the live video generation unit 403 creates real-time video data from unprocessed video data without delay, and the server Therefore, the server 100 can acquire a real-time video, and the essential requirement as a surveillance camera system for real-time browsing is not impaired.

According to the second modified example, since the monitoring camera 400 transmits real-time video in a stream format, traffic on the network 500 may increase as compared with the above-described embodiment, but it may be in a local environment. Will not be a problem.

(Third Modification)
A third modification will be described with reference to FIG. FIG. 14 is a diagram illustrating a schematic configuration of a monitoring camera 400 according to the third modification.

As shown in FIG. 14, in the third modified example, the live video generation unit 403C of the monitoring camera 400 performs unprocessed video at a snapshot time interval (for example, every second) set in advance by the timer 409 or the like. It repeats that real-time still image data is cut out from data (RAW data) and transmitted to the server 100. In the transfer of still image data from the monitoring camera 400 to the server 100, a TCP / IP-based transfer method such as FTP or HTTP is used.

Also according to the third modified example, as in the above-described embodiment, the encoding unit 402 of the surveillance camera 400 encodes raw video data into moving image data including a bi-directional prediction interframe (B frame). Because the data is transmitted to 100, optimized data storage can be performed without imposing a load on the server 100, and the live video generation unit 403 creates real-time video data from unprocessed video data without delay, and the server Therefore, the server 100 can acquire a real-time video, and the essential requirement as a surveillance camera system for real-time browsing is not impaired.

The description of the above-described embodiments and individual modifications and the disclosure of the drawings are merely examples for explaining the invention described in the claims, and the description of the above-described embodiments and individual modifications. The invention described in the scope of claims is not limited by the description or the disclosure of the drawings. The components of the above-described embodiments and individual modifications can be arbitrarily combined without departing from the gist of the invention.

Claims (10)

1. A surveillance camera,
   A server connected to the surveillance camera via a network;
   With
   The surveillance camera is
   An imaging unit that captures the received video and generates raw video data;
   An encoding unit that encodes the raw video data into video data including a bi-directional prediction interframe created by referring to both past and future frames, and transmits the encoded video data to the server;
   Creating a real-time video data from the raw video data and transmitting it to the server;
   Having
   A surveillance camera system characterized by that.
2. The surveillance camera system according to claim 1, wherein the encoding unit transmits the moving image data to the server by a stream method.
3. The encoding unit creates divided data in order of shooting time from the moving image data in GOP units, stores the divided data in a memory, and transfers the divided data from the memory to the server one by one each time a divided data request is received from the server. And
   The surveillance server system according to claim 1, wherein the server creates moving image data by combining divided data received from the surveillance camera in order of photographing time.
4. The live video generation unit cuts out real-time still image data from the raw video data and transmits the live video request to the server every time a live image request is received from the server. Surveillance camera system according to Crab.
5. The live video generation unit encodes the raw video data into real-time video data not including the bidirectional prediction interframe, and transmits the encoded video data to the server in a stream format. 4. The surveillance camera system according to any one of 3.
6. The live video generation unit repeats cutting out real-time still image data from the raw video data and transmitting it to the server at predetermined snapshot time intervals. The surveillance camera system according to any one of the above.
7. The server
   A primary video storage unit for storing still image data received from the surveillance camera in a storage device;
   A live image transmission unit that transmits the still image data as a live image to a terminal device connected via a network;
   When receiving a rewind request from a live image from the terminal device, the terminal device acquires still image data at a time point that is a certain time past from the previously transmitted still image data from the storage device, and the terminal device as a rewind live image. A rewind live image transmitter to transmit to
   When a fast-forward request from a rewind live image is received from the terminal device, still image data at a time point that is a certain time later than the previously transmitted still image data is acquired from the storage device until the current still image data is reached. A fast-forward live image transmission unit for transmitting to the terminal device as a fast-forward live image;
   Having
   The surveillance camera system according to claim 4 or 6, wherein
8. The server includes a secondary video storage unit that adjusts time according to a delay related to encoding in the encoding unit and stores the video data received from the monitoring camera in a storage device.
   The surveillance camera system according to any one of claims 1 to 7, characterized in that:
9. A surveillance camera connected to a server via a network,
   An imaging unit that captures the received video and generates raw video data;
   An encoding unit that encodes the raw video data into video data including a bi-directional prediction interframe created by referring to both past and future frames, and transmits the encoded video data to the server;
   Creating a real-time video data from the raw video data and transmitting it to the server;
   Having
   A surveillance camera characterized by that.
10. An imaging unit that captures the received video and generates raw video data;
    An encoding unit that encodes the raw video data into video data including a bi-directional prediction interframe created by referring to both past and future frames, and transmits the encoded video data to the server;
    Creating a real-time video data from the raw video data and transmitting it to the server;
    A server connected to a surveillance camera via a network,
    A primary video storage unit that stores real-time video data received from the surveillance camera in a storage device;
    A secondary video storage unit that saves the video data received from the monitoring camera in the storage device by adjusting the time according to the delay related to the encoding in the encoding unit;
    A server characterized by comprising:

Images