CN210274298U

CN210274298U - Robot body security system based on camera array image analysis

Info

Publication number: CN210274298U
Application number: CN201920603452.7U
Authority: CN
Inventors: 张俊业; 范芸; 朱维和
Original assignee: First Research Institute of Ministry of Public Security
Current assignee: First Research Institute of Ministry of Public Security
Priority date: 2019-04-28
Filing date: 2019-04-28
Publication date: 2020-04-07
Anticipated expiration: 2029-04-28

Abstract

The utility model discloses a robot body security system based on camera array image analysis, which comprises a camera array, a video image analysis device, a cache memory, a nonvolatile memory, a safety module and a wireless communication module; the camera shooting array comprises a plurality of camera shooting modules or cameras, and the camera shooting modules or the cameras are circumferentially distributed on the surface of the robot body around the robot body; the camera array, the cache memory, the nonvolatile memory, the safety module and the wireless communication module are all in communication connection with the video image analysis equipment. The utility model discloses but the outside personnel of automatic identification robot body or article are to the attack or the injury action of self to in time report to the police and the record when discovering these actions, guide the police officer to stop the further development of injury, and provide the evidence in order to carry out corresponding punishment to the agent.

Description

Robot body security system based on camera array image analysis

Technical Field

The utility model relates to a security protection equipment technical field, concretely relates to robot body security protection system based on camera array image analysis.

Background

In recent years, under the support of a series of policies and the pull of market demands, the robot industry in China is rapidly developed and becomes the first big market of the world robots in 2013. The quantity of the autonomous brand industrial robots in 2014 reaches 1.7 thousands, and is increased by 78% compared with the last year. The service robot has developed a series of representative products in the fields of scientific investigation, medical rehabilitation, education and entertainment, family service and the like and realizes large-scale application.

The police robot is a multifunctional comprehensive system which comprehensively utilizes technologies such as Internet of things, artificial intelligence, cloud computing and big data, integrates environment sensing, route planning, dynamic decision, behavior control and an alarm device, has the capabilities of autonomous sensing, autonomous walking, autonomous protection, interactive communication and the like, and can help police to complete basic, repetitive and dangerous work. With the continuous aggravation of the aging of the population in China and the rapid development of the public security business, the problems of insufficient public security police strength, potential safety hazards existing in manual patrol and the like are increasingly highlighted, and an intelligent police robot capable of assisting or replacing a police officer to execute a specific task is urgently needed to promote the rapid upgrade of the public security business, reduce the labor intensity and the on-duty risk of the public security police officer and promote the change of the on-duty mode of the police officer. At present, various security patrol robots and security service robots are introduced in the market to assist the police in performing tasks.

The police robot is mainly used for preventing crimes and protecting people, and the existing robot in the market has very weak protection capability to the robot, so that the police robot cannot be protected by police officers or auxiliary personnel during working, the labor intensity and risks of the police officers cannot be reduced, and the large-scale popularization and application of the police robot are limited.

At present, the safety protection of the robot body is completely carried out manually, and the following two modes are mainly adopted:

the first is that the worker comes near the robot, and when finding that another person invades the robot, the worker stops the robot.

The second method is to install a camera in the robot body, shoot and record the surrounding video images, and transmit the shot video images to a monitoring center through a communication link (WLAN, 4G/5G, etc.), and a monitoring person in the monitoring center judges whether a person infringes the robot body by monitoring the video images. When the method is used, intelligent video image analysis equipment is sometimes installed in a monitoring center to reduce the labor intensity of monitoring personnel.

In the system of the first mode, the robot must be accompanied by staff in the vicinity when working, and when the range of motion of the robot is large, at least 1 staff must be provided for each robot to protect the robot. In consideration of the physiological characteristics of people, 2-3 people need to be equipped in most cases. Although the safety of the robot body can be ensured in this way, the robot cannot perform the function of replacing the human work or reducing the labor intensity of the human work, and the like, and has no use value.

When the second method is adopted, workers do not need to move along with the robots, one monitoring person can monitor videos sent back by the robots at the same time, part of manual labor can be saved for a lot of time, and especially when video image intelligent analysis equipment (server) capable of automatically alarming attack actions is installed in a monitoring center, the saving effect is more obvious.

However, such systems also have significant disadvantages that are difficult to overcome:

1. no dead angle monitoring around the robot body is possible.

The cameras installed on most robots are mainly used for carrying out video monitoring on the surrounding environment, but not for protecting the body, so that the purpose of monitoring the periphery of the body by 360 degrees without dead angles cannot be achieved. Even though some robots are equipped with a plurality of cameras and can shoot and record images at 360 degrees, all the video images cannot be transmitted back to the monitoring center due to the limitation of communication link capacity and cost, so that monitoring personnel in the monitoring center cannot monitor the periphery of the robot body without dead angles.

2. Communication links are costly and availability is difficult to guarantee.

One path of 1080P @25fps high-definition video compressed by an H.264 algorithm has an average code rate of over 4Mbps (considering that the actual code rate is much higher than the value when a camera moves along with a robot), and the peak code rate can reach tens of megabits. Estimated at an average code rate of 4Mbps, the amount of data transmitted per hour is:

4M*3600/8＝1.8Gbytes

the data volume per day was: 1.8x24 ═ 43.2 Gbytes.

Such long, uninterrupted bandwidth requirements, whether WLAN or 4G, are difficult to guarantee. Even after 5G business, although the bandwidth is improved much compared with the current 4G, the cost burden caused by the high data volume is hard to bear by the general users.

3. The attack behavior is difficult to accurately judge by the monitoring personnel.

The video images transmitted back to the monitoring center are restored after the processes of compression, wireless transmission, decompression and the like, the image quality of the video images is lost compared with that acquired by a camera, and factors such as time delay jitter, error codes, frame loss and the like generated in the transmission process make monitoring personnel and intelligent analysis equipment difficult to accurately judge the character behaviors appearing in the monitored video, so that the damage activities to the robot body cannot be found in time.

4. The recorded video images are difficult to use as evidence for the disposal of offenders.

Even if the monitoring personnel timely discover the invasion behavior to the robot body and the robot or the monitoring center records the video of the invasion behavior process of the invader, the video is not authenticated, so that when the invader is treated based on the video, the invader is easily repudiated because the video is tampered, and the evidence of disposing the invader is difficult to be used.

SUMMERY OF THE UTILITY MODEL

The utility model aims at providing a robot security protection system based on camera array image analysis, but the outside personnel of automatic identification robot or article are to the attack or the injury action of self to in time report to the police and the record when discovering these actions, guide the police officer to stop the further development of injury, and provide the evidence and carry out corresponding punishment with the agent.

In order to achieve the above purpose, the utility model adopts the following technical scheme:

a robot body security system based on camera array image analysis comprises a camera array, video image analysis equipment, a cache memory, a nonvolatile memory, a safety module and a wireless communication module; the camera shooting array comprises a plurality of camera shooting modules or cameras, and the camera shooting modules or the cameras are arranged around the robot body and are circumferentially distributed on the surface of the robot body; the camera array, the cache memory, the nonvolatile memory, the security module and the wireless communication module are all in communication connection with the video image analysis equipment;

the cache memory is used for temporarily storing video images shot by the camera array; the video image analysis equipment is used for analyzing the video images shot by the camera array, judging whether the robot body is invaded or not, extracting the video images related to the invasion from the cache memory and transmitting the video images to the safety module; the security module is used for carrying out digital signature operation on video images related to the occurrence of the infringement and storing the video images and corresponding digital signature data in a nonvolatile memory; the nonvolatile memory is used for storing video images related to the invasion and digital signature data corresponding to the video images; the wireless communication module is used for establishing a data transmission channel between the robot body and the robot monitoring center.

Further, the shooting ranges of adjacent camera modules or cameras are partially overlapped.

Further, the nonvolatile memory is a solid state disk or a mechanical hard disk.

Further, the wireless communication module includes one or more combinations of a WiFi wireless communication module, a WLAN wireless communication module, a 3G wireless communication module, a 4G wireless communication module, and a 5G wireless communication module.

Furthermore, the robot monitoring center is provided with a monitoring terminal, an information memory, a certificate management and trusted verification server and a wireless communication module, wherein the information memory, the certificate management and trusted verification server and the wireless communication module are all in communication connection with the monitoring terminal.

The beneficial effects of the utility model reside in that:

the utility model carries out 360-degree dead-angle-free monitoring on the periphery of the robot body by installing a camera array consisting of a plurality of camera modules or cameras on the robot body; installing a cache memory to store videos shot by all cameras for a short time; installing video image analysis equipment to analyze and process video images shot by all cameras so as to judge whether acts of infringing the robot body occur or not; when the infringement behavior is judged, the corresponding video clip temporarily stored in the cache memory can be taken out, digital signature and the like are carried out through the security module, the video clip and the corresponding signature data are stored in the nonvolatile memory, and meanwhile, the video clip and the corresponding signature data are transmitted to the robot monitoring center through the communication link.

The utility model discloses can effectively solve that prior art exists can't carry out around the robot no dead angle and keep watch on, communication link expense is high and the usability is difficult to guarantee, the control personnel is difficult to accurate judgement, the video image of record is difficult to as the evidence scheduling problem of dealing with the invader to the infringement action, improves the suitability of robot.

Drawings

Fig. 1 is a schematic diagram of a system structure according to embodiment 1 of the present invention;

fig. 2 is a schematic flow chart of the method according to embodiment 2 of the present invention.

Detailed Description

The present invention will be further described with reference to the accompanying drawings, and it should be noted that the present embodiment is based on the technical solution, and the detailed embodiments and the specific operation processes are provided, but the protection scope of the present invention is not limited to the present embodiment.

Example 1

The embodiment provides a robot body security system based on camera array image analysis, as shown in fig. 1, comprising a camera array, a video image analysis device, a cache memory, a nonvolatile memory, a security module, and a wireless communication module; the camera shooting array comprises a plurality of camera shooting modules or cameras, and the camera shooting modules or the cameras are arranged around the robot body and are circumferentially distributed on the surface of the robot body; the camera array, the cache memory, the nonvolatile memory, the security module and the wireless communication module are all in communication connection with the video image analysis equipment;

the cache memory is used for temporarily (within minutes or hours) storing video images shot by the camera array; the video image analysis equipment is used for analyzing the video images shot by the camera array, judging whether the robot body is invaded or not, extracting the video images related to the invasion from the cache memory and transmitting the video images to the safety module; the security module is used for carrying out digital signature operation on video images related to the occurrence of the infringement and storing the video images and corresponding digital signature data in a nonvolatile memory; the non-volatile memory is used for storing (for a long period of time, such as days, weeks or months) video images related to the occurrence of the invasion and digital signature data corresponding to the video images; the wireless communication module is used for establishing a data transmission channel between the robot body and the robot monitoring center, and the video image analysis equipment sends video images related to the occurrence of the invasion and digital signature data corresponding to the video images to the robot monitoring center through the wireless communication module.

It should be noted that the video image analysis device mainly determines whether there is an infringement on the robot body by the existing methods of face detection, face snapshot, human shape detection, border crossing alarm, and the like.

For example, when a face area of a person detected in a certain screen exceeds a certain proportion of the entire field of view (screen), it is indicated that the person is too close to the robot body, and it is possible that an attempt is made to perform an attack or an attack is being performed.

When the camera uses a wide-angle lens, the human detection function can be used to determine an infringement event: when a person whose human-shaped area exceeds a certain proportion of the entire field of view (screen) is detected in a certain screen, it indicates that the person is too close to the robot body, and it is likely that an intrusion is intended to be performed or is being performed.

Further, the shooting ranges of adjacent camera modules or cameras are partially overlapped. This setting can further guarantee that whole array of making a video recording can carry out the control at 360 degrees no dead angles around the robot.

Further, the cache memory may be a memory bank or a memory card installed in the video image analysis device, or may be a separate storage device. When full, subsequent video images overwrite previously stored video images.

Further, the non-volatile memory may be a solid state disk or a mechanical hard disk, or other types of memory.

Further, the security module stores a private key certificate, which can generate a public key/private key pair and a random number for securely storing the private key, completing digital signature and encryption/decryption operations. The process of carrying out digital signature operation on the video image related to the occurrence of the invasion by utilizing the security module adopts a trusted data calculation process disclosed in Chinese patent application ZL200910080677. X:

(1) firstly, HASH operation is carried out on video frame data (such as algorithms of MD5, SHA-1, SHA-256 and the like) to generate a video frame data abstract HD (abbreviation of Hashed data, video frame data abstract).

(2) The HD is Signed with a private key stored in the security module (e.g., using algorithms such as RSA1024, RSA2048, etc.), and a video frame signature Data digest SHD (abbreviation for Signed Hashed Data, video frame signature Data digest) is generated.

Trusted Data TED (abbreviation for Trust Encoded Data) is assembled according to the format shown in Table 1

TABLE 1

The device ID is a device identification code of a security module which is registered on a certificate management and trusted verification server of the robot monitoring center, each device ID is unique, and a public key certificate corresponding to the device ID can be found on the certificate management and trusted verification server through the device ID. The corresponding frame number is the sequence number of the video frame data corresponding to the trusted data, and each video frame data has a unique sequence number in the same video image data (a file or a video stream). The digest algorithm and the signature algorithm are determined in tables 2 and 3, respectively, and indicate the algorithm used for the operation.

TABLE 2

Summary algorithm value	Summarization algorithm used
		00	SHA-I
01	SHA-256
		10	MD5
11	Retention

TABLE 3

Signature algorithm value	Summarization algorithm used
		00	RSA1024
01	RSA2048
		10	Retention
11	Retention

The encoding frame information indicates the type of the original video data and the mapping relation between the credible data and the original video data. Except signing every video frame digital abstract, the utility model discloses still support to carry out HASH operation and signature (being called for short "allies oneself with and signs" for a short time ") jointly to a plurality of video frames. The method of the joint signing is used for some non-important frames, the operation amount can be greatly reduced on the premise of not obviously reducing the safety, and therefore the real-time performance of the trusted computing is improved, and the power consumption of the whole machine is reduced. See table 4.

TABLE 4

Further, the wireless communication module may be one or a combination of a WiFi wireless communication module, a WLAN wireless communication module, a 3G wireless communication module, a 4G wireless communication module, and a 5G wireless communication module, and is configured to establish a data transmission channel (i.e., a communication link) between the robot body and the robot monitoring center (through a corresponding communication network).

Furthermore, the robot monitoring center is provided with a monitoring terminal, an information memory, a certificate management and trusted verification server and a wireless communication module, wherein the information memory, the certificate management and trusted verification server and the wireless communication module are all in communication connection with the monitoring terminal; the information memory is used for storing alarm information which is sent by the robot body and comprises video images related to the invasion and digital signature data corresponding to the video images; the certificate management and trusted verification server is used for performing public key certificate management and trusted verification on the video image; the monitoring terminal is used for monitoring, storing and replaying the video image sent by the robot body in real time, controlling the certificate management and the credible verification server to carry out credible verification on the stored alarm information when needed, and displaying a credible verification result; the wireless communication module is matched with the wireless communication module and used for realizing data transmission between the robot monitoring center and the robot body. And the monitoring terminal receives the video image related to the invasion and the corresponding digital signature data thereof transmitted by the robot body through the wireless communication module.

If the infringer disagrees with the originality, authenticity and integrity of the video image related to the infringement after the digital signature operation, the video image related to the infringement can be authenticated by credibility according to the following procedures to prove the originality, authenticity and integrity of the video image (namely the video image is shot by the infringement robot and is not tampered or cut):

1) and finding out the credible data TED with the same frame sequence number through the frame sequence number of each frame of video frame data.

2) The device ID is separated in the TED and the corresponding public key certificate is solicited based on this device ID.

3) And decrypting the SHD line by using the obtained public key certificate and the TED or a signature algorithm specified in the TED to obtain RSHD (shortened version of Recovered Signed Hashed Data, recovery Data abstract, namely video Data abstract Recovered from credible Data).

4) The frame of video frame Data is summarized using TED or the summarization algorithm specified in TED to obtain SCHD (short for Self-calculated Data).

5) Comparing RSHD and SCHD, if RSHD ═ SCHD, it is indicated that the frame data has not been tampered; otherwise, RSHD ≠ SCHD, which indicates that the frame data has been tampered.

Example 2

The embodiment provides a robot body security system based on camera array image analysis, and the composition of the robot body security system is basically the same as that of the system in embodiment 1.

In this embodiment, the camera array is a camera module array composed of 6 camera modules (reference model: golden dry high definition USB camera module sony IMX322 starlight level), each camera module has a horizontal field angle of 80 degrees, a resolution of 1280x720, and a frame rate of 25fps, and all camera modules are arranged around the robot body at 60-degree intervals and connected to the video image analysis device through a USB interface.

In this embodiment, the video image analysis device is a general-purpose server equipped with a GPU acceleration card (reference model: Tes la M44 GB), and is capable of analyzing and processing more than 8 channels of video images in real time and determining whether there is a behavior or an action infringing the robot body.

In this embodiment, the cache memory is a DDR4 memory installed in the video image analysis device, and has a capacity of 32GB (reference model: SAMSUNG server memory bank DDR 42400 RECC 2Rx 432G), and is capable of temporarily storing all the video images for more than 3 minutes.

In the present embodiment, the nonvolatile memory is an SSD solid state disk installed within the video image analysis apparatus, has a capacity of 250GB (reference model: SAMSUNG 860 EVO 250G), and is capable of holding video images relating to an attack and digital signature data corresponding to the video images for a long period of time.

In the embodiment, the security module is an encryption card (reference model: Jiangnan Tianan SJK1322) installed in the video image analysis device, and the encryption card supports domestic cryptographic algorithms such as SM1, SM2, SM3 and SM 4.

In this embodiment, the wireless communication module is a wireless network card (reference model: mercury UD13H driver-free 1300M gigabit dual-band USB wireless network card), and can implement data transmission between the video image analysis device and the robot monitoring center through a WLAN network.

In the robot monitoring center, a wireless communication module adopts a wireless network card (reference model: Mercury UD13H non-drive 1300M gigabit dual-frequency USB wireless network card), an information storage is an SSD solid state disk installed in a monitoring terminal, the capacity is 1TB (reference model: SAMSUNG 860 QVO 1TB), the reference model of a certificate management and trusted authentication server is Jiangnan Tianan SRJ1303, and the monitoring terminal is a general computer terminal or server.

Example 3

In this embodiment, the camera array IS composed of 8 high-definition webcams (reference model: Haikang 200 ten thousand high-definition infrared webcams DS-2CD3125-45FV2-IS) with audio sensors (microphones), each camera has a horizontal field angle of 86 degrees, a resolution of 1920x1080 and a frame rate of 30fps, all the cameras are arranged around the robot body at 45-degree intervals, and the captured audio and video are transmitted to the digital hard disk video recorder through an internal local area network by using Ethernet cables.

In the embodiment, the video image analysis equipment comprises a main controller and a digital video recorder (reference model: Haokangwei video brain NVR face recognition snapshot monitoring hard disk recorder iDS-7908NX-K4/FA) with a video analysis function, wherein the main controller is connected with the digital video recorder in a communication way; the digital hard disk video recorder receives 8 paths of video images sent by the camera array, sends the video images to the cache memory for temporary storage, and simultaneously analyzes the content of the 8 paths of video images. The main controller is a general industrial personal computer, receives video images related to the invasion and obtained by analysis of the digital hard disk video recorder, sends the video images to the security module, receives the video images related to the invasion and digital signature data corresponding to the video images from the security module, stores the video images in the nonvolatile memory and sends the video images to the robot monitoring center.

In the embodiment, the cache memory is an SSD solid state disk installed in a digital video recorder, has a capacity of 250GB (reference model: SAMSUNG 860 EVO 250G), and can store all video images transmitted by the camera array in a short time.

In this embodiment, the nonvolatile memory is an SSD solid state disk installed in the main controller, has a capacity of 250GB (reference model: SAMSUNG 860 EVO 250G), and can store video image data related to an attack and corresponding digital signature data for a long period of time.

In this embodiment, the security module is an encryption card (reference model: Jiangnan Tianan SJK1322) installed in the main controller, and the encryption card supports domestic cryptographic algorithms such as SM1, SM2, SM3 and SM 4.

In this embodiment, the wireless communication module is a 4G wireless router (reference model: hua 4G wireless router E8372h-155 full network), and can implement data transmission with the monitoring center through a 4G network.

In the robot monitoring center, a monitoring terminal is a universal computer terminal or a server, an information storage is an SSD solid state disk installed in the monitoring terminal, the capacity is 1TB (reference model: SAMSUNG 860 QVO 1TB), the reference model of a certificate management and trusted verification server is Jiangnan Tianan SRJ1303, and a wireless communication module is a 4G wireless router (reference model: Hua 4G wireless router E8372h-155 full network communication).

Example 4

This example provides a method for using the system of examples 1-3, as shown in fig. 2, comprising the steps of:

s1, continuously acquiring video images around the robot by the camera array and transmitting the video images to video image analysis equipment;

s2, the video image analysis equipment firstly sends the received video image to a cache memory for temporary storage, and simultaneously analyzes the video image in real time to judge whether the damage to the robot body occurs or not;

s3, when the analysis result of a certain video image discovers that the robot body is damaged, the video image analysis equipment reads the video images which are shot by each camera module or camera and are related to damage occurrence in the set time length (for example, half an hour before and after the damage occurrence) containing the generation time from the cache memory according to the generation time of the certain video image and sends the video images to the safety module;

s4, the security module carries out digital signature operation on the received video images related to the occurrence of the invasion, and sends the video images related to the occurrence of the invasion and digital signature data corresponding to the video images to video image analysis equipment, and the video image analysis equipment receives the video images and sends the video images to a nonvolatile memory for storage, and sends the video images to a robot monitoring center through a wireless communication module for alarming;

s5, after receiving video images related to the occurrence of the invasion and digital signature data corresponding to the video images, a monitoring terminal of the robot monitoring center stores the video images in an information memory, displays the video images in real time for rechecking by monitoring personnel, and sends out a worker to go to the position of the robot body for disposal after the monitoring personnel confirms that the invasion acts; and when needed, the monitoring terminal controls the certificate management and the trusted verification server to perform trusted verification on the video image.

Various corresponding changes and modifications can be made by those skilled in the art according to the above technical solutions and concepts, and all such changes and modifications should be included in the protection scope of the present invention.

Claims

1. A robot body security system based on camera array image analysis is characterized by comprising a camera array, video image analysis equipment, a cache memory, a nonvolatile memory, a security module and a wireless communication module; the camera shooting array comprises a plurality of camera shooting modules or cameras, and the camera shooting modules or the cameras are arranged around the robot body and are circumferentially distributed on the surface of the robot body; the camera array, the cache memory, the nonvolatile memory, the security module and the wireless communication module are all in communication connection with the video image analysis equipment;

2. The robot body security system based on camera array image analysis according to claim 1, wherein shooting ranges of adjacent camera modules or cameras are partially overlapped.

3. The robot body security system based on camera array image analysis of claim 1, wherein the non-volatile memory is a solid state disk or a mechanical hard disk.

4. The robot body security system based on camera array image analysis of claim 1, wherein the wireless communication module comprises one or more of a combination of a WiFi wireless communication module, a WLAN wireless communication module, a 3G wireless communication module, a 4G wireless communication module, and a 5G wireless communication module.

5. The robot body security system based on camera array image analysis according to claim 1, wherein the robot monitoring center is provided with a monitoring terminal, an information memory, a certificate management and trusted verification server and a wireless communication module, and the information memory, the certificate management and trusted verification server and the wireless communication module are all in communication connection with the monitoring terminal.