CN116600087A

CN116600087A - Integrated machine device for video analysis, video analysis method and system

Info

Publication number: CN116600087A
Application number: CN202310681145.1A
Authority: CN
Inventors: 吴正中; 张辉; 孔祥开; 汪永刚; 王晓东; 韩建军; 唐才荣
Original assignee: Beijing Urban Construction Intelligent Control Technology Co ltd
Current assignee: Beijing Urban Construction Intelligent Control Technology Co ltd
Priority date: 2023-06-09
Filing date: 2023-06-09
Publication date: 2023-08-15

Abstract

The invention provides an all-in-one machine device for video analysis, a video analysis method and a system, wherein the device comprises the following components: the comprehensive monitoring system is connected with the ARM processor through the exchange module and is used for collecting video monitoring images of the station; and the GPU module is connected with the ARM processor and used for analyzing the video monitoring image according to a control instruction of the ARM processor and judging whether a target object invades the warning area. According to the invention, the ARM processor and the GPU module are utilized to complete analysis of the monitoring video, so that the in-station video analysis function is realized, the requirement on the networking video bandwidth is reduced, and the equipment cost of the whole project is saved.

Description

Integrated machine device for video analysis, video analysis method and system

Technical Field

The invention relates to the technical field of urban rail transit video monitoring, in particular to an integrated machine device for video analysis, a video analysis method and a video analysis system.

Background

In the current urban intelligent rail express system station, a plurality of professional systems are arranged according to different service functions. The professional systems are mutually independent and perform their functions. The main service function of the video monitoring system (CCTV) is to monitor real-time video in a station and transmit video data to a line center server through a network by a three-layer switch in an in-station network transmission system. The line center can check the real-time video dynamic picture of each station in real time so as to find abnormal conditions and perform corresponding operations. Meanwhile, a storage device is arranged in the line center, and video data collected by all station cameras in a certain time are stored according to customer requirements, so that the post-inquiring is facilitated.

In general, CCTV professionals at a station of an intelligent track express system are equipped with 4-8 cameras (respectively having 12 and 16 demands) which are arranged at both sides of the station and at the exit of the station for monitoring the condition of passenger flow; the middle of the station is used for monitoring the requirements of passengers on and off the doors and security in the station. The in-station real-time monitoring video data is required to be transmitted to the line center for display, analysis and storage through an in-station switch, the in-station device is generally not provided with a video analysis device, and equipment of other systems in the station device also uploads local service data to the line center through a switch in a network transmission system, receives and executes control instructions from the line center, and each equipment is distributed with different network segments or VLANs and is relatively independent.

If a certain station has the actions of crossing a fence, running a gate, reversing, and the like, and generally relies on station staff to find and intervene, the actions cannot be timely processed, so that the running safety hazards of passengers and trains are caused, the workload of station patrol staff is increased intangibly, and the labor management cost is increased.

At present, the traditional video monitoring system, the automatic ticket vending and checking system, the broadcasting system and the terminal equipment of the passenger information service system all realize the data interaction with the control center server through the exchanger of the network system. In the line network center, each professional terminal server often runs the application software of the professional. And each professional server processes the service data uploaded by each subsystem, configures the equipment of each station terminal through a network, and is provided with an independent control operation unit.

It can be seen that the prior art has the following disadvantages:

1. because all data services of the subsystems of each station are transmitted to the network center through the network, the service data volume transmitted by the network is large, the network is often required to have higher bandwidth, the hardware resources of the networking switch are higher, and the networking cost is high.

2. Because the server end is all in the online network center, the video monitoring system and the broadcasting, linkage and platform door control system are coordinated by workers, and the efficiency is low. And all stations are centralized in an online network center, the hardware requirements on each service server and centralized management equipment in the center are higher, and once the service is failed, the service operation of all stations is often influenced.

3. The video monitoring service system in the intelligent rail station is generally provided with cameras only at the station, the station is provided with no calculation unit and no storage unit, and the intelligent rail station has no capability of video analysis and other system linkage. When a network device or network fails, local in-station traffic is in an out-of-control state.

4. The linkage timeliness is poor, the traditional networking mode requires data transmission in a station to a network center, data interaction and linkage control are carried out with other subsystem servers in the network center after analysis processing, human participation is required, certain misjudgment and missed judgment are possible, and network transmission with large data volume also causes certain time delay.

Disclosure of Invention

In order to overcome the defects in the prior art, the invention aims to provide an all-in-one machine device for video analysis, a video analysis method and a video analysis system.

A all-in-one device for video analysis, comprising:

the comprehensive monitoring system is connected with the ARM processor through the exchange module and is used for collecting video monitoring images of the station;

and the GPU module is connected with the ARM processor and used for analyzing the video monitoring image according to a control instruction of the ARM processor and judging whether a target object invades the warning area.

Preferably, the method further comprises:

and the audio module is respectively connected with the ARM processor and each loudspeaker and is used for sending alarm voice according to a control instruction of the ARM processor.

Preferably, the method further comprises:

and the SATA hard disk is connected with the ARM processor and is used for storing the video monitoring image.

The invention also provides a video analysis method, which comprises the following steps:

step 1: acquiring a video monitoring image and a marked warning area thereof;

step 2: identifying and detecting whether a target object exists in the video monitoring image;

step 3: when the target object is in the video monitoring image, acquiring a detection frame of the target object;

step 4: and when the detection frame of the target object is overlapped with the warning area, sending out a violation warning signal.

Preferably, in said step 3: when the video monitoring image has the target object, acquiring a detection frame of the target object further comprises:

and obtaining a corresponding mark picture in the video monitoring image according to the marked warning area.

Preferably, step 4: when the detection frame of the target object is overlapped with the warning area, a violation warning signal is sent out, and the method further comprises the following steps:

step 4.1: drawing a rectangle according to the coordinates of each vertex of the mark picture and the detection frame to obtain a new warning area and a new detection frame;

step 4.2: judging whether the new warning area and the new detection frame are overlapped or not according to the vertex coordinates of the new warning area and the new detection frame;

step 4.3: when the new warning area is overlapped with the new detection frame, judging whether the overlapped area is larger than a threshold value or not; wherein, the calculation formula of the superposition area is:

S= MIN(|X23-X21|,|X13-X21)* MIN(|Y23-Y21|,|Y13-Y21)

wherein S is the combined area, (X13, Y13) is the vertex coordinate of the new warning area, (X23, Y23) is the first vertex coordinate of the new detection frame, and (X21, Y21) is the second vertex coordinate of the new detection frame;

step 4.4: and when the overlapping area exceeds a threshold value, sending out a violation alarm signal.

The invention also provides a video analysis system, comprising:

the image acquisition module is used for acquiring the video monitoring image and the marked warning area;

the target detection module is used for identifying and detecting whether a target object exists in the video monitoring image;

the detection frame acquisition module is used for acquiring a detection frame of the target object when the target object is in the video monitoring image;

and the coincidence detection module is used for sending out a violation alarm signal when the detection frame of the target object coincides with the warning area.

The present invention also provides a computer readable storage medium having stored thereon a computer program, characterized in that the computer program when executed by a processor implements the steps of a video analysis method as described above.

The integrated machine device for video analysis, the video analysis method and the system provided by the invention have the beneficial effects that: compared with the prior art, the invention realizes the function of in-station video analysis by utilizing the ARM processor and the GPU module to analyze the monitoring video, reduces the requirement on the bandwidth of the networking video, and saves the equipment cost of the whole project.

In order to make the above objects, features and advantages of the present invention more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 shows a schematic structural diagram of an integrated machine device for video analysis according to an embodiment of the present invention;

fig. 2 shows a schematic diagram of connection of an all-in-one device according to an embodiment of the present invention;

FIG. 3 shows a schematic diagram of the operation of the integrated machine device according to the embodiment of the present invention;

fig. 4 is a schematic diagram of a coincidence determination process according to an embodiment of the present invention.

Description of the embodiments

In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present invention.

Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present invention, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.

In the present invention, unless explicitly specified and limited otherwise, the terms "mounted," "connected," "secured," and the like are to be construed broadly and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.

Referring to fig. 1-2, an all-in-one device for video analysis includes: the system comprises an integrated monitoring system, a GPU module, an ARM processor (CPU), an audio module and a SATA hard disk.

The comprehensive monitoring system is connected with the ARM processor through the exchange module and is used for collecting video monitoring images of the station; and the GPU module is connected with the ARM processor and used for analyzing the video monitoring image according to a control instruction of the ARM processor and judging whether a target object invades the warning area. And the audio module is respectively connected with the ARM processor and each loudspeaker and is used for sending alarm voice according to a control instruction of the ARM processor. And the SATA hard disk is connected with the ARM processor and is used for storing the video monitoring image.

It should be noted that, in the present invention, the CPU adopts the ARM processor, and compared with the high-performance X86 processor of the station center server, the ARM processor has obvious advantages in terms of space, cost, power consumption, etc. In the invention, LS1043 of NXP company is adopted as a main processor, and LS1043 has 4 cores and 1.6Hz main frequency and is compatible with 4-32 GB of memory. The on-chip Ethernet port is provided with 7 Ethernet ports, so that abundant internet resources are provided for the internal networking of each board card.

In the invention, a BCM56150 of the Botong company is adopted as a main chip of the switching circuit, and a high-performance ARM processor Cortex-A9 is integrated in the SOC. 24 gigabit ethernet ports and 4 megaoptical ports are supported. The 24 gigabit Ethernet ports can be used for accessing cameras of CCTV professions in stations, and can also be used for accessing professional equipment such as ISCS, PIS, PA, and the 4 megalight ports can be used for accessing stations and line center ring networks.

Compared with other GPU servers, such as GPU products with high computing power, high power consumption and high cost, for example, the model number of NVIDIA is T4 and A100, the video analysis circuit adopts the Jetson series GPU module of NVIDIA company, and the Jetson series GPU module is more suitable for application scenes such as lightweight video data, analysis and processing due to moderate computing power, low cost and lower power consumption.

The design adopts a solid state disk, the storage capacity can be selected according to the requirements of clients, and can be up to 2TB, so that the problem that local service cannot be stored due to network faults is solved, and a SATA interface is adopted as a hard disk interface.

The station video monitoring system collects real-time image data through network cameras arranged at various positions, uploads the collected real-time image data to a GPU module and stores the collected real-time image data to a solid state disk after the real-time image data are assembled through a switching circuit, the video stream is decoded through a video analysis unit GPU module, target identification operation is carried out on decoded pictures, the pictures after target identification and pretreatment are compared with pictures with preset warning areas, whether the identified targets coincide with the preset warning areas or not and how much the identified targets coincide with the preset warning areas are obtained, and when the coincidence degree reaches or exceeds a preset warning value, the identified targets are judged to invade the warning areas. And the GPU module feeds back the analysis result to the ARM processor.

And the Core Processor (CPU) judges whether other professional terminal equipment needs to be called according to a preset service rule, and responds. For example, when the passengers outside the station cross the fence and the passengers waiting for the bus enter the safe driving guard zone by mistake, the broadcasting system can be used for carrying out voice broadcasting warning on the passengers crossing the fence, the train dispatching system is linked, the door control system is shielded, the station personnel are notified to carry out on-site treatment, and the occurrence of safety accidents is avoided. The method can also realize target identification, and can track the moving route of the identified target according to the time sequence of collecting pictures in the video stream. When detecting that a certain target does not move in the conventional direction or the moving speed is abnormal, the target can be locked for key tracking, and if the target moves in the reverse direction or breaks the gate, the broadcasting system can be driven to alarm, and the gate of the incoming station is closed or an acousto-optic alarm instruction is sent. If the target number in a certain area is detected to be larger than the preset threshold value within the set time, the passenger flow in the current station can be considered to be larger, and the passenger information system is required to be mobilized to prompt passengers to enter and exit orderly, or the gate of the ticket vending and checking system is notified to be closed reasonably, so that the purpose of passenger flow restriction is achieved.

Referring to fig. 3, in the present invention, the integrated machine has an active alarm reporting function, and reports an alarm signal to the net control center according to a preset priority according to an analysis result, and switches the control operation of the response according to a command returned by the net center. The system also has a command for responding to the network center to retrieve the local storage data, so that the center terminal can monitor and manage each station conveniently.

The invention integrates the functions of exchange, calculation, GPU video analysis and storage on one device. The network camera in the comprehensive monitoring system collects real-time video data of each point in the station, video summarizing and uploading are achieved through the switching circuit, the CPU controls the solid state disk to store the video data, the GPU module is controlled to achieve intelligent analysis of the video, and the IO board card is controlled to achieve linkage control of the peripheral according to analysis results.

For example, when the camera collects the movement of the passengers crossing the fence, the video is forwarded to the CPU through the exchange circuit connected with the network cable, the CPU controls the data storage and simultaneously sends the data to the GPU module, the information is fed back and processed after the video analysis, the response linkage program is started according to the video analysis result, the output linkage program comprises the alarm voice sent to the broadcasting system, and the related alarm text and picture information is output through the passenger information system. And outputting a voice alarm signal with the position information to a patrol personnel in the station, and uploading the processing result and the alarm log to a relevant server of the line center.

The system can be used for monitoring the passenger flow condition of the platform, flexibly starting and closing the number of gates of the automatic ticket vending and checking system and the number of doors of the platform door system according to the passenger flow, and the process is similar to the process.

In a word, the integrated machine device for video analysis is particularly suitable for being deployed in a station of an urban intelligent rail express system, and solves the problems of poor linkage of a video monitoring system, a ticket selling and checking system, a passenger information system, a comprehensive monitoring system and a broadcasting system and poor real-time event processing. The ARM processor, the exchange core module and the small GPU module scheme adopted by the invention have high cost performance on the premise of not increasing occupied space and power consumption. In addition, compared with the scheme of deploying the GPU analysis server in the line center, the video analysis method and the video analysis device have the advantages of being low in communication bandwidth occupancy rate and strong in linkage instantaneity among professions in a local station, and the advantage of edge calculation is fully exerted.

step 1: acquiring a video monitoring image and a marked warning area thereof;

after the step 3, the method further comprises:

Further, the method comprises the following steps. Step 4 comprises:

step 4.3: when the new warning area is overlapped with the new detection frame, judging whether the overlapped area is larger than a threshold value or not;

The following describes a video analysis method according to the present invention with reference to specific embodiments:

the video analysis algorithm of the invention combines the YOLO algorithm and the OpenCV database to realize intelligent video analysis, and the specific process is as follows:

aiming at the method for judging that the target object enters the warning area, the invention adopts the following steps:

1. defining a warning area: acquiring video monitoring images and marking warning areas

2. Detecting a target object: identifying and detecting whether target object exists in video monitoring image

3. And (3) marking picture acquisition: the marked image P1 of the warning area is obtained according to the marked warning area, when the video monitoring image is detected to contain a target object, a detection frame surrounding the target object is defined for the target problem, and the marked image P2 of the target object is obtained according to the detection frame

4. And (3) area intrusion detection: calculating each pixel point of the P1 and P2 pictures to obtain a result value, judging whether the detection frame is overlapped with the warning area or not according to the result value and a preset value, and judging that the detection frame is overlapped with the warning area when the result value is larger than the preset value; and when the result is smaller than the preset value, judging that the detection frame is not coincident with the warning area.

5. In the technical scheme of the invention, an alarm region is marked on a video monitoring image, and a marked picture P1 of the alarm region is obtained according to the marked alarm region. And identifying and detecting whether the video monitoring image contains a target object, and when the video monitoring image contains the target object, defining a detection frame surrounding the target object for the target object, and obtaining a picture P2 of the target object according to the detection frame.

6. And respectively finding out the vertex coordinates (possibly a rectangle or an irregular pattern) of the warning area in the P1 picture and the detection frame in the P2 picture.

7. According to the coordinates of each vertex of the warning area in the P1 picture and the detection frame in the P2 picture, a regular rectangle D1 (x 11, y11, x12, y12, x13, y13, x14, y 14) and D2 (x 21, y21, x22, y22, x23, y23, x24, y 24) are respectively drawn again, so that a new regular rectangle is ensured, and the original warning area and the detection frame can be covered. Wherein D1 is a new warning area marked in the P1 picture, and D2 is a new detection frame marked in the P2 picture.

8. Comparing the coordinates of the vertexes of 4 of the rectangle D1 and the rectangle D2, judging whether the 2 rectangles are overlapped, if so, indicating that the detected object in the detection frame is possibly close to the warning area or invades the warning area, and if so, further calculating whether the overlapped area exceeds a preset threshold value.

Description of the determination process of coincidence and the calculation process of the coincidence area:

the overlapping judgment basis of the D2 and the D1 is that the coordinates of any one or more of the 4 vertexes of the rectangle D2 are positioned in the range of the rectangle D1.

As shown in fig. 4, the lower left corner coordinates (X21, Y21) of D2 are located within the D1 range. The specific judgment basis is (X21 > X11& & X21 < X13) & gt & (Y21 > Y11& Y21 < Y13). Similarly, the other 3 vertices of D2 are judged in a similar manner. If there are points satisfying the above judgment among the 4 vertices of D2, it is judged that D2 coincides with D1.

The overlapping area calculating method is that from any vertex of D2 in the range of D1 as a starting point, the overlapping area calculating method extends to the intersection point of D2 and D1 or the opposite side of D2 to the X axis and the Y axis respectively, determines the length of the X axis and the length of the Y axis of the overlapping part, and takes the product of the X axis and the Y axis to obtain the area of the overlapping part.

Given that the vertex (X21, Y21) of D2 located within the D1 range is the lower left corner of D2, the X-axis side length of the overlapping area can be determined by taking the minimum value from the right X-coordinate value of D2 and the right boundary X-coordinate value of D1. Namely, the overlapping portion has an X-axis direction side length of MIN (|X23-X21|, |X13-X21).

Similarly, since the vertex (X21, Y21) of D2 located within the D1 range is the lower left corner of D2, the Y-axis side length of the overlapping area can be determined by taking the minimum value from the upper Y coordinate of D2 and the upper Y coordinate of D1. Namely, the overlapping portion has a side length of MIN in the Y-axis direction (|Y23-Y21|, |Y13-Y21)

The overlapping area is:

S= MIN(|X23-X21|,|X13-X21)* MIN(|Y23-Y21|,|Y13-Y21)。

9. when it is detected that the detected object has invaded the warning zone, continuous detection is started and the overlapping area of D1 and D2 of each detection time node is calculated.

10. When the duration of the intrusion of the detected object into the warning area is prolonged, the intrusion area is increased, and the intrusion areas respectively reach a preset threshold value, the detected object is judged to be in illegal and illegal action.

The warning area demarcating step comprises the steps of obtaining video monitoring images of the monitoring points and establishing a coordinate system of the obtained frames of the video monitoring images. For example, the lowest left corner of the picture is marked as the origin of coordinates (0, 0), the lowest side of the picture is the x-axis, the leftmost side is the y-axis, and each pixel point in the picture has one coordinate. And (3) dividing a fence between the driving area and the waiting area in the picture and a gate between the payment area and the non-payment area into warning areas. And marking the coordinate values of the boundary vertexes of the relevant warning areas in the picture.

The demarcation of the detection frame is similar to the demarcation of the warning zone. The detection target object is typically a human passenger. When the video image is detected to contain the target object, a detection frame which can surround the target object is defined. The coordinate values of the boundary vertexes of the detection frame can be calibrated by using the same set of coordinate systems. The difference from the warning area is that the number of detection targets is not fixed, and the positions (i.e., coordinate values) where the detection targets are located are changed in each frame of picture.

The algorithm must mark all the target objects (passengers or staffs) in each frame of picture and number them, and in the next frame of picture, not only all the target objects (passengers or staffs) marked in the previous frame are tracked, but also the newly added target objects (new passengers and passengers getting off) in the picture need to be numbered. And meanwhile, the target objects (the boarding and the alighting personnel) which are disappeared in the picture are subjected to marketing. Only then can a plurality of targets be tracked respectively, and the purpose of tracking the movement track of the targets is achieved.

The alert region construction method is to draw a given vertex into a straight line in the picture P1 by connecting the given vertex with the cv2.Line () function of the opencv library.

The target object detection method is that under the pytorch framework, a yolo algorithm is used for identifying and detecting a target object, wherein the target object mainly comprises people, and particularly comprises passengers and staffs.

According to the specific embodiment of the invention, the invention discloses the following technical effects:

1. compared with the prior art, the video local intelligent analysis and storage are realized, the bandwidth requirement of networking between a station and a network control center is reduced, the forwarding and data throughput pressures of a core switch of the network center are reduced, the hardware requirement of the network center on storage equipment is reduced, and the networking cost is greatly reduced.

2. The timeliness of linkage control under emergency is improved, and local video acquisition, analysis and linkage control are realized. The method can detect the violations of station passengers crossing fences, running a gate, going backward and the like in real time, improve the recognition rate of the violations and reduce the workload of on-site staff.

The invention also provides a video analysis system, comprising:

The present invention also provides a computer readable storage medium having stored thereon a computer program, characterized in that the computer program when executed by a processor implements the steps of a video analysis method as described above. Compared with the prior art, the beneficial effects of the computer readable storage medium provided by the invention are the same as those of the video analysis method described in the technical scheme, and the description is omitted here.

The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art can easily think about variations or alternatives within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A all-in-one device for video analysis, comprising:

2. The all-in-one device for video analysis of claim 1, further comprising:

3. The all-in-one device for video analysis of claim 2, further comprising:

4. A method of video analysis, comprising:

step 1: acquiring a video monitoring image and a marked warning area thereof;

5. The video analysis method according to claim 4, wherein in the step 3: when the video monitoring image has the target object, acquiring a detection frame of the target object further comprises:

6. The video analysis method according to claim 5, wherein step 4: when the detection frame of the target object is overlapped with the warning area, a violation warning signal is sent out, and the method further comprises the following steps:

S= MIN(|X23-X21|,|X13-X21)* MIN(|Y23-Y21|,|Y13-Y21)

7. A video analysis system, comprising:

8. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of a video analysis method according to any of claims 4-6.