CN115700854A

CN115700854A - Vehicle image display method and system, electronic device and readable storage medium

Info

Publication number: CN115700854A
Application number: CN202110857331.7A
Authority: CN
Inventors: 应良佳
Original assignee: Fengtu Technology Shenzhen Co Ltd
Current assignee: Fengtu Technology Shenzhen Co Ltd
Priority date: 2021-07-28
Filing date: 2021-07-28
Publication date: 2023-02-07

Abstract

The application provides a vehicle image display method, a vehicle image display system, electronic equipment and a computer readable storage medium. The vehicle image display method comprises the following steps: establishing connection between the small program end and the vehicle-mounted acquisition equipment through a local area network of the vehicle-mounted acquisition equipment, wherein images acquired by an acquisition program based on the vehicle-mounted acquisition equipment are stored in the vehicle-mounted acquisition equipment; sending an image reading request of a target vehicle to the vehicle-mounted acquisition equipment based on an http interface of a miniature server in the acquisition program; sending a target image of the target vehicle to the small program end, wherein the target image is an image of the target vehicle fed back by the vehicle-mounted acquisition equipment based on the image reading request; and displaying the target image at the small program end. The vehicle image/video file acquired by the vehicle-mounted screen-free acquisition equipment can be displayed in real time, and the real-time reading of the image or the video acquired by the vehicle-mounted screen-free acquisition equipment is improved.

Description

Vehicle image display method and system, electronic device and readable storage medium

Technical Field

The present application relates to the field of automotive technologies, and in particular, to a method and a system for displaying an image of a vehicle, an electronic device, and a computer-readable storage medium.

Background

Currently, vehicle-mounted acquisition devices for recording pictures or videos, such as automobile data recorders, are available in two types, namely with and without a display screen. The vehicle-mounted acquisition equipment without the display screen has the advantages of small size, convenience in installation, good concealment, no shielding of sight, low price and the like, and is deeply loved by users.

However, since the body of the vehicle-mounted acquisition device does not have a display screen, if a user wants to view a picture or a video recorded by the vehicle-mounted acquisition device, a professional operation and maintenance person is required to log in a back-end server to search for the related picture or video and send the picture or video to the user.

In actual use, for example, in a traffic accident, the requirement on the real-time performance of image and video acquisition is high, and the image or video viewing mode of the conventional vehicle-mounted screen-free acquisition equipment has high hysteresis.

Disclosure of Invention

The application provides a vehicle image display method, a vehicle image display system, electronic equipment and a computer readable storage medium, and aims to solve the problem that the high real-time requirement for reading pictures or videos in an actual scene cannot be met due to the fact that the picture or video viewing mode of the conventional screen-free vehicle-mounted acquisition equipment has large hysteresis.

In a first aspect, the present application provides a vehicle image display method, where the method is applied to a vehicle image display system, where the vehicle image display system includes a vehicle-mounted acquisition device and an applet terminal, and the method includes:

establishing connection between the small program end and the vehicle-mounted acquisition equipment through a local area network of the vehicle-mounted acquisition equipment, wherein images acquired by an acquisition program based on the vehicle-mounted acquisition equipment are stored in the vehicle-mounted acquisition equipment;

sending an image reading request of a target vehicle to the vehicle-mounted acquisition equipment based on an http interface of a miniature server in the acquisition program;

sending a target image of the target vehicle to the small program end, wherein the target image is an image of the target vehicle fed back by the vehicle-mounted acquisition equipment based on the image reading request;

and displaying the target image at the small program end.

In one possible implementation manner of the present application, the method further includes:

identifying vehicle behavior of the target vehicle based on the target image;

and determining whether the target vehicle violates the rule or not according to the vehicle behavior.

In one possible implementation manner of the present application, the determining whether the target vehicle violates the rule according to the vehicle behavior further includes:

when the target vehicle is determined to be in violation, outputting warning information of violation behavior of the target vehicle through the vehicle-mounted acquisition equipment;

or when the target vehicle is determined to be in violation, outputting warning information of violation behavior of the target vehicle through the small program terminal.

In a possible implementation manner of the present application, the vehicle image display system further includes a vehicle background management server, and the determining, according to the vehicle behavior, whether the target vehicle violates a rule further includes:

when the target vehicle is determined to be in violation, acquiring violation information of the target vehicle, wherein the violation information of the target vehicle comprises at least one of the target image, the position information of the target vehicle, the violation behavior of the target vehicle, basic information of the target vehicle and driver information of the target vehicle;

and storing the violation information of the target vehicle to the vehicle background management server.

receiving vehicle information of a vehicle to be inquired input by a user based on the applet terminal;

searching violation information of the vehicle to be inquired from the vehicle background management server based on the vehicle information;

and displaying violation information of the vehicle to be inquired at the applet terminal.

In one possible implementation manner of the present application, the identifying the vehicle behavior of the target vehicle based on the target image includes:

intercepting at least one frame of image based on the target image;

performing feature extraction on the basis of the at least one frame of image through a pre-trained vehicle behavior recognition model to obtain image features of the target image; and performing prediction processing based on the image characteristics to obtain the vehicle behavior of the target vehicle.

In a second aspect, the present application provides a vehicle image display system, which includes a vehicle-mounted acquisition device and an applet terminal;

the vehicle-mounted acquisition equipment is used for acquiring a target image of a target vehicle based on an acquisition program of the vehicle-mounted acquisition equipment;

the small program end is used for sending an image reading request of a target vehicle to the vehicle-mounted acquisition equipment based on an http interface of a miniature server in the acquisition program;

the vehicle-mounted acquisition equipment is also used for feeding back a target image of the target vehicle to the small program end according to the image reading request through an http interface of a miniature server in the acquisition program;

and the small program end is also used for displaying the target image.

In a possible implementation manner of the present application, the applet or the vehicle-mounted collection device is further configured to identify a vehicle behavior of the target vehicle according to the target image, and determine whether the target vehicle violates a rule based on the vehicle behavior;

when the target vehicle is determined to be in violation, the small program end or the vehicle-mounted acquisition equipment is further used for outputting warning information of violation behaviors of the target vehicle.

In a possible implementation manner of the present application, the vehicle image display system further includes a vehicle background management server, and when it is determined that the target vehicle violates a rule, the vehicle background management server is further configured to obtain violation information of the target vehicle; the vehicle background management server is also used for storing violation information of the target vehicle; wherein the violation information of the target vehicle comprises at least one of the target image, the position information of the target vehicle, the violation of the target vehicle, the basic information of the target vehicle, and the driver information of the target vehicle.

In a possible implementation manner of the present application, the applet is further configured to receive vehicle information of a vehicle to be queried, which is input by a user; the vehicle background management server is also used for searching violation information of the vehicle to be inquired based on the vehicle information and feeding back the violation information of the vehicle to be inquired to the small program end; and the small program end is also used for displaying violation information of the vehicle to be inquired.

In a possible implementation manner of the present application, the vehicle-mounted acquisition device, the applet terminal, or the vehicle backend management server is further configured to intercept at least one frame of image based on the target image; performing feature extraction on the basis of the at least one frame of image through a pre-trained vehicle behavior recognition model to obtain image features of the target image; and performing prediction processing based on the image characteristics to obtain the vehicle behavior of the target vehicle.

In a third aspect, the present application further provides an electronic device, where the electronic device includes a processor and a memory, where the memory stores a computer program, and the processor executes the steps in any one of the vehicle image display methods provided in the present application when calling the computer program in the memory.

In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, the computer program being loaded by a processor to execute the steps in the vehicle image display method.

The method comprises the steps that a small program end in a vehicle image display system can send an image reading request of a target vehicle to a vehicle-mounted acquisition device based on an http interface of a miniature server in an acquisition program of the vehicle-mounted acquisition device; the vehicle-mounted acquisition equipment transmits the image/video of the target vehicle acquired by the vehicle-mounted acquisition equipment to the small program end in a streaming file form through the http interface of the miniature server in the acquisition program, so that the image/video acquired by the vehicle-mounted acquisition equipment can be displayed in real time at the small program end, and the problems that the image or video viewing mode of the conventional screen-free vehicle-mounted acquisition equipment has large hysteresis and cannot meet the high real-time requirement for reading the image or video in an actual scene are solved. The vehicle image/video file collected by the vehicle-mounted collection device without the screen can be displayed in real time, and the reading real-time performance of the image or video collected by the vehicle-mounted collection device without the screen is improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the description below are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic view of a scene of a vehicle image display system according to an embodiment of the present disclosure;

FIG. 2 is a schematic view of a vehicle image display provided in an embodiment of the present application;

fig. 3 is a schematic view of another scene of the vehicle image display system according to the embodiment of the present application;

fig. 4 is a schematic flowchart of a vehicle image display method according to an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of an interaction between an applet end and a vehicle background management server in an embodiment of the application;

fig. 6 is a schematic structural diagram of an embodiment of a vehicle image display device provided in the embodiment of the present application;

fig. 7 is a schematic structural diagram of an embodiment of an electronic device provided in the embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In the description of the embodiments of the present application, it should be understood that the terms "first", "second", and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or to implicitly indicate the number of technical features indicated. Thus, features defined as "first", "second", may explicitly or implicitly include one or more of the described features. In the description of the embodiments of the present application, "a plurality" means two or more unless specifically defined otherwise.

The following description is presented to enable any person skilled in the art to make and use the application. In the following description, details are set forth for the purpose of explanation. It will be apparent to one of ordinary skill in the art that the present application may be practiced without these specific details. In other instances, well-known processes have not been described in detail in order not to obscure the description of the embodiments of the present application with unnecessary detail. Thus, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed in the embodiments of the present application.

The embodiment of the application provides a vehicle image display method, a vehicle image display system, electronic equipment and a computer readable storage medium.

First, a vehicle image display system provided in an embodiment of the present application is described below, referring to fig. 1, where fig. 1 is a schematic view of a scene of the vehicle image display system provided in the embodiment of the present application. This vehicle image display system 100 includes on-vehicle collection equipment 101 and applet end 102, and wherein, on-vehicle collection equipment 101 has the networking function, and on-vehicle collection equipment 101 can open the local area network, like the WIFI focus, and applet end 102 can be connected to the local area network that on-vehicle collection equipment 101 opened to establish on-vehicle collection equipment 101 and applet end 102's communication connection.

The vehicle-mounted acquisition equipment 101 is used for acquiring a target image of a target vehicle based on an acquisition program of the vehicle-mounted acquisition equipment 101. The on-board capturing device 101 is an image capturing device with a camera, such as a car recorder, and can be used to capture images/videos of the vehicle in some cases, such as in a driving state. The vehicle-mounted acquisition device 101 completes image/video acquisition under the driving of an acquisition program and stores the acquired image/video into a memory card, such as an sdcard directory, of the vehicle-mounted acquisition device 101. The vehicle-mounted acquisition device 101 may be a hardware acquisition device without a screen or a hardware acquisition device with a screen. The vehicle-mounted acquisition equipment 101 can acquire images of a target vehicle, acquire multiple frames of images and form a video through a time line; the target image refers to an image/video of the target vehicle acquired by the vehicle-mounted acquisition device 101 in some cases, such as a driving state.

The micro server (NanoHTTPD) is a lightweight HTTP server designed for embedded device applications, which facilitates the start-up of a local server in the embedded device (e.g., android device) to receive client local part requests. In order to transmit the images/videos acquired by the vehicle-mounted acquisition device 101 to the applet terminal 102, in the embodiment of the present application, the acquisition program of the vehicle-mounted acquisition device 101 uses a NanoHTTPD as an HTTP server, and provides services to the outside by using the images/videos in the memory card, such as an sdcard directory, as a stream file.

And the applet terminal 102 is configured to send an image reading request of the target vehicle to the vehicle-mounted acquisition device 101 based on an http interface of the mini server in the acquisition program. The applet terminal 102 is a carrier terminal of the applet, such as a mobile phone, a tablet Computer, a Personal Computer (PC), and so on. The applet may be an applet carried in any APP, such as a WeChat applet. The image reading request is a reading request of an image or video, and is not a reading request of only a video. Specifically, when the vehicle-mounted acquisition device 101 establishes a communication connection with the applet terminal 102, the applet terminal 102 requests an image/video stream based on an http interface of the applet calling the mini server.

The vehicle-mounted acquisition equipment 101 is further configured to feed back a target image of the target vehicle to the applet terminal 102 according to the image reading request through an http interface of the mini server in the acquisition program. Specifically, the image/video in the memory card directory is used as a streaming file by the acquisition program of the vehicle-mounted acquisition device 101, and when the vehicle-mounted acquisition device 101 receives an image reading request sent by the applet terminal 102 (based on the applet calling the http interface of the mini server), the mini server in the acquisition program of the vehicle-mounted acquisition device 101 transmits the image/video in the memory card directory to the applet terminal 102 in a streaming file manner.

The applet terminal 102 is further configured to display a target image of the target vehicle. The applet terminal 102 may receive the image/video stream file transmitted by the vehicle-mounted capturing device 101 based on the applet and may display the image/video stream file on the applet terminal 102. The applet terminal 102 is a terminal with a display unit, so as to display images/videos acquired by the vehicle-mounted acquisition device 101.

By the vehicle image display system provided by the embodiment of the application, the applet terminal 102 can send an image reading request of a target vehicle to the vehicle-mounted acquisition equipment 101 based on an http interface of a miniature server in an acquisition program of the vehicle-mounted acquisition equipment 101; the vehicle-mounted acquisition equipment 101 transmits the image/video of the target vehicle acquired by the vehicle-mounted acquisition equipment 101 to the small program end 102 in a streaming file form through an http interface of the miniature server in the acquisition program, so that the image/video acquired by the vehicle-mounted acquisition equipment 101 can be displayed in real time at the small program end 102, and the problems that the image or video viewing mode of the conventional screen-free vehicle-mounted acquisition equipment has large hysteresis and cannot meet the requirement of high real-time performance on image or video reading in an actual scene are solved.

A specific example is illustrated below. As shown in fig. 2, fig. 2 is a schematic view of a scene of a vehicle image display provided in the embodiment of the present application. The vehicle-mounted acquisition device 101 is a non-screen hardware Android acquisition device, the vehicle-mounted acquisition device 101 completes image/video acquisition under the drive of an acquisition program, and the acquired image/video is stored in a memory card of the vehicle-mounted acquisition device 101, such as an sdcard directory. The acquisition program uses the NanoHTTPD as an HTTP server, and provides the images/videos as streaming files for external services. The vehicle-mounted acquisition equipment 101 can open a WIFI hotspot, and the applet end 102 can be connected to the WIFI hotspot opened by the vehicle-mounted acquisition equipment 101. The applet terminal 102 uses a WIFI local area network to call an http interface of the NanoHTTPD to send an image/video reading request to the vehicle-mounted acquisition device 101. The vehicle-mounted collection device 101 feeds back the stream file of the image/video to the applet terminal 102 based on the NanoHTTPD. Therefore, the small program end 102 can receive the images/videos acquired by the vehicle-mounted acquisition equipment 101, and the images/videos acquired by the vehicle-mounted acquisition equipment 101 are displayed on the small program end 102 in real time through the small program, so that the problems that the image or video viewing mode of the conventional screen-free vehicle-mounted acquisition equipment has large hysteresis and cannot meet the high real-time requirement on image or video reading in an actual scene are solved.

Further, in some embodiments, the applet terminal 102 may be further configured to identify a vehicle behavior of the target vehicle according to the target image, and determine whether the target vehicle has an illegal phenomenon based on the vehicle behavior. When it is determined that the target vehicle has the violation phenomenon, the applet terminal 102 or the vehicle-mounted acquisition device 101 may further be configured to output warning information that the target vehicle has the violation behavior. So that the driver of the target vehicle can standardize the driving behavior after receiving the warning information, thereby improving the driving safety of the target vehicle.

Further, in some embodiments, the vehicle-mounted capturing device 101 may also be configured to identify a vehicle behavior of the target vehicle according to the target image, and determine whether the target vehicle has an illegal phenomenon based on the vehicle behavior. When it is determined that the target vehicle has the violation phenomenon, the applet terminal 102 or the vehicle-mounted acquisition device 101 may further be configured to output warning information that the target vehicle has the violation behavior. So that the driver of the target vehicle can standardize the driving behavior after receiving the warning information, thereby improving the driving safety of the target vehicle.

Further, the vehicle-mounted collection device 101 may also collect the location information of the target vehicle, for example, the vehicle-mounted collection device 101 may integrate a Global Positioning System (GPS) to locate the target vehicle through the GPS, so as to collect the location information of the target vehicle.

Further, referring to fig. 3, fig. 3 is a schematic view of another scene of the vehicle image display system provided in the embodiment of the present application, and the vehicle image display system 100 further includes a vehicle background management server 103. When the target vehicle is determined to be in violation, the vehicle background management server 103 is further configured to obtain violation information of the target vehicle; the vehicle background management server 103 is further configured to store violation information of the target vehicle; the violation information of the target vehicle comprises at least one of a target image, position information of the target vehicle, violation behaviors of the target vehicle, basic information of the target vehicle and driver information of the target vehicle.

For example, the vehicle background management server 103 is used for storing violation images/videos of vehicles, vehicle information (such as license plate number, owner information, driving track, etc.) of violation vehicles, violation places, violation classifications, and the like. Specifically, after the vehicle behavior of the target vehicle is identified according to the target image, and when it is determined that the target vehicle has an illegal behavior, the vehicle-mounted collection device 101 or the applet terminal 102 transmits the target image to the vehicle background management server 103. The vehicle-mounted acquisition device 101 is further configured to acquire a position and/or a driving track of the target vehicle, and transmit the position and/or the driving track to the vehicle background management server 103. In this case, the vehicle backend management server 103 may store the target images, vehicle information of the target vehicle (including the license plate number, owner information, and travel track of the target vehicle), an illegal point of the target vehicle, and illegal behavior of the target vehicle in association with each other. So that the relevant management personnel can know the violation condition of the vehicle in time and deal with the violation of the vehicle in time.

Further, the applet terminal 102 is further configured to receive vehicle information of the vehicle to be queried, which is input by the user; the vehicle background management server 103 is further configured to search violation information of the vehicle to be queried based on the vehicle information, and feed back the violation information of the vehicle to be queried to the applet terminal 102; the applet terminal 102 is also used for displaying violation information of the vehicle to be queried.

For example, the applet terminal 102 is further configured to receive, under the driving of the applet, vehicle information of the vehicle to be queried, which is input by a user, such as a license plate number of the vehicle to be queried, an owner of the vehicle, and the like. So that the applet terminal 102 can initiate a query request to the vehicle background management server 103 based on the vehicle information of the vehicle to be queried, and the vehicle background management server 103 can also query the violation information of the vehicle to be queried from the stored vehicle violation information. After inquiring the violation information of the vehicle to be inquired, the vehicle background management server 103 may also feed back the violation information of the vehicle to be inquired to the applet terminal 102. The applet end 102 is also used for displaying violation information of the vehicle to be queried so that a user can view the violation information.

In a possible implementation manner of the present application, the vehicle-mounted collection device 101, the applet terminal 102, or the vehicle background management server 103 is further configured to intercept at least one frame of image based on the target image; performing feature extraction on the basis of at least one frame of image through a pre-trained vehicle behavior recognition model to obtain image features of a target image; and performing prediction processing based on the image characteristics of the target image to obtain the vehicle behavior of the target vehicle.

It can be understood that, in the embodiments of the present application, a vehicle image display method, a vehicle image display system, an electronic device, and a computer-readable storage medium are provided, different descriptions will be given to different topics, and in view of that contents of various topic parts may be repeated, in the embodiments of the present application, reference may be made to descriptions of other topic parts for a part where a certain topic is not repeatedly described. For example, the implementation of "recognizing the vehicle behavior of the target vehicle according to the target image" will be described in detail in the following description of the vehicle image display method, and specific reference may be made to the following description of the vehicle image display method.

An execution main body of the vehicle image display method according to the embodiment of the present application may be the vehicle image display system provided in the embodiment of the present application, or may also be the vehicle image display apparatus provided in the embodiment of the present application, or may also be different types of electronic devices such as a server device, a physical host, or a User Equipment (UE) that integrates the vehicle image display apparatus, where the vehicle image display apparatus may be implemented in a hardware or software manner, and the UE may specifically be a terminal device such as a smart phone, a tablet computer, a notebook computer, a palm computer, a desktop computer, or a Personal Digital Assistant (PDA). The electronic device may adopt a working mode of independent operation, or may also adopt a working mode of a device cluster.

In the following, a vehicle image display method provided by an embodiment of the present application is described, where a vehicle image display system is used as an execution subject in the embodiment of the present application, as shown in fig. 1, the vehicle image display system may include a vehicle-mounted capturing device 101 and an applet terminal 102; further, as shown in fig. 3, the vehicle image display system may further include a vehicle background management server 103.

Referring to fig. 4, fig. 4 is a schematic flowchart of a vehicle image display method according to an embodiment of the present disclosure. It should be noted that, although a logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in an order different than that shown or described herein. The vehicle image display method comprises steps 401 to 404, wherein:

401. and establishing the connection between the small program end and the vehicle-mounted acquisition equipment through a local area network of the vehicle-mounted acquisition equipment.

The vehicle-mounted acquisition equipment is an image acquisition device with a camera, such as a vehicle data recorder and the like, and can be used for acquiring images/videos of a vehicle in a driving state under certain conditions.

The vehicle-mounted acquisition equipment can be hardware acquisition equipment without a screen or hardware acquisition equipment with a screen. The vehicle-mounted acquisition equipment can acquire images of a target vehicle, acquire multiple frames of images and form a video through a time line.

In this application embodiment, on-vehicle collection equipment has networking function, and on-vehicle collection equipment can open the local area network, like the WIFI focus to supply other equipment and on-vehicle collection equipment to establish communication connection.

The applet end is a bearer terminal of the applet, such as a mobile phone, a tablet computer, a personal computer, and the like. The applet may be an applet carried in any APP, such as a WeChat applet.

In some embodiments, before step 401, further comprising: local area network for starting vehicle-mounted acquisition equipment

In some embodiments, in step 401, a connection request may be actively initiated to the vehicle-mounted acquisition device through the applet terminal, and a communication connection between the applet terminal and the vehicle-mounted acquisition device is established through a local area network of the vehicle-mounted acquisition device.

In some embodiments, in step 401, a connection request may be actively initiated to the applet terminal by the vehicle-mounted acquisition device, and a communication connection between the applet terminal and the vehicle-mounted acquisition device may be established through a local area network of the vehicle-mounted acquisition device.

402. And sending an image reading request of the target vehicle to the vehicle-mounted acquisition equipment based on an http interface of the miniature server in the acquisition program.

The vehicle-mounted acquisition equipment stores images acquired by an acquisition program based on the vehicle-mounted acquisition equipment. In the embodiment of the application, the image acquired by the vehicle-mounted acquisition equipment is an image or a video acquired by the vehicle-mounted acquisition equipment.

The acquisition program of the vehicle-mounted acquisition equipment is a driving program of the vehicle-mounted acquisition equipment. The vehicle-mounted acquisition equipment completes the acquisition of images/videos under the drive of an acquisition program and stores the acquired images/videos into a memory card of the vehicle-mounted acquisition equipment, such as an sdcard directory.

In some embodiments, the vehicle-mounted collecting device may be installed on the target vehicle and used for collecting the traffic behavior of the target vehicle, such as whether to run a red light, whether to stop the vehicle in an illegal way, whether to turn the head in an illegal way, and the like.

In other embodiments, the vehicle-mounted collecting device may also be installed on another vehicle outside the target vehicle, and is configured to collect the traffic behavior of the target vehicle and the vehicle information of the target vehicle, so as to subsequently query the traffic behavior of the target vehicle based on the vehicle information of the target vehicle.

The micro server (NanoHTTPD) is a lightweight HTTP server designed for embedded device applications, which facilitates the start-up of a local server in the embedded device (e.g., android device) to receive client local part requests. In order to transmit the images/videos acquired by the vehicle-mounted acquisition device to the applet terminal, in the embodiment of the application, the acquisition program of the vehicle-mounted acquisition device uses a NanoHTTPD as an HTTP server, and the images/videos in the memory card, such as an sdcard directory, are used as a streaming file to provide external services.

In step 402, the applet terminal may call an http interface of the mini server in the acquisition program to send an image reading request of the target vehicle to the vehicle-mounted acquisition device. After receiving an image reading request sent by the small program terminal, the vehicle-mounted acquisition equipment feeds back an image/video obtained by real-time shooting of the vehicle-mounted acquisition equipment to the small program terminal based on the image reading request; or feeding back images/videos obtained by historical shooting of the vehicle-mounted acquisition equipment to the small program side based on the image reading request.

403. And sending the target image of the target vehicle to the small program end.

The target image is the image of the target vehicle fed back by the vehicle-mounted acquisition equipment based on the image reading request. The target image may be an image/video of the target vehicle acquired by the vehicle-mounted acquisition device under some conditions, such as a driving state.

The target image may be an image/video acquired by the vehicle-mounted acquisition device in real time, or an image/video acquired by the vehicle-mounted acquisition device at a historical time, such as the past 1h, 1 day, 1 week or 1 month, and may be specifically set according to the actual service scene requirements, where the image acquisition time corresponding to the image reading request is not limited.

In some embodiments, after the vehicle-mounted acquisition device receives the image reading request of the target vehicle, the vehicle-mounted acquisition device may directly feed back the target image of the target vehicle to the applet terminal according to the image reading request through an http interface of the mini server in the acquisition program of the vehicle-mounted acquisition device.

Specifically, the image/video in the memory card directory is used as a streaming file by the acquisition program of the vehicle-mounted acquisition device, and when the vehicle-mounted acquisition device receives an image reading request sent by an applet terminal (calling an http interface of a micro server based on the applet), the micro server in the acquisition program of the vehicle-mounted acquisition device transmits the image/video in the memory card directory to the applet terminal in a streaming file manner.

404. And displaying the target image at the small program end.

The small program end can display a target image of the target vehicle through the small program such as a WeChat small program, so that a user can view an image/video file of the target vehicle acquired by the non-screen acquisition equipment in real time.

As can be seen from the above, in the embodiment of the present application, the applet terminal in the vehicle image display system may send an image reading request of the target vehicle to the vehicle-mounted acquisition device based on the http interface of the mini server in the acquisition program of the vehicle-mounted acquisition device; the vehicle-mounted acquisition equipment transmits the image/video of the target vehicle acquired by the vehicle-mounted acquisition equipment to the small program end in a streaming file form through the http interface of the miniature server in the acquisition program, so that the image/video acquired by the vehicle-mounted acquisition equipment can be displayed in real time at the small program end, and the problems that the image or video viewing mode of the conventional screen-free vehicle-mounted acquisition equipment has high hysteresis and cannot meet the high real-time requirement for reading the image or video in an actual scene are solved. The vehicle image/video file acquired by the vehicle-mounted acquisition equipment without the screen can be displayed in real time, and the reading real-time performance of the image or video acquired by the vehicle-mounted acquisition equipment without the screen is improved

Due to the fact that roads are criss-cross and large in number, it is difficult to comprehensively arrange cameras to monitor violation behaviors of vehicles, and the arrangement of a large number of cameras requires huge time, labor, material resources and other costs, and therefore the violation behaviors of the vehicles cannot be effectively and comprehensively monitored in the existing traffic monitoring. To this end, in some embodiments, traffic behavior of the subject vehicle is also monitored based on-board collection devices installed on the subject vehicle. Namely, the vehicle image display method may further include: identifying vehicle behavior of the target vehicle based on the target image; and determining whether the target vehicle violates the rule or not according to the vehicle behavior.

The vehicle behavior refers to the traffic behavior of the target vehicle, and includes whether to run a red light, whether to stop illegally, whether to turn around illegally, whether to overspeed and the like.

In some embodiments, the vehicle behavior of the target vehicle may be identified based on the target imagery through an on-board acquisition device.

In some embodiments, the vehicle behavior of the target vehicle may also be identified by the applet based on the target image.

In some embodiments, the target image of the target vehicle may also be transmitted to a vehicle background management server, and the vehicle behavior of the target vehicle is identified based on the target image through the vehicle background management server. The vehicle background management system comprises a vehicle background management server, a vehicle-mounted acquisition device, a vehicle-mounted monitoring device and a vehicle-mounted monitoring device, wherein a target image of a target vehicle can be directly transmitted to the vehicle background management server through the vehicle-mounted acquisition device; or the target image of the target vehicle can be transmitted to the small program end from the vehicle-mounted acquisition equipment, and then the small program end transmits the target image of the target vehicle to the vehicle background management server.

The vehicle behavior recognition model trained in advance can be used for recognizing the vehicle behavior of the target vehicle based on the target image. In this case, the step "recognizing the vehicle behavior of the target vehicle based on the target image" may specifically include the following steps a1 to a2:

a1, intercepting at least one frame of image based on the target image.

For example, to detect whether the target vehicle is parking illegally, one frame of image may be captured based on the target image, so that whether the target vehicle is parking illegally is classified and predicted based on the frame of image, and whether the target vehicle is parking illegally is determined.

For another example, to detect whether the target vehicle runs a red light, two frames of images may be captured based on the target image, so as to predict whether the target vehicle runs the red light based on the two captured images, and determine whether the target vehicle runs the red light.

a2, extracting features based on the at least one frame of image in a pre-trained vehicle behavior recognition model to obtain image features of the target image; and performing prediction processing based on the image characteristics to obtain the vehicle behavior of the target vehicle.

The vehicle behavior recognition model can be obtained by training the following steps:

1. and constructing a preliminary vehicle behavior recognition model.

For example, an open source network (e.g., YOLOv network) with model parameters as default values (available for classification tasks) may be used as a preliminary recognition model, which may include a feature extraction module and a prediction module. The feature extraction module can be a convolutional neural network structure, and is used for performing feature extraction on the sample image to obtain the image features of the sample image. The prediction module may be a fully connected layer connected to the feature extraction module, and is configured to perform prediction based on image features of sample images, predict probabilities that the sample images are vehicle behaviors of respective categories, and output a vehicle behavior category with a highest probability as a vehicle behavior prediction category of the sample images.

2. A training data set is obtained.

Wherein the training data set comprises a plurality of sample images, which may be images containing vehicle violations. Each sample in the training dataset is labeled, and the labeling information comprises the category of the vehicle violation.

3. And (3) taking the marking information of the sample as supervision, training the preliminary vehicle behavior recognition model by adopting a training data set, and obtaining the trained vehicle behavior recognition model until the preliminary recognition model is converged. At this time, the trained vehicle behavior recognition model may be applied to recognize the vehicle behavior of the target vehicle in the target image.

The trained vehicle behavior recognition model can fully learn the relationship between the vehicle behavior and the image characteristics, so that the vehicle behavior of the target vehicle can be accurately recognized based on the image.

The training process of the vehicle behavior recognition model is similar to that of the existing classification network model, and for the training process which is not described in detail, the training mode of the existing classification network model can be referred to, and is not repeated here.

Specifically, after a vehicle behavior recognition model is obtained through training, at least one frame of image obtained by intercepting a target image can be input into the vehicle behavior recognition model, so that a feature extraction module in the vehicle behavior recognition model performs feature extraction on the at least one frame of image to obtain image features of the target image; and enabling a prediction module in the vehicle behavior recognition model to predict based on the image characteristics of the target image, determining the probability of each vehicle behavior type, and outputting the vehicle behavior type with the highest probability as the vehicle behavior of the target vehicle. Therefore, the vehicle behavior of the target vehicle can be recognized based on the target image, and accurate data basis is provided for traffic behaviors of drivers who subsequently standardize the target vehicle.

Furthermore, when the target vehicle breaks rule, the warning information of the target vehicle with the illegal behavior can be output, so that a driver of the target vehicle can be reminded to standardize the traffic behavior in time, and the traffic accident can be reduced.

In some embodiments, when the target vehicle is determined to be in violation, outputting, by the vehicle-mounted collection device, warning information that the violation occurs in the target vehicle. So that the driver of the target vehicle can standardize the driving behavior after receiving the warning information, thereby improving the driving safety of the target vehicle.

For example, the vehicle behavior of the target vehicle can be identified through the vehicle-mounted collection device, and when the violation of the target vehicle is determined, the warning information that the violation occurs in the target vehicle is output through the vehicle-mounted collection device. The output mode of the warning information may be voice broadcast through the vehicle-mounted acquisition device or turning on an indicator light of the vehicle-mounted acquisition device, and the specific form of the output mode of the warning information is not limited here.

For another example, the vehicle behavior of the target vehicle can be identified through the applet terminal, and when the violation behavior of the target vehicle is determined, the violation behavior of the target vehicle is transmitted to the vehicle-mounted acquisition device through the applet terminal, and the warning information that the violation behavior occurs in the target vehicle is output through the vehicle-mounted acquisition device.

In some embodiments, when the target vehicle is determined to be in violation, the applet end outputs a warning message that the violation of the target vehicle occurs. So that the driver of the target vehicle can standardize the driving behavior after receiving the warning information, thereby improving the driving safety of the target vehicle.

For example, the vehicle behavior of the target vehicle can be identified through the applet terminal, and when the violation of the target vehicle is determined, the applet terminal outputs the warning information that the violation of the target vehicle occurs. The output mode of the warning information may be to perform voice broadcast on the applet through the applet end, or to light up an indicator light of the applet end, where the specific form of the output mode of the warning information is not limited.

For another example, the vehicle behavior of the target vehicle may be identified by the vehicle-mounted collection device, and when the violation of the target vehicle is determined, the violation of the target vehicle is transmitted to the applet terminal by the vehicle-mounted collection device, and the applet terminal outputs the warning information that the violation of the target vehicle occurs.

Since the warning information is mainly used for prompting the driver to regulate the traffic behavior, in some cases, the traffic behavior of the driver is difficult to regulate without punishment due to lack of active awareness of individuals. Therefore, when the violation of the target vehicle is identified, the violation information of the target vehicle can be acquired and uploaded to a vehicle background management server of a judicial administration department to serve as the basis of traffic punishment.

To this end, in some embodiments, the step of determining whether the target vehicle is in violation based on the vehicle behavior further comprises: when the target vehicle is determined to be in violation, acquiring violation information of the target vehicle; and storing the violation information of the target vehicle to the vehicle background management server. The violation information of the target vehicle comprises at least one of a target image, position information of the target vehicle, violation behaviors of the target vehicle, basic information of the target vehicle and driver information of the target vehicle.

Furthermore, the vehicle-mounted acquisition equipment can also acquire the position information of the target vehicle, for example, the vehicle-mounted acquisition equipment can integrate a GPS system, and the target vehicle is positioned through the GPS system, so that the position information of the target vehicle is acquired.

After the violation information of the target vehicle is stored in the vehicle background management server, the violation information can be used as a basis for traffic violation punishment, and a traffic violation punishment sheet is sent to a driver of the target vehicle, so that the traffic behavior of the vehicle can be automatically monitored.

Therefore, the vehicle behavior of the target vehicle is identified through the image information acquired by the vehicle-mounted acquisition equipment of the target vehicle, so as to determine whether the target vehicle violates rules or not; and when the target vehicle breaks rules and regulations, the violation information of the target vehicle is uploaded to the vehicle background management server, so that the vehicle behavior of the target vehicle is effectively monitored, and the problem that the violation behavior of the vehicle cannot be effectively and comprehensively monitored in the conventional traffic monitoring is solved.

Further, the user can also inquire the violation information of the vehicle based on the applet, and at this time, the vehicle image display method further includes the following steps b1 to b3:

and b1, receiving vehicle information of the vehicle to be inquired, which is input by the user based on the small program terminal.

The vehicle information of the vehicle to be queried is identification information of the vehicle to be queried, for example, a license plate number of the vehicle to be queried.

b2, searching the violation information of the vehicle to be inquired from the vehicle background management server based on the vehicle information of the vehicle to be inquired.

The vehicle violation information query method includes the steps that a query request can be sent to a vehicle background management server based on an applet end, so that the vehicle background management server can query violation information of a vehicle to be queried from stored vehicle violation information.

And b3, displaying violation information of the vehicle to be inquired at the applet end.

After the vehicle background management server inquires the violation information of the vehicle to be inquired, the violation information can be fed back to the small program end through the vehicle background management server, and the violation information of the vehicle to be inquired is displayed through the small program end.

As shown in fig. 5, for example, based on the fact that the user inputs the license plate number of the vehicle to be queried, the applet terminal sends the license plate number "X123" of the vehicle to be queried to the vehicle backend management server to initiate a query request of violation information of the vehicle to be queried to the vehicle backend management server. After receiving an inquiry request sent by the small program terminal, the vehicle background management server inquires the violation information of the vehicle to be inquired from the stored vehicle violation information (for example, the violation information comprises vehicle illegal parking, license plate number X123 and violation place 'people square') based on the license plate number X123 of the vehicle to be inquired, and feeds the violation information of the vehicle to be inquired back to the small program terminal, so that the violation information of the vehicle to be inquired can be displayed on the small program terminal, and a driver of the vehicle to be inquired can know the traffic violation behavior of the driver in time, and further the traffic behavior is standardized.

In order to better implement the vehicle image display method in the embodiment of the present application, on the basis of the vehicle image display method, an embodiment of the present application further provides a vehicle image display apparatus, as shown in fig. 6, which is a schematic structural diagram of an embodiment of the vehicle image display apparatus in the embodiment of the present application, the vehicle image display apparatus 600 may be integrated in a vehicle-mounted acquisition device, an applet terminal, or a vehicle backend management server in the vehicle image display system provided in the embodiment of the present application, and the vehicle image display apparatus 600 includes:

the connection unit 601 is configured to establish connection between the applet terminal and the vehicle-mounted acquisition device through a local area network of the vehicle-mounted acquisition device, where images acquired by an acquisition program of the vehicle-mounted acquisition device are stored in the vehicle-mounted acquisition device;

a request unit 602, configured to send an image reading request of a target vehicle to the vehicle-mounted acquisition device based on an http interface of the mini server in the acquisition program;

a sending unit 603, configured to send a target image of the target vehicle to the applet terminal, where the target image is an image of the target vehicle fed back by the vehicle-mounted acquisition device based on the image reading request;

a display unit 604, configured to display the target image at the applet end.

In some embodiments, the vehicle image display device 600 may further include an identification unit (not shown in the figure), where the identification unit is specifically configured to:

identifying vehicle behavior of the target vehicle based on the target image;

In some embodiments, the vehicle image display apparatus 600 may further include an alarm unit (not shown in the figure), and after the step of determining whether the target vehicle violates the rule according to the vehicle behavior, the alarm unit is specifically configured to:

or when the target vehicle is determined to be illegal, outputting warning information of illegal behaviors of the target vehicle through the small program terminal.

In some embodiments, the vehicle image display apparatus 600 may further include a storage unit (not shown in the figure), and the vehicle image display system further includes a vehicle background management server, where the storage unit is specifically configured to:

In some embodiments, the display unit 604 is specifically configured to:

In some embodiments, the identification unit is specifically configured to:

intercepting at least one frame of image based on the target image;

In specific implementation, the above units may be implemented as independent entities, or may be combined arbitrarily, and implemented as the same or several entities, and specific implementations of the above units may refer to the foregoing method embodiment, which is not described herein again.

Since the vehicle image display device can execute the steps of the vehicle image display method in any embodiment corresponding to fig. 1 to 5 of the present application, the beneficial effects that can be realized by the vehicle image display method in any embodiment corresponding to fig. 1 to 5 of the present application can be realized, for details, see the foregoing description, and are not repeated herein.

In addition, in order to better implement the vehicle image display method in the embodiment of the present application, based on the vehicle image display method, the embodiment of the present application further provides an electronic device, referring to fig. 7, fig. 7 shows a schematic structural diagram of the electronic device in the embodiment of the present application, specifically, the electronic device in the embodiment of the present application includes a processor 701, and when the processor 701 is used for executing a computer program stored in a memory 702, each step of the vehicle image display method in any embodiment corresponding to fig. 1 to 5 is implemented; alternatively, the processor 701 is configured to implement the functions of the units in the corresponding embodiment of fig. 6 when executing the computer program stored in the memory 702.

Illustratively, a computer program may be partitioned into one or more modules/units, which are stored in the memory 702 and executed by the processor 701 to implement embodiments of the present application. One or more modules/units may be a series of computer program instruction segments capable of performing certain functions, the instruction segments being used to describe the execution of a computer program in a computer device.

The electronic device may include, but is not limited to, a processor 701, a memory 702. It will be appreciated by those skilled in the art that the illustration is merely an example of an electronic device, and does not constitute a limitation of the electronic device, and may include more or less components than those illustrated, or combine certain components, or different components, for example, the electronic device may further include an input output device, a network access device, a bus, etc., and the processor 701, the memory 702, the input output device, the network access device, etc., are connected via the bus.

The Processor 701 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc. The general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, the processor being the control center for the electronic device and various interfaces and lines connecting the various parts of the overall electronic device.

The memory 702 may be used to store computer programs and/or modules, and the processor 701 may implement various functions of the computer apparatus by running or executing the computer programs and/or modules stored in the memory 702 and invoking data stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, application programs (such as a sound playing function, an image playing function, etc.) required by at least one function, and the like; the storage data area may store data (such as audio data, video data, etc.) created according to the use of the electronic device, etc. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.

It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the detailed working processes of the vehicle image display apparatus, the electronic device and the corresponding units thereof described above may refer to the description of the vehicle image display method in any embodiment corresponding to fig. 1 to 5, and are not described herein again in detail.

It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor.

For this reason, an embodiment of the present application provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute steps in a vehicle image display method in any embodiment corresponding to fig. 1 to 5 in the present application, and specific operations can refer to descriptions of the vehicle image display method in any embodiment corresponding to fig. 1 to 5, which are not described herein again.

Wherein the computer-readable storage medium may include: read Only Memory (ROM), random Access Memory (RAM), magnetic or optical disks, and the like.

Since the instructions stored in the computer-readable storage medium can execute the steps in the vehicle image display method according to any embodiment of the present application, such as those shown in fig. 1 to fig. 5, the beneficial effects that can be achieved by the vehicle image display method according to any embodiment of the present application, such as those shown in fig. 1 to fig. 5, can be achieved, for details, see the foregoing description, and are not repeated herein.

The foregoing detailed description is directed to a vehicle image display method, a vehicle image display system, an electronic device, and a computer-readable storage medium, which are provided by embodiments of the present application, and specific examples are applied in the detailed description to explain the principles and implementations of the present application, and the descriptions of the foregoing embodiments are only used to help understand the methods and the core ideas of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and the application scope may be changed, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. A vehicle image display method is applied to a vehicle image display system, the vehicle image display system comprises a vehicle-mounted acquisition device and an applet terminal, and the method comprises the following steps:

and displaying the target image at the applet end.

2. The vehicle image display method according to claim 1, further comprising:

identifying vehicle behavior of the target vehicle based on the target image;

3. The vehicle image display method according to claim 2, wherein the determining whether the target vehicle is in violation according to the vehicle behavior further comprises:

4. The vehicle image display method according to claim 2, wherein the vehicle image display system further comprises a vehicle background management server, and the determining whether the target vehicle violates the rule according to the vehicle behavior further comprises:

when the target vehicle is determined to be illegal, acquiring violation information of the target vehicle, wherein the violation information of the target vehicle comprises at least one of the target image, the position information of the target vehicle, the illegal behavior of the target vehicle, basic information of the target vehicle and driver information of the target vehicle;

5. The vehicle image display method according to claim 4, further comprising:

and displaying the violation information of the vehicle to be inquired at the applet end.

6. The vehicle image display method according to claim 2, wherein the identifying the vehicle behavior of the target vehicle based on the target image includes:

intercepting at least one frame of image based on the target image;

7. The vehicle image display system is characterized by comprising vehicle-mounted acquisition equipment and an applet end;

the small program end is also used for displaying the target image.

8. The vehicle image display system according to claim 7, wherein the applet side or the vehicle-mounted collection device is further configured to identify a vehicle behavior of the target vehicle according to the target image, and determine whether the target vehicle violates a rule based on the vehicle behavior;

9. An electronic device comprising a processor and a memory, wherein the memory stores a computer program, and the processor executes the vehicle image display method according to any one of claims 1 to 6 when calling the computer program in the memory.

10. A computer-readable storage medium, having stored thereon a computer program, which is loaded by a processor, to execute the steps in the vehicle image display method according to any one of claims 1 to 6.