WO2022166625A1 - 一种车辆行驶场景中信息推送的方法以及相关装置 - Google Patents

一种车辆行驶场景中信息推送的方法以及相关装置 Download PDF

Info

Publication number
WO2022166625A1
WO2022166625A1 PCT/CN2022/073344 CN2022073344W WO2022166625A1 WO 2022166625 A1 WO2022166625 A1 WO 2022166625A1 CN 2022073344 W CN2022073344 W CN 2022073344W WO 2022166625 A1 WO2022166625 A1 WO 2022166625A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
scene
push
vehicle
driving
Prior art date
Application number
PCT/CN2022/073344
Other languages
English (en)
French (fr)
Inventor
孙中阳
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Publication of WO2022166625A1 publication Critical patent/WO2022166625A1/zh
Priority to US17/952,466 priority Critical patent/US20230013451A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/953—Querying, e.g. by the use of web search engines
    • G06F16/9537—Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/55—Push-based network services
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20132—Image cropping
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Definitions

  • the present application relates to the field of computer technology, and in particular, to information push in vehicle driving scenarios.
  • the message is pushed during the driving process by means of real-time pushing of information.
  • the information received by different applications is pushed according to the timing of receiving the information during the driving of the vehicle.
  • the present application provides a method for information push in a vehicle driving scene, which can effectively improve the efficiency of information push in the vehicle driving scene and the safety of the driving process.
  • an embodiment of the present application provides a method for pushing information in a vehicle driving scene, which can be applied to a system or program that includes a function of information pushing in a vehicle driving scene in a terminal device, and specifically includes:
  • the image information of the driving scene collected by the vehicle-mounted image acquisition device, where the image information of the driving scene is used to record the environmental information corresponding to the vehicle in the driving scene of the vehicle;
  • scene category identification information identified based on the driving scene image information, where the scene category identification information is used to indicate the category of the environment information
  • an information push device including:
  • a receiving unit configured to receive the information to be pushed in the vehicle driving scene
  • the receiving unit is further configured to acquire the image information of the driving scene collected by the vehicle-mounted image acquisition device, and the image information of the driving scene is used to record the environmental information corresponding to the vehicle in the vehicle driving scene;
  • the receiving unit is further configured to obtain scene category identification information identified based on the driving scene image information, where the scene category identification information is used to indicate the category of the environment information;
  • a push unit configured to push the information to be pushed in the vehicle driving scene if the scene category identification information satisfies the push condition.
  • an embodiment of the present application provides a method for pushing information of a virtual application, including:
  • an information push device for a virtual application including:
  • a receiving unit configured to receive the information to be pushed in the vehicle driving scene
  • the receiving unit is further configured to obtain the image information of the driving scene collected by the vehicle-mounted image collection device;
  • an input unit configured to input the driving scene image information into a preset model, so as to identify the driving scene image information to obtain scene category identification information
  • a push unit configured to push the information to be pushed in the vehicle driving scene if the scene category identification information satisfies the push condition.
  • an embodiment of the present application provides a computer device, including: a memory, a processor, and a bus system; the memory is used to store program codes; the processor is used to execute the above aspects according to instructions in the program codes the method described.
  • a fourth aspect of the present application provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, when the computer-readable storage medium runs on a computer, the computer executes the method described in the above aspects.
  • a computer program product or computer program comprising computer instructions stored in a computer readable storage medium.
  • the processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method described in the above aspects.
  • the embodiments of the present application have the following advantages:
  • the driving scene image information collected by the in-vehicle image acquisition device is obtained, and the driving scene image information is used to record the environmental information corresponding to the vehicle in the vehicle driving scene.
  • the identification information of the scene category obtained by the identification of the image information of the driving scene. Since the scene category identification information is used to indicate the category of the environment information, if the scene category identification information satisfies the push conditions, the information to be pushed is pushed in the vehicle driving scene. In this way, the intelligent information push method that recognizes the scene where the driving object is located, and then pushes the information in the appropriate scene, ensures that the user can accurately perceive the information when the information is pushed, and greatly improves the efficiency of the information push. Distraction also does not affect safety during driving.
  • Fig. 1 is the network architecture diagram of the system operation of the information push in the vehicle driving scene
  • FIG. 2 is a flowchart structure diagram of information push in a vehicle driving scene provided by an embodiment of the present application
  • FIG. 3 is a flowchart of a method for pushing information in a vehicle driving scene provided by an embodiment of the present application
  • FIG. 4 is a scene schematic diagram of a method for information pushing in a vehicle driving scene provided by an embodiment of the present application
  • FIG. 5 is a scene schematic diagram of another method for information pushing in a vehicle driving scene provided by an embodiment of the present application.
  • FIG. 6 is a schematic scene diagram of another method for information pushing in a vehicle driving scene provided by an embodiment of the present application.
  • FIG. 7 is a flowchart of another method for pushing information in a vehicle driving scenario provided by an embodiment of the present application.
  • FIG. 8 is a flowchart of another method for pushing information of a virtual application provided by an embodiment of the present application.
  • FIG. 9 is a schematic diagram of a scenario of another information pushing method for a virtual application provided by an embodiment of the present application.
  • FIG. 10 is a schematic diagram of a scenario of another method for pushing information of a virtual application provided by an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of an information push device provided by an embodiment of the application.
  • FIG. 12 is a schematic structural diagram of an information push device for a virtual application provided by an embodiment of the present application.
  • FIG. 13 is a schematic structural diagram of a terminal device according to an embodiment of the present application.
  • FIG. 14 is a schematic structural diagram of a server according to an embodiment of the present application.
  • the embodiments of the present application provide a method and a related device for pushing information in a vehicle driving scene, which ensure that users can accurately perceive information when pushing information, so that the efficiency of information pushing is greatly improved, and distracted attention based on information pushing will not be lost. affect the safety of driving.
  • In-vehicle visual assistant refers to the application program used to recognize the driving environment in the vehicle scene. Specifically, it can use the recognition result of the driving environment to control the information push in the vehicle scene.
  • Outside the vehicle environment perception refers to the process of classifying the collected images by means of deep learning using the images captured by the vehicle camera.
  • the in-vehicle camera here includes a dash cam located in the upper center of the windshield of the vehicle or a 360-degree panoramic camera around the vehicle, which can capture the surrounding environment of the vehicle.
  • the method for pushing information in a vehicle driving scene can be applied to a system or program that includes a function of information pushing in a vehicle driving scene in a terminal device, such as a vehicle-mounted visual assistant, specifically, information pushing in a vehicle driving scene.
  • the system can run in the network architecture shown in Figure 1. As shown in Figure 1, it is the network architecture diagram of the information push system in the vehicle driving scene.
  • the information push system in the vehicle driving scene can provide The process of information push in the vehicle driving scene with multiple information sources, that is, the terminal device receives the push instruction issued by the server, and pushes and displays it in the driving object, and the process of pushing and displaying is determined by the scene where the driving object is located; it can be It should be understood that a variety of terminal devices are shown in FIG. 1 , and the terminal devices may be computer devices. The number and type depend on the actual scenario and are not limited here. In addition, Figure 1 shows one server, but in an actual scenario, multiple servers can also participate, and the specific number of servers depends on the actual scenario.
  • the server may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or may provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, and network services , cloud communications, middleware services, domain name services, security services, CDN, and cloud servers for basic cloud computing services such as big data and artificial intelligence platforms.
  • the terminal may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto.
  • the terminal and the server can be connected directly or indirectly through wired or wireless communication, and the terminal and the server can be connected to form a blockchain network, which is not limited in this application.
  • the system for pushing information in the above-mentioned vehicle driving scene can run on a mobile terminal, for example, as an application such as an in-vehicle visual assistant, on a server, or on a third-party device to provide vehicle driving scenarios.
  • information push in order to obtain the processing result of the information push in the vehicle driving scene of the information source; the information push system in the specific vehicle driving scene can be run in the above-mentioned device in the form of a program, or can also be used as a system in the above-mentioned device.
  • the system components can also be used as a kind of cloud service program.
  • the specific operation mode depends on the actual scene and is not limited here.
  • messages are pushed during driving by means of instant push information. Since many scenarios during driving may require the user to focus on the operation of the driving object, the user cannot pay attention to the instant push information, resulting in information omission, and it is easy to distract the attention of the user as a driver when viewing the pushed information, which affects the efficiency of information push in the vehicle driving scene and the safety of the driving process.
  • this application proposes a method for information push in a vehicle driving scene.
  • the method uses computer vision technology (Computer Vision, CV) to solve the problem. Further, it refers to the use of cameras and computers instead of human eyes to identify, track and measure targets, and further perform graphics processing to make computer processing images more suitable for human eyes to observe or transmit to instruments for detection.
  • Computer Vision As a scientific discipline, computer vision studies related theories and technologies, trying to build artificial intelligence systems that can obtain information from images or multidimensional data.
  • Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content/behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping It also includes common biometric identification technologies such as face recognition and fingerprint recognition.
  • Machine learning is used in the application of computer vision technology.
  • Machine learning is a multi-domain interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc. subject. It specializes in how computers simulate or realize human learning behaviors to acquire new knowledge or skills, and to reorganize existing knowledge structures to continuously improve their performance.
  • Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent, and its applications are in all fields of artificial intelligence.
  • Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, teaching learning and other technologies.
  • the method is applied to the process framework of information push in the vehicle driving scene shown in FIG. 2 .
  • FIG. 2 a flow structure diagram of the information push in the vehicle driving scene provided by the embodiment of the present application, namely On the terminal side, the outside environment of the vehicle is identified according to the captured images, so as to carry out the process of information exchange and push with the server side, that is, to select a situation with less driving burden on the driver (such as waiting for a traffic light), which will not be used.
  • Emergency services and messages are sent centrally to the driver. This not only reduces the driving safety problem when the driver is distracted by the push message, but also increases the probability that the push message is reached.
  • FIG. 3 is a flowchart of a method for information pushing in a vehicle driving scene provided by an embodiment of the present application.
  • the push method may be performed by a terminal device, and this embodiment of the present application includes at least the following steps:
  • the information to be pushed may include one or more of conversation messages, application messages, and navigation messages;
  • the information to be pushed received in the driving object is a collection of information received by the terminal device in the driving object;
  • the The terminal device corresponds to the target user, so the information to be pushed can be the received message, such as the push of the user login program (video website, music application, communication application, etc.); or the short message of IM such as in-vehicle communication; It can also be navigation information, such as lane departure warning, prompting for the start of the preceding vehicle, destination parking lot recommendation, etc.
  • the specific information form depends on the actual scene.
  • the type of the information can be judged, that is, first obtain the information corresponding to the information to be pushed.
  • Information type if the information type indicates that the information to be pushed is instant push information, the information to be pushed is immediately pushed, thereby ensuring the normal guidance process for the driving process.
  • the information type to be pushed immediately may be preset, that is, the preset push type is determined in response to the permission setting information as the instant push information; if the information type indicates that the information to be pushed is the preset push type, then Immediately push the information to be pushed, thereby improving the flexibility of information pushing and facilitating the user's control and use.
  • the image information of the driving scene is used to record the environmental information corresponding to the vehicle in the driving scene of the vehicle, that is, to record the environmental information during the driving process of the vehicle in real time; the process of collecting image information may be performed by a collection device on the vehicle , such as car cameras, driving recorders, etc.
  • FIG. 4 is a scene schematic diagram of a method for information pushing in a vehicle driving scene provided by an embodiment of the present application; the figure shows an interactive interface for information interaction And vehicle-mounted image acquisition equipment, such as a driving recorder, the driving recorder is equipped with a vehicle-mounted camera to collect vehicle environment images.
  • vehicle-mounted image acquisition equipment such as a driving recorder
  • the driving recorder is equipped with a vehicle-mounted camera to collect vehicle environment images.
  • the specific hardware composition can also be that the interactive interface is integrated in the recorder; or the recorder is integrated in the interactive interface, such as a mobile terminal. Connection, the specific hardware composition depends on the actual scene and is not limited here.
  • the acquisition parameters can be called in response to the target command, and the acquisition parameters are set based on the driving scene. It can be an instruction to turn on the acquisition device; then image acquisition is performed according to the acquisition parameters to obtain the acquired image; and then the acquired image is preprocessed to obtain the image information of the driving scene, so as to obtain a clear scene image to ensure the image recognition process. precise.
  • this part may be deleted to reduce the amount of data. That is, first preprocess the captured image to determine the driving object elements in the captured image; then, the driving object elements are cropped to obtain the cropped image; and then the parameters of the cropped image are adjusted to obtain the driving scene image information.
  • the process of preprocessing the collected images may include: for example, scaling the collected images to use a smaller and more suitable vehicle network model, etc.; or cropping the collected images, For example, cutting the engine cover and other areas that are not related to the air quality outside the vehicle, etc.; or enhancing the captured image, such as increasing the contrast, performing histogram equalization or normalization, etc.
  • the specific preprocessing process can be: The combination of one or more of the above is not limited here.
  • the image information of the driving scene is used to record the environmental information corresponding to the vehicle in the driving scene of the vehicle, the currently corresponding scene category information can be identified based on the recorded environmental information, and the scene category identification information is used to indicate the environmental information category.
  • this application does not limit the execution subject that determines the scene category identification information, which may be the aforementioned terminal device, or may be a server having a network connection with the terminal device.
  • the scene category identification information can be determined through the terminal device.
  • the terminal device can also send the driving scene image information to The server determines the scene category identification information from the server and then provides it to the terminal device.
  • This application does not limit the implementation conditions for using terminal devices or servers to identify the scene category identification information.
  • the above method of selecting a terminal device or server based on network status is only exemplary, and other conditions such as processing capabilities can also determine the implementation of scene category identification information. Execution subject for identification.
  • step 303 includes:
  • the server may identify the image information of the driving scene based on fixed features, such as signs, traffic lights, etc.; it may also input the image information of the driving scene into a first model for identification, and the first model is an image Recognition model, the recognition result of this model is used for scene classification, so it can also be called image classification model.
  • the specific model types can be VGG16, VGG19, InceptionV3, Xception, MobileNet, AlexNet, LeNet, ZF_Net, ResNet18, ResNet34, ResNet50 , ResNet_101, ResNet_152 and other image classification models, the specific type depends on the actual scene.
  • the identification result obtained by the first model may be the attribution probability of different scenes, that is, input the image information into the local preset model of the terminal device to obtain a scene score sequence, and the scene score sequence is used to indicate multiple presets.
  • the attribution probability of the scene and then determine the target item in the scene score sequence based on the attribution probability, so as to identify the image information to obtain the scene category corresponding to the driving scene.
  • the attribution probability of the parking lot scene is 0.9
  • the attribution probability of the traffic jam scene is 0.8, then the parking lot scene is determined as the scene category.
  • the process of determining the scene category may also be a binary classification process, that is, the output result is suitable for information push or not suitable for information push. Specifically, that is, first determine the target item in the scene score sequence based on the attribution probability; then determine the push category corresponding to the target item, such as pushable or unavailable push; and then determine the scene category corresponding to the driving scene according to the push category, thereby improving Efficiency of push judgment.
  • the above-mentioned model recognition process is to input the image into a discrete classification model (for example, the first model, the second model or the preset model) and output the image, and the output content is the subdivision scene represented by the shooting content , similar to traffic jams, cloudy days, rainy days, tunnels, underground parking lots, etc. It can also be a binary result, such as whether it is suitable for pushing messages.
  • a discrete classification model for example, the first model, the second model or the preset model
  • the output content is the subdivision scene represented by the shooting content , similar to traffic jams, cloudy days, rainy days, tunnels, underground parking lots, etc. It can also be a binary result, such as whether it is suitable for pushing messages.
  • the identification process can also be performed on the terminal device, such as when the network is unstable; specifically, network resource information can be obtained in real time, such as the traffic usage of the mobile terminal, network delay, etc.; then based on the network resource information
  • network resource information can be obtained in real time, such as the traffic usage of the mobile terminal, network delay, etc.; then based on the network resource information
  • determine the network status information for example, determine the network status information based on the numerical relationship between the network resource information and the preset value, such as whether the traffic usage exceeds 80% or the network delay is less than 100ms, etc.; if the network status information meets the preset conditions, it will be sent to the server.
  • Driving scene image information so that the server inputs the driving scene image information into the first model, and recognizes the driving scene image information to obtain scene category identification information.
  • the server For the process of identifying at the server, that is, sending the image information of the driving scene to the server, so that the server inputs the image information into the first model, and obtains a sequence of scene scores, and the sequence of scene scores is used to indicate the attribution probability of multiple preset scenes; Then, the scene category corresponding to the driving scene sent by the server is received, and the scene category is determined according to the target item in the scene score sequence used to indicate the attribution probability of the plurality of preset scenes.
  • the first model can be the MobileNet V3 structure.
  • MobileNetV3 is constructed based on AutoML, and then manual fine-tuning is performed to optimize the search results.
  • the search method uses platform-aware NAS and NetAdapt, which are used for global search and local search respectively. search, while manual fine-tuning adjusts the structure of the front and rear layers of the network, the bottleneck adds the SE module, and proposes a computationally efficient h-swish nonlinear activation.
  • the difference in model structure between multi-classification and binary classification where the output of the first model is discrete is limited to the number of final output categories, that is, multi-classification can divide a scene into one or more categories at the same time, such as rainy days and rainy days + traffic jams etc., and the binary classification can only output whether it is a certain scene, and the specific category form depends on the actual scene.
  • the second model is called to perform local feature extraction on the image information of the driving scene to obtain feature elements.
  • the computing resources occupied by the second model are smaller than those of the first model.
  • the second model is a lightweight model; then identify based on the feature elements to obtain the feature category, for example, determine the current road category through the identification of the lane line, when the lane line contains double solid lines, it indicates that the vehicle is on the road In the scene, it is not suitable for information push; and then the scene category identification information is determined according to the scene information corresponding to the feature category.
  • the second model performs local recognition on the image information of the driving scene, such as the recognition of lane lines, the recognition of signs, etc.
  • the data processing amount is reduced, and it can be adapted to the recognition of in-vehicle equipment to ensure that the network condition is not good. recognition accuracy.
  • a lightweight model mounted on the vehicle or mobile terminal can be calculated by depthwise separable convolution or channel shuffle operation, wherein depthwise separable convolution includes channel-by-channel convolution (Depthwise Convolution). ) and pointwise convolution (Pointwise Convolution) process; one convolution kernel of channel-by-channel convolution is responsible for one channel, and one channel is only convolved by one convolution kernel, the number of feature map channels generated by this process and the number of input channels exactly the same.
  • the lightweight model can achieve better detection results with a smaller amount of calculation.
  • model pruning and quantization operations can be performed, combined with arm assembly to accelerate the hardware platform on the terminal side.
  • the lightweight model (the second model) can be a vehicle lane line model, specifically using the MobileNet series or Shuffle Net as the front end, and the simplified yolo or full convolutional layer with skip connections as the back end to achieve light weight quantified effect.
  • the identification process on the terminal side may be performed based on the second model, or may be performed based on the first model; specifically, the model on the terminal side may be mounted in the vehicle-mounted image acquisition device, or may be mounted on the vehicle-mounted image acquisition device.
  • the specific form depends on the specific scene.
  • the scene category identification information is used to indicate the category of the environmental information, which may specifically include the scene type corresponding to the environmental information of the driving scene, or the climate type where the vehicle driving scene is located, or the push type corresponding to the vehicle driving scene (for example, whether push).
  • receiving the scene category identification information sent by the server may be performed in real time, that is, once the server recognizes and obtains the scene category identification information, it is sent so that the terminal can receive it; the scene category identification information sent by the receiving server
  • the information can also be carried out intermittently. Since the vehicle has a certain dwell time in each scene, in order to save channel resources, the scene type identification information can be periodically sent.
  • the scene category identification information satisfies the push conditions, that is, the driver is in an idle state or a waiting state in this scene, that is, a scene in a non-driving state, and the information push is performed in the above-mentioned scene, that is, the normal driving process will not be affected. It can also ensure the reach rate of information to users and improve the push efficiency.
  • the scene category identification information may include a driving environment category and a weather environment category, and the scene category identification information that satisfies the push conditions includes a scene where the driving environment category indicates that the driving object is driving safely; or a scene where driving is stopped; or a weather environment category indicates that Weather scenarios that affect the driving of the driving subject.
  • the push judgment can also be carried out in combination with the current specific driving process, that is, when the scene category meets the push conditions, the driving parameters of the vehicle in the preset duration are counted, such as the past 5 minutes. Average vehicle speed to avoid the influence of instantaneous speed changes on the judgment of push conditions; then determine the push parameters based on the numerical fluctuations corresponding to the driving parameters, that is, push when the fluctuation is gentle and the average vehicle speed is at a low value; and then generate pushes according to the push parameters
  • the sequence table is based on the push sequence table to push the information to be pushed. Through the setting of the push sequence table, the timing of the push information is digitized, thereby ensuring the accuracy of the push timing.
  • the model output scene category identification information output by the model is identified as an underground parking lot and the driving parameter identification information is zero, or the model output scene category identification information is identified as a traffic jam and the driving parameter identification indicates that the average vehicle speed within a period of time is less than a certain value Thresholds are suitable for push, and the model output scene category identification information is marked as tunnel and the driving parameter is marked as unsuitable for push when the vehicle speed is high.
  • FIG. 5 is a scene schematic diagram of another method for pushing information in a vehicle driving scene provided by an embodiment of the present application; the figure shows the navigation interface of the terminal, when the user clicks to start the push control A1 After that, the push process of the above embodiment is executed, and "whether the push information is displayed in the current scene" is performed on the navigation interface, and the user can click the push information view A2 to view the push information list when idle.
  • timings that are particularly unsuitable for push are distinguished, that is, when it is unsuitable for push, message push is not performed, and the message can be pushed during the remaining time.
  • statistics of multi-frame results can also be performed, that is, firstly obtain the scene category set output by the first model within a preset time period, and the scene category set is the preset time period.
  • the preset time period may be a fixed period of time, or may be a period of time corresponding to a certain number of frames; and then count the category distribution information of the scene categories identified by the plurality of historical scene category identification information in the scene category set, that is, different frames.
  • the corresponding category set and update the scene category identification information based on the extreme value items in the category distribution information, for example, update it to the scene category corresponding to the frame with the most votes; if the updated scene category identification information meets the push conditions, the vehicle will be counted.
  • the driving parameters within a preset time period thereby avoiding the influence of the scene category recognition result caused by the change of the instantaneous scene, and enhancing the robustness of the scene recognition process.
  • the classification model (the first model, the second model or the preset model) can vote on the output of the driving scene image information of the nearest x frames, and determine the current scene according to the principle of majority, so as to prevent distance
  • the purpose of group point interference is to ensure the accuracy of scene recognition.
  • the information to be pushed may be cached in the push pool, that is, in response to the reception of the information to be pushed in the driving object, the information corresponding to the information to be pushed is obtained Type; if the information type indicates that the information to be pushed is non-instant information, the information to be pushed will be input into the push pool cache; and when the scene category meets the push conditions, the information will be pushed based on the push pool.
  • the delivery order can be the chronological order of the input push pool, or the priority order of the messages.
  • the timing information corresponding to the information to be pushed is obtained, and the timing information is input into the timing setting of the push pool based on the information to be pushed; then the information is pushed based on the timing information.
  • the priority information corresponding to the information to be pushed is obtained, and the priority information is set based on the type of the information to be pushed; information is pushed based on the priority information, thus ensuring that The degree of restoration of information push, and the reach rate of information.
  • FIG. 6 is a scene schematic diagram of another method for information pushing in a vehicle driving scene provided by an embodiment of the present application; the figure shows that the scene is suitable for information push B1, and the terminal interface is based on priority Rank (with @ prompt) to sort and push messages.
  • the specific push can be voice reading, automatic interface display, or automatic zooming of images, etc., which is not limited here.
  • the information capacity corresponding to the push pool can also be detected; if the information capacity reaches the preset value, the information to be pushed is immediately pushed, thereby ensuring the normal operation of the terminal device.
  • the driving scene image information collected by the vehicle-mounted image acquisition device is obtained, and the driving scene image information is used to record the environmental information corresponding to the vehicle in the vehicle driving scene, and is represented by This can acquire the scene category identification information identified based on the driving scene image information. Since the scene category identification information is used to indicate the category of the environment information, if the scene category identification information satisfies the push conditions, the information to be pushed is pushed in the vehicle driving scene. In this way, the scene where the driving object is located can be recognized, and then information is pushed in the appropriate scene, which ensures that the user can perceive accurately when the information is pushed, and greatly improves the efficiency of information push. It will affect the safety during driving.
  • the recognition process of the model may also be performed on the terminal side, that is, it may be performed on the mobile terminal. It can also be in the vehicle terminal.
  • the in-vehicle device (which can be a mobile terminal or an in-vehicle terminal) does not have the conditions for performing model operations, such as weak CPU performance. communication, the collected driving scene image information can be sent to the server for processing.
  • the second model is run on the in-vehicle device to identify the scene category identification information for the driving scene image information.
  • the in-vehicle device can also communicate with the server through the network.
  • the device that recognizes the scene category identification information can be dynamically determined according to the preset rule. For example, when the traffic is sufficient and the delay is small, the image information of the driving scene can be uploaded as much as possible to the server in the cloud for processing, and the processing results can be received. When the traffic is insufficient or the accuracy of the model output is not high, the second model can be run on the in-vehicle device to reduce traffic consumption.
  • the mobile terminal and the vehicle-mounted terminal device it can be determined that the mobile phone is near the known vehicle-mounted terminal through Bluetooth, wireless hotspot connection, etc., and the identification scene shown in FIG. 5 is displayed in the mobile terminal or vehicle-mounted terminal.
  • FIG. 7 is a flowchart of a method for pushing information in a vehicle driving scene provided by an embodiment of the present application.
  • the embodiment of the present application includes at least the following steps:
  • the in-vehicle terminal can perform image acquisition through an in-vehicle image acquisition device, which is an in-vehicle image acquisition device, which collects images of the environment near the vehicle at a certain frame rate. It is required here that the image acquisition device needs to be able to capture color images on the road with sufficient clarity, the viewing angle cannot be too inclined downward or upward, and at the same time, the focal length should be as short as possible to obtain a wider range of road information.
  • the process of preprocessing the collected image by the vehicle-mounted terminal may include: for example, scaling the collected image to use a smaller model that is more suitable for vehicle-mounted; or cropping the collected image, such as Cut off areas that are not related to the air quality outside the car, such as the hood, etc.; or enhance the captured image, such as increasing the contrast, performing histogram equalization or normalization, etc.
  • the above process of image preprocessing may include the steps of digitization, geometric transformation, normalization, smoothing, restoration and enhancement.
  • the grayscale value of an image is a continuous function of spatial variables (continuous values of position)
  • the grayscale of the image can be sampled and quantized on an M ⁇ N lattice (classified as one of 2b grayscale levels)
  • M ⁇ N lattice classified as one of 2b grayscale levels
  • a digital image that can be processed by a computer can be obtained.
  • M, N and b there are certain requirements for the values of M, N and b.
  • the larger the values of M, N, and b the better the quality of the reconstructed image.
  • the sampling period is equal to or less than half of the minimum detail period in the original image
  • the spectrum of the reconstructed image is equal to that of the original image, so the reconstructed image can be exactly the same as the original image. Since the product of M, N and b determines the storage amount of an image in the computer, under the condition of a certain storage amount, it is necessary to select the appropriate M, N and b values according to the different properties of the image to obtain the best processing effect.
  • smoothing technology for a smooth process. That is, a technique for eliminating random noise in an image.
  • the basic requirement for smoothing technology is to eliminate the noise without blurring the contours or lines of the image.
  • Commonly used smoothing methods are median method, local averaging method and k-nearest neighbor averaging method.
  • the size of the local area can be fixed, or it can be changed point by point with the gray value.
  • spatial frequency domain bandpass filtering methods are sometimes applied.
  • an image enhancement system can highlight the contours of an image through a high-pass filter, allowing the machine to measure the shape and perimeter of the contours.
  • image enhancement techniques Contrast stretching, logarithmic transformation, density layering, and histogram equalization can all be used to change the grayscale of an image and highlight details. In practical application, different methods are often used, and repeated experiments can achieve satisfactory results.
  • the driving scene image information for inputting the model is obtained.
  • the vehicle-mounted terminal performs model calculation by using a local second model or a preset model.
  • the process of model calculation is to input the image into the above-mentioned model, and the output content of the above-mentioned model is the subdivision scene represented by the shooting content, similar to traffic jam, cloudy Days, rainy days, tunnels, underground parking lots, etc., can also be just a binary result (whether it is suitable for pushing messages).
  • the scene and corresponding image features are used as sample pairs to train the preset model, and the corresponding push results can also be marked to facilitate the two-classification process.
  • training data can be enhanced during training.
  • data enhancement can be performed, that is, by increasing the amount of relevant data in your data set, to ensure that the distribution of samples is uniform enough. This is due to the fact that road scenes may exist in different conditions, such as in different directions, locations, scales, brightness, etc. Therefore, these situations are solved by training the target recognition model with additional synthetic data.
  • the specific data enhancement process can be offline augmentation. This approach is suitable for smaller datasets. Eventually, the dataset will increase by a certain multiple, which is equal to the number of transformations. For example, flipping all the pictures, the data set is equivalent to multiplying by 2.
  • data augmentation can also be used online augmentation (online augmentation) or in-flight augmentation (augmentation on the fly). This approach is more suitable for larger datasets, i.e. to avoid scale that cannot withstand explosive increases.
  • mini-batch transformations are performed before input to the model, such as machine learning frameworks that support online enhancement, which can be accelerated on GPUs.
  • the process of assisting judgment may be performed in combination with driving parameters, such as vehicle speed, gear position, and handbrake status.
  • the messages acquired by the mobile terminal may be messages sent by multiple applications, and may be a set of all messages that may affect the driving process.
  • the process of message classification includes judging between immediate push and non-instant push, and the information to be pushed in real time may be navigation information or safety prompt information, and the specific information type may also be adjusted according to the user's settings.
  • the instant push message needs to be pushed immediately to ensure the normal progress of the driving process.
  • messages that are not pushed immediately that is, messages that can be pushed later, are stored in the push pool. If there are too many messages in the push pool, or the storage time of some messages exceeds the set value, the excess or time-out messages will be pushed immediately.
  • the message is pushed through the vehicle-mounted terminal, for example, the speaker or the interface display of the vehicle-mounted terminal.
  • FIG. 8 is a flowchart of a method for pushing information of a virtual application provided by an embodiment of the present application.
  • the embodiment of the present application includes at least the following steps:
  • the target operation may be a start operation on the vehicle navigation, as shown in FIG. 9 , which is a schematic diagram of a scenario of a method for pushing information of a virtual application provided by the embodiment of the application; the figure shows the vehicle Navigate the above start button C1, after clicking the start button C1, the scene information will be recognized, and the relevant recognition result will be displayed in the interactive interface.
  • the function of the virtual element with information playing therein is associated with the process of background scene recognition, that is, the play permission is occupied. That is, firstly, the application interface corresponding to the target application is started in response to the target operation; then the virtual element used for information playback in the application interface is associated with the terminal device in the driving object to start the target application.
  • FIG. 10 is a schematic scene diagram of another information push method of a virtual application provided by the embodiment of the application; the figure shows the voice push D1, but the playback function of the voice push D1 is used by the vehicle
  • the visual assistant is occupied, and the scene recognition needs to be performed in the in-vehicle visual assistant, and the playback function will be aroused only after the push judgment is obtained.
  • the target application runs normally, but the function of playing information is occupied by the vehicle-mounted visual assistant.
  • the process of executing information push is the embodiment shown in FIG. 3 or FIG. 7 , and details are not described here.
  • the deep learning technology is used to identify the pictures captured by the on-board camera to classify the current vehicle driving environment, and the permissions of related applications are managed to control the interaction process applied to the user.
  • the driver has Push messages and services centrally in an environment where there is spare effort to process information outside of driving, so as to reduce the burden on drivers and increase the efficiency of message and service push.
  • FIG. 11 is a schematic structural diagram of an information pushing apparatus provided by an embodiment of the present application.
  • the information pushing apparatus 1100 includes:
  • the receiving unit 1101 is further configured to acquire the image information of the driving scene collected by the vehicle-mounted image acquisition device, and the image information of the driving scene is used to record the environmental information corresponding to the vehicle in the vehicle driving scene;
  • the receiving unit 1101 is further configured to acquire scene category identification information identified based on the driving scene image information, where the scene category identification information is used to indicate the category of the environment information;
  • Pushing unit 1103 configured to push the information to be pushed in the vehicle driving scene if the scene category identification information satisfies the push condition.
  • the receiving unit 1101 is further configured to:
  • the scene category identification information sent by the server is received, where the scene category identification information is used to indicate the category of the environment information.
  • the receiving unit 1101 is specifically configured to send acquisition parameters to the vehicle-mounted image acquisition device in response to a target instruction, so that the vehicle-mounted image acquisition device can performing image acquisition with parameters to obtain acquired images, and the acquisition parameters are adapted to different driving scenarios;
  • the receiving unit 1101 is specifically configured to receive the captured image sent by the vehicle-mounted image capture device, and to preprocess the captured image to determine the driving object elements in the captured image;
  • the receiving unit 1101 is specifically configured to determine the pixel range corresponding to the driving object element, so as to cut the pixel range to obtain a cropped image;
  • the receiving unit 1101 is specifically configured to adjust parameters of the cropped image based on the scene characteristics of the driving scene, so as to obtain the image information of the driving scene.
  • the sending unit 1102 is specifically configured to acquire network resource information in real time
  • the sending unit 1102 is specifically configured to send the driving scene image information to a server if the network status information satisfies a preset condition, so that the server inputs the driving scene image information into the first model, and sends the information to the server.
  • the driving scene image information is identified to obtain the scene category identification information.
  • the scene category information is determined based on an attribution probability, and the attribution probability is determined according to a sequence of scene scores used to indicate attribution probabilities of multiple preset scenes. If the target item is determined, the scene score sequence is obtained by the server inputting the image information of the driving scene into the first model.
  • the sending unit 1102 is specifically configured to call the second model to perform local features on the image information of the driving scene if the network state information does not meet the preset conditions. Extracting to obtain feature elements, the computing resources occupied by the second model are smaller than those of the first model;
  • the sending unit 1102 is specifically configured to identify based on the feature elements to obtain feature categories;
  • the sending unit 1102 is specifically configured to determine the scene category identification information according to scene information corresponding to the feature category.
  • the pushing unit 1103 is specifically configured to count the driving parameters of the vehicle for a preset duration if the scene category identification information satisfies the pushing condition;
  • the pushing unit 1103 is specifically configured to determine the pushing parameters based on the numerical fluctuations corresponding to the driving parameters
  • the pushing unit 1103 is specifically configured to generate a push sequence list according to the push parameters, so as to push the information to be pushed based on the push sequence list.
  • the pushing unit 1103 is specifically configured to acquire a set of scene categories output by the first model within a preset time period, where the set of scene categories is the set of scene categories in the preset time period.
  • the pushing unit 1103 is specifically configured to count the category distribution information of the scene categories identified by the multiple historical scene category identification information in the scene category set;
  • the pushing unit 1103 is specifically configured to update the scene category identification information based on the extreme value item in the category distribution information.
  • the pushing unit 1103 is specifically configured to acquire the information type corresponding to the information to be pushed in response to receiving the information to be pushed in the vehicle;
  • the pushing unit 1103 is specifically configured to input the information to be pushed into the push pool cache if the information type indicates that the information to be pushed is non-instant information;
  • the pushing unit 1103 is specifically configured to push information based on the push pool if the scene category identification information satisfies the push condition.
  • the pushing unit 1103 is specifically configured to acquire timing information corresponding to the information to be pushed if the scene category identification information satisfies the pushing condition, and the The timing information is input into the timing setting of the push pool based on the information to be pushed;
  • the pushing unit 1103 is specifically configured to generate a push sequence based on the timing information, so as to push the information to be pushed.
  • the pushing unit 1103 is specifically configured to acquire priority information corresponding to the information to be pushed, and the priority information is set based on the type of the information to be pushed. ;
  • the push unit 1103 is specifically configured to update the push sequence based on the push level corresponding to the priority information
  • the pushing unit 1103 is further configured to push the information to be pushed based on the updated pushing sequence.
  • the pushing unit 1103 is specifically configured to acquire the information type corresponding to the information to be pushed;
  • the pushing unit 1103 is specifically configured to immediately push the information to be pushed if the information type indicates that the information to be pushed is instant push information.
  • the driving scene image information is used to record the environmental information corresponding to the vehicle in the vehicle driving scene; further sending the driving scene to the server image information, so that the server can identify the driving scene image information to obtain the scene category identification information; and then receive the scene category identification information sent by the server, and the scene category identification information is used to indicate the category of the environmental information; if the scene category identification information meets the push conditions, Then push the information to be pushed in the vehicle driving scene.
  • the process of intelligent information push based on the driving environment is realized.
  • the preset model is used to identify the scene where the driving object is located, and then the information is pushed in the appropriate scene, which ensures that the user can accurately perceive the information when the information is pushed.
  • the efficiency of push is greatly improved, and the safety of information push during driving is improved.
  • FIG. 12 is a schematic structural diagram of an apparatus for pushing information of a virtual application provided by an embodiment of the present application.
  • the interaction apparatus 1200 includes:
  • a receiving unit 1201, configured to receive the information to be pushed in the vehicle driving scene
  • the receiving unit 1201 is further configured to obtain the image information of the driving scene collected by the vehicle-mounted image collection device;
  • an input unit 1202 configured to input the image information of the driving scene into a preset model, so as to identify the image information of the driving scene to obtain scene category identification information;
  • Pushing unit 1203 configured to push the information to be pushed in the vehicle driving scene if the scene category identification information satisfies the push condition.
  • An embodiment of the present application further provides a terminal device.
  • FIG. 13 it is a schematic structural diagram of another terminal device provided by an embodiment of the present application.
  • the terminal can be any terminal device including a mobile phone, a tablet computer, a personal digital assistant (PDA), a point of sales (POS), a vehicle-mounted computer, etc.
  • the terminal is a mobile phone as an example:
  • FIG. 13 is a block diagram showing a partial structure of a mobile phone related to a terminal provided by an embodiment of the present application.
  • the mobile phone includes: a radio frequency (RF) circuit 1310, a memory 1320, an input unit 1330, a display unit 1340, a sensor 1350, an audio circuit 1360, a wireless fidelity (WiFi) module 1370, and a processor 1380 , and the power supply 1390 and other components.
  • RF radio frequency
  • the RF circuit 1310 can be used for receiving and sending signals during transmission and reception of information or during a call.
  • the processor 1380 After receiving the downlink information of the base station, it is processed by the processor 1380; in addition, it sends the designed uplink data to the base station.
  • the memory 1320 can be used to store software programs and modules, and the processor 1380 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1320 .
  • the input unit 1330 may be used for receiving inputted numerical or character information, and generating key signal input related to user setting and function control of the mobile phone.
  • the input unit 1330 may include a touch panel 1331 and other input devices 1332 .
  • the display unit 1340 may be used to display information input by the user or information provided to the user and various menus of the mobile phone.
  • the display unit 1340 may include a display panel 1341.
  • the display panel 1341 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.
  • the cell phone may also include at least one sensor 1350, such as light sensors, motion sensors, and other sensors.
  • the audio circuit 1360, the speaker 1361, and the microphone 1362 can provide an audio interface between the user and the mobile phone.
  • the audio circuit 1360 can transmit the received audio data converted electrical signals to the speaker 1361, and the speaker 1361 converts them into sound signals for output; on the other hand, the microphone 1362 converts the collected sound signals into electrical signals, and the audio circuit 1360 converts the collected sound signals into electrical signals. After receiving, it is converted into audio data, and then the audio data is output to the processor 1380 for processing, and then sent to, for example, another mobile phone through the RF circuit 1310, or the audio data is output to the memory 1320 for further processing.
  • WiFi is a short-distance wireless transmission technology.
  • the mobile phone can help users to send and receive emails, browse web pages, and access streaming media through the WiFi module 1370. It provides users with wireless broadband Internet access.
  • the processor 1380 is the control center of the mobile phone, using various interfaces and lines to connect various parts of the entire mobile phone, by running or executing the software programs and/or modules stored in the memory 1320, and calling the data stored in the memory 1320. Various functions of the phone and processing data.
  • the processor 1380 included in the terminal also has the function of executing each step of the method for pushing information in a vehicle driving scenario as described above.
  • FIG. 14 is a schematic structural diagram of a server provided by an embodiment of the present application.
  • the server 1400 may vary greatly due to different configurations or performance, and may include One or more central processing units (CPUs) 1422 (eg, one or more processors) and memory 1432, one or more storage media 1430 (eg, one or more storage media 1430) that store applications 1442 or data 1444 mass storage devices).
  • the memory 1432 and the storage medium 1430 may be short-term storage or persistent storage.
  • the program stored in the storage medium 1430 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server.
  • the central processing unit 1422 may be configured to communicate with the storage medium 1430 to execute a series of instruction operations in the storage medium 1430 on the server 1400 .
  • Server 1400 may also include one or more power supplies 1426, one or more wired or wireless network interfaces 1450, one or more input output interfaces 1458, and/or, one or more operating systems 1441, such as Windows Server TM , Mac OSX TM , Unix TM , Linux TM , FreeBSD TM and many more.
  • operating systems 1441 such as Windows Server TM , Mac OSX TM , Unix TM , Linux TM , FreeBSD TM and many more.
  • the steps performed by the information pushing apparatus in the above embodiment may be based on the server structure shown in FIG. 14 .
  • Embodiments of the present application further provide a computer-readable storage medium, where the computer-readable storage medium stores an instruction for pushing information in a vehicle driving scene, and when it runs on a computer, causes the computer to execute the operations shown in FIGS. 3 to 3 . Steps performed by the information pushing apparatus in the method described in the embodiment 10.
  • Embodiments of the present application also provide a computer program product including an instruction for pushing information in a vehicle driving scene, and when it runs on a computer, it enables the computer to execute the information in the methods described in the foregoing embodiments shown in FIGS. 3 to 10 .
  • the steps performed by the pusher are performed by the pusher.
  • the disclosed system, apparatus and method may be implemented in other manners.
  • the apparatus embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium.
  • the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , including several instructions to cause a computer device (which may be a personal computer, an information push device, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program codes .

Abstract

本申请公开了一种车辆行驶场景中信息推送的方法以及相关装置，应用于人工智能的计算机视觉技术。通过接收车辆行驶场景中的待推送信息；然后获取车载图像采集设备采集的行驶场景图像信息，由此可以获取基于所述行驶场景图像信息识别得到的场景类别识别信息。由于该场景类别识别信息用于指示所述环境信息的类别，若场景类别识别信息满足推送条件，则将待推送信息在车辆行驶场景中进行信息推送。从而实现基于行驶环境的智能信息推送的过程，从而实现采用预设模型对驾驶对象所处的场景进行识别，进而在合适的场景进行信息推送的智能信息推送方式，使得信息推送的效率大大提升，基于信息推送所分散的注意力也不会影响驾驶过程中的安全性。

Description

一种车辆行驶场景中信息推送的方法以及相关装置
本申请要求于2021年02月04日提交中国专利局、申请号为202110155620.2、申请名称为“一种车辆行驶场景中信息推送的方法以及相关装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，尤其涉及车辆行驶场景中信息推送。
背景技术
随着互联网技术的迅速发展，越来越多的应用程序出现在人们的生活中，例如在用户驾驶时使用车载应用的过程中，可以进行地图信息的接收以及相关通讯信息的接收。
相关技术中，通过即时推送信息的方式在驾驶过程中进行消息推送，例如对于不同应用所接收到的信息，在车辆行驶过程中按照接收信息的时序进行推送。
发明内容
有鉴于此，本申请提供一种车辆行驶场景中信息推送的方法，可以有效提高车辆行驶场景中信息推送的效率以及驾驶过程的安全性。
一方面，本申请实施例提供了一种车辆行驶场景中信息推送的方法，可以应用于终端设备中包含车辆行驶场景中信息推送的功能的系统或程序中，具体包括：
接收车辆行驶场景中的待推送信息；
获取车载图像采集设备采集的行驶场景图像信息，所述行驶场景图像信息用于记录车辆在所述车辆行驶场景中对应的环境信息；
获取基于所述行驶场景图像信息识别得到的场景类别识别信息，所述场景类别识别信息用于指示所述环境信息的类别；
若所述场景类别识别信息满足推送条件，则将所述待推送信息在所述车辆行驶场景中进行信息推送。
另一方面，本申请实施例提供了一种信息推送装置，包括：
接收单元，用于接收车辆行驶场景中的待推送信息；
所述接收单元，还用于获取车载图像采集设备采集的行驶场景图像信息，所述行驶场景图像信息用于记录车辆在所述车辆行驶场景中对应的环境信息；
所述接收单元，还用于获取基于所述行驶场景图像信息识别得到的场景类别识别信息，所述场景类别识别信息用于指示所述环境信息的类别；
推送单元，用于若所述场景类别识别信息满足推送条件，则将所述待推送信息在所述车辆行驶场景中进行信息推送。
另一方面，本申请实施例提供了一种虚拟应用的信息推送方法，包括：
接收车辆行驶场景中的待推送信息；
获取车载图像采集设备采集的行驶场景图像信息；
将所述行驶场景图像信息输入预设模型，以对所述行驶场景图像信息进行识别得到场景类别识别信息；
若所述场景类别识别信息满足推送条件，则将所述待推送信息在所述车辆行驶场景中 进行信息推送。
另一方面，本申请实施例提供了一种虚拟应用的信息推送装置，包括：
接收单元，用于接收车辆行驶场景中的待推送信息；
所述接收单元，还用于获取车载图像采集设备采集的行驶场景图像信息；
输入单元，用于将所述行驶场景图像信息输入预设模型，以对所述行驶场景图像信息进行识别得到场景类别识别信息；
推送单元，用于若所述场景类别识别信息满足推送条件，则将所述待推送信息在所述车辆行驶场景中进行信息推送。
又一方面，本申请实施例提供了一种计算机设备，包括：存储器、处理器以及总线系统；所述存储器用于存储程序代码；所述处理器用于根据所述程序代码中的指令执行上述方面所述的方法。
本申请第四方面提供一种计算机可读存储介质，所述计算机可读存储介质中存储有指令，当其在计算机上运行时，使得计算机执行上述方面所述的方法。
根据本申请的一个方面，提供了一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算机设备执行上述方面所述的方法。
从以上技术方案可以看出，本申请实施例具有以下优点：
若接收到车辆行驶场景中的待推送信息，获取车载图像采集设备采集的行驶场景图像信息，该行驶场景图像信息用于记录车辆在车辆行驶场景中对应的环境信息，由此可以获取基于所述行驶场景图像信息识别得到的场景类别识别信息。由于该场景类别识别信息用于指示所述环境信息的类别，若场景类别识别信息满足推送条件，则将待推送信息在车辆行驶场景中进行信息推送。从而实现对驾驶对象所处的场景进行识别，进而在合适的场景进行信息推送的智能信息推送方式，保证了信息推送时用户可以进行准确的感知，使得信息推送的效率大大提升，基于信息推送所分散的注意力也不会影响驾驶过程中的安全性。
附图说明
图1为车辆行驶场景中信息推送的系统运行的网络架构图；
图2为本申请实施例提供的一种车辆行驶场景中信息推送的流程架构图；
图3为本申请实施例提供的一种车辆行驶场景中信息推送的方法的流程图；
图4为本申请实施例提供的一种车辆行驶场景中信息推送的方法的场景示意图；
图5为本申请实施例提供的另一种车辆行驶场景中信息推送的方法的场景示意图；
图6为本申请实施例提供的另一种车辆行驶场景中信息推送的方法的场景示意图；
图7为本申请实施例提供的另一种车辆行驶场景中信息推送的方法的流程图；
图8为本申请实施例提供的另一种虚拟应用的信息推送方法的流程图；
图9为本申请实施例提供的另一种虚拟应用的信息推送方法的场景示意图；
图10为本申请实施例提供的另一种虚拟应用的信息推送方法的场景示意图；
图11为本申请实施例提供的一种信息推送装置的结构示意图；
图12为本申请实施例提供的一种虚拟应用的信息推送装置的结构示意图；
图13为本申请实施例提供的一种终端设备的结构示意图；
图14为本申请实施例提供的一种服务器的结构示意图。
具体实施方式
本申请实施例提供了一种车辆行驶场景中信息推送的方法以及相关装置，保证了信息推送时用户可以进行准确的感知，使得信息推送的效率大大提升，基于信息推送所分散的注意力也不会影响驾驶过程中的安全性。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第三”、“第四”等(如果存在)是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本申请的实施例例如能够以除了在这里图示或描述的那些以外的顺序实施。此外，术语“包括”和“对应于”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
首先，对本申请实施例中可能出现的一些名词进行解释。
车载视觉助手：指用于车辆场景中对行驶环境进行识别的应用程序，具体可以利用对行驶环境的识别结果控制车辆场景中的信息推送。
车外环境感知：指使用车载摄像头拍摄到的图像以深度学习的方式对采集的图像进行分类的过程。
车载摄像头：这里的车载摄像头包括位于车辆挡风玻璃正中靠上的行车记录仪摄像头或车身四周的360度全景摄像头等能够拍摄到车辆周围环境的摄像头。
应理解，本申请提供的车辆行驶场景中信息推送的方法可以应用于终端设备中包含车辆行驶场景中信息推送的功能的系统或程序中，例如车载视觉助手，具体的，车辆行驶场景中信息推送的系统可以运行于如图1所示的网络架构中，如图1所示，是车辆行驶场景中信息推送的系统运行的网络架构图，如图可知，车辆行驶场景中信息推送的系统可以提供与多个信息源的车辆行驶场景中信息推送的过程，即终端设备接收服务器下发的推送指令，并在驾驶对象中进行推送展示，且推送展示的过程由驾驶对象所处的场景确定；可以理解的是，图1中示出了多种终端设备，终端设备可以为计算机设备，在实际场景中可以有更多或更少种类的终端设备参与到车辆行驶场景中信息推送的过程中，具体数量和种类因实际场景而定，此处不做限定，另外，图1中示出了一个服务器，但在实际场景中，也可以有多个服务器的参与，具体服务器数量因实际场景而定。
本实施例中，服务器可以是独立的物理服务器，也可以是多个物理服务器构成的服务器集群或者分布式系统，还可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、CDN、以及大数据和人工智能平台等基础云计算服务的云服务器。终端可以是智能手机、平板电脑、笔记本电脑、台式计算机、智能音箱、智能手表等，但并不局限于此。终端以及服务器可以通过有线或无线通信方式进行直接或间接地连接，终端以及服务器可以连接组成区块链网络，本申请在此不做 限制。
可以理解的是，上述车辆行驶场景中信息推送的系统可以运行于移动终端，例如：作为车载视觉助手这样的应用，也可以运行于服务器，还可以作为运行于第三方设备以提供车辆行驶场景中信息推送的，以得到信息源的车辆行驶场景中信息推送的处理结果；具体的车辆行驶场景中信息推送的系统可以是以一种程序的形式在上述设备中运行，也可以作为上述设备中的系统部件进行运行，还可以作为云端服务程序的一种，具体运作模式因实际场景而定，此处不做限定。
在相关技术中通过即时推送信息的方式在驾驶过程中进行消息推送，由于驾驶过程中很多场景中能够需要用户将注意力集中在驾驶对象的操作上，用户无法留意到即时推送的信息，造成信息遗漏，且查看推送的信息时容易分散作为驾驶者的用户的注意力，影响车辆行驶场景中信息推送的效率以及驾驶过程的安全性。
为了解决上述问题，本申请提出了一种车辆行驶场景中信息推送的方法，该方法利用计算机视觉技术(Computer Vision，CV)解决，计算机视觉是一门研究如何使机器“看”的科学，更进一步的说，就是指用摄影机和电脑代替人眼对目标进行识别、跟踪和测量等机器视觉，并进一步做图形处理，使电脑处理成为更适合人眼观察或传送给仪器检测的图像。作为一个科学学科，计算机视觉研究相关的理论和技术，试图建立能够从图像或者多维数据中获取信息的人工智能系统。计算机视觉技术通常包括图像处理、图像识别、图像语义理解、图像检索、OCR、视频处理、视频语义理解、视频内容/行为识别、三维物体重建、3D技术、虚拟现实、增强现实、同步定位与地图构建等技术，还包括常见的人脸识别、指纹识别等生物特征识别技术。
同时，在计算机视觉技术的应用过程中使用到了机器学习(Machine Learning，ML)，机器学习是一门多领域交叉学科，涉及概率论、统计学、逼近论、凸分析、算法复杂度理论等多门学科。专门研究计算机怎样模拟或实现人类的学习行为，以获取新的知识或技能，重新组织已有的知识结构使之不断改善自身的性能。机器学习是人工智能的核心，是使计算机具有智能的根本途径，其应用遍及人工智能的各个领域。机器学习和深度学习通常包括人工神经网络、置信网络、强化学习、迁移学习、归纳学习、式教学习等技术。
具体的，该方法应用于图2所示的车辆行驶场景中信息推送的流程框架中，如图2所示，为本申请实施例提供的一种车辆行驶场景中信息推送的流程架构图，即在终端侧根据采集拍摄到的画面对车外环境进行识别，从而进行与服务器侧的信息交互与推送的过程，即从中选择出驾驶员驾驶负担较小的情形(如等待红绿灯时)将并不紧急的服务与消息集中发送给驾驶员。从而不仅可降低驾驶员分心于推送消息时的驾驶安全性问题，而且增大推送被触达的概率。
可以理解的是，本申请所提供的方法可以为一种程序的写入，以作为硬件系统中的一种处理逻辑，也可以作为一种信息推送装置，采用集成或外接的方式实现上述处理逻辑。
本申请实施例提供的方案涉及人工智能的计算机视觉技术，具体通过如下实施例进行说明：
结合上述流程架构，下面将对本申请中车辆行驶场景中信息推送的方法进行介绍，请 参阅图3，图3为本申请实施例提供的一种车辆行驶场景中信息推送的方法的流程图，该推送方法可以是由终端设备执行的，本申请实施例至少包括以下步骤：
301、接收车辆行驶场景中的待推送信息。
本实施例中，待推送信息可以包括会话消息、应用消息、导航消息中的一种或多种；驾驶对象中接收到的待推送信息即驾驶对象中的终端设备接收到的信息的集合；该终端设备对应于目标用户，故待推送信息可以是收到的消息，如用户登录程序(视频网站、音乐应用、通讯应用等)的推送；或车载通信等IM的短信；也可以是车辆自身产生的消息；还可以是导航信息，如车道偏离预警，前车启动提示，目的地停车场推荐等，具体的信息形式因实际场景而定。
可以理解的是，对于接收到的多个待推送信息，其中可能存在需要即时推送的信息，例如导航信息、安全提示等，故可以对其信息的类型进行判断，即首先获取待推送信息对应的信息类型；若信息类型指示待推送信息为即时推送信息，则立即对待推送信息进行信息推送，从而保证了对于驾驶过程的正常引导过程。
在一种可能的场景中，判断消息是否有必要立刻推送给用户，可以包括车道偏离预警、前车启动等消息等即时性强或较为重要的消息。而对于分类于无必要立刻推送的信息，如有声读物应用的会员活动，影评类应用中国最近上映的电影等并不需要马上查看，但需要较长时间浏览有商业价值的消息，可以统一在某一时刻推送。
可选的，对于立即推送的信息类型可以是预先设定的，即响应于权限设定信息确定预设推送类型，以作为即时推送信息；若信息类型指示待推送信息为预设推送类型，则立即对待推送信息进行信息推送，从而提高了信息推送的灵活性，便于用户的控制与使用。
302、获取车载图像采集设备采集的行驶场景图像信息。
本实施例中，行驶场景图像信息用于记录车辆在车辆行驶场景中对应的环境信息，即为实时记录车辆行驶过程中的环境信息；采集图像信息的过程可以是由车辆上的采集设备执行的，例如车载摄像头、行车记录仪等。
在一种可能的实现方式中，如图4所示，图4为本申请实施例提供的一种车辆行驶场景中信息推送的方法的场景示意图；图中示出了用于信息交互的交互界面以及车载图像采集设备，例如行车记录仪，该行车记录仪搭载了车载摄像头，以进行车辆环境图像的采集。
可以理解的是，具体的硬件组成还可以是交互界面集成在记录仪中；或记录仪集成在交互界面中，例如移动终端，该移动终端可通过蓝牙、无线热点连接等方式与车载图像采集设备连接，具体的硬件组成因实际场景而定，此处不做限定。
具体的，对于图像信息的采集过程，需要保证信息的准确性，故可以响应于目标指令调用采集参数，该采集参数基于行驶场景设定，该目标指令可以是用户开启信息推送控制的指令，也可以是采集设备开启的指令；然后根据采集参数进行图像采集，以得到采集图像；进而对采集图像进行预处理，以得到行驶场景图像信息，从而可以得到清晰的场景图像，以保证图像识别过程的准确。
可选的，由于车辆上的采集设备可能采集到车辆本身的相关图像元素，对于该部分可以删除，以减小数据量。即首先对采集图像进行预处理，以确定采集图像中的驾驶对象元 素；然后将驾驶对象元素进行裁剪，以得到裁剪图像；进而对裁剪图像进行参数调节，以得到行驶场景图像信息。
在一种可能的场景中，对采集到的图像进行预处理的过程可以包括：如缩放采集到的图像，以使用更小的更适合车载的网络模型等；或者对采集到的图像进行裁剪，如裁掉发动机盖等和车外空气质量无关的区域等；亦或对采集到的图像进行增强，如增大对比度，进行直方图均衡化或归一化等操作，具体的预处理过程可以是上述一种或多种的组合，此处不做限定。
303、获取基于所述行驶场景图像信息识别得到的场景类别识别信息。
由于行驶场景图像信息用于记录车辆在所述车辆行驶场景中对应的环境信息，故可以基于所记录的环境信息识别出当前对应的场景类别信息，该场景类别识别信息用于指示所述环境信息的类别。
需要注意的是，本申请不限定确定出场景类别识别信息的执行主体，可以是前述终端设备，也可以是与终端设备具有网络连接的服务器。例如在终端设备具有足够的数据处理能力时，可以通过终端设备确定出场景类别识别信息，例如在网络传输速度较佳的场景下如5G网络环境下，终端设备也可以将行驶场景图像信息发送给服务器，由服务器确定出场景类别识别信息后再提供给终端设备。本申请不限定使用终端设备或服务器识别场景类别识别信息的实施条件，上述基于网络状态选择终端设备或服务器的方式仅为示例性的，也可以通过处理能力等其他条件确定实施对场景类别识别信息进行识别的执行主体。
在一种可能的实现方式中，步骤303包括：
3031：向服务器发送行驶场景图像信息，以使服务器对行驶场景图像信息进行识别得到场景类别识别信息。
3032：接收服务器发送的场景类别识别信息
本实施例中，服务器对行驶场景图像信息进行识别可以是基于固定特征进行的，例如标志牌、红绿灯等；也可以是将行驶场景图像信息输入第一模型进行识别的，该第一模型为图像识别模型，该模型的识别结果用于场景的分类，故也可以称为图像分类模型，具体的模型种类可以是VGG16、VGG19、InceptionV3、Xception、MobileNet、AlexNet、LeNet、ZF_Net、ResNet18、ResNet34、ResNet50、ResNet_101、ResNet_152等图像分类模型，具体的类型因实际场景而定。
可选的，通过第一模型得到的识别结果可以是不同场景的归属概率，即将图像信息输入终端设备本地的预设模型，以得到场景分值序列，场景分值序列用于指示多个预设场景的归属概率；然后基于归属概率确定场景分值序列中的目标项，以对图像信息进行识别得到行驶场景对应的场景类别。例如停车场场景的归属概率为0.9，堵车场景的归属概率为0.8，则确定停车场场景为场景类别。
可选的，确定场景类别的过程还可以是二分类的过程，即输出的结果为适合信息推送或不适合信息推送。具体的，即首先基于归属概率确定场景分值序列中的目标项；然后确定目标项对应的推送类别，例如可以推送或不可用推送；进而根据推送类别确定行驶场景对应的场景类别，从而提高了推送判断的效率。
在一种可能的场景中，上述模型识别的过程即将图像输入一输出为离散量的分类模型(例如第一模型、第二模型或预设模型)，输出内容为拍摄内容所表示的细分场景，类似堵车，阴天，雨天，隧道，地下停车场等，也可以是一个二分结果，例如是否适合推送消息。
可选的，对于识别过程也可以是在终端设备执行的，例如在网络不稳定时；具体的，可以实时获取网络资源信息，例如移动终端的流量使用情况、网络延迟等；然后基于网络资源信息来确定网络状态信息，例如基于网络资源信息与预设值的数值关系确定网络状态信息，例如流量使用是否超过80％或网络延迟小于100ms等；若网络状态信息满足预设条件，则向服务器发送行驶场景图像信息，以使服务器将行驶场景图像信息输入第一模型，并对行驶场景图像信息进行识别得到场景类别识别信息。
对于在服务器识别的过程，即向服务器发送行驶场景图像信息，以使服务器将图像信息输入第一模型，并得到场景分值序列，场景分值序列用于指示多个预设场景的归属概率；然后接收服务器发送的行驶场景对应的场景类别，该场景类别是根据用于指示多个预设场景的归属概率的场景分值序列中的目标项确定的。
在一种可能的场景中，第一模型可以为MobileNet V3结构，MobileNetV3基于AutoML构建，然后进行人工微调对搜索结果进行优化，搜索方法使用了platform-aware NAS以及NetAdapt，分别用于全局搜索以及局部搜索，而人工微调则调整了网络前后几层的结构、bottleneck加入SE模块以及提出计算高效的h-swish非线性激活。第一模型输出为离散量的多分类和二分类在模型结构上的区别仅限于最终的输出类别数量，即多分类可以把某一场景同时分为一类或多类，如雨天和雨天+堵车等，而二分类仅能输出是否是某个场景，具体的类别形式因实际场景而定。
对于终端设备搭载模型的场景，即若网络状态信息不满足预设条件，则调用第二模型对行驶场景图像信息进行局部特征提取，以得到特征元素，第二模型占用的计算资源小于第一模型，即第二模型为轻量级模型；然后基于特征元素进行识别，以得到特征类别，例如通过车道线的识别确定当前所处的道路类别，当车道线包含双实线时，指示车辆处于道路场景中，不适合信息推送；进而根据特征类别对应的场景信息确定场景类别识别信息。由于第二模型对行驶场景图像信息进行局部识别，例如车道线的识别、标志牌的识别等，从而减小了数据处理量，可以适配于车载设备的识别中，保证了网络状况不佳时的识别准确性。
可以理解的是，搭载于车端或移动端的轻量级模型，可以通过深度可分离卷积或通道随机混合操作(channel shuffle)进行计算，其中深度可分离卷积包括逐通道卷积(Depthwise Convolution)和逐点卷积(Pointwise Convolution)的过程；逐通道卷积的一个卷积核负责一个通道，一个通道只被一个卷积核卷积，该过程产生的特征图通道数和输入的通道数完全一样。从而使得轻量级模型在较小的计算量下实现更好的检测效果，在实际场景还可以通过模型剪枝、量化的操作，并结合arm汇编实现对终端侧硬件平台的加速。具体的，轻量级模型(第二模型)可以是车载车道线模型，具体采用MobileNet系列或Shuffle Net等作为前端，经简化的yolo或带跳跃连接的全卷积层作为后端，以达到轻量化的效果。
可选的，对于终端侧的识别过程可以是基于第二模型进行的，也可以是基于第一模型 进行的；具体的，终端侧的模型可以是搭载于车载图像采集设备中，也可以搭载于移动终端中，具体的形式因具体场景而定。
本实施例中，场景类别识别信息用于指示环境信息的类别，具体可以包括行驶场景的环境信息对应的场景类型，或车辆行驶场景所在的气候类型，或车辆行驶场景对应的推送类型(例如是否推送)。
在一种可选的实现方式中，接收服务器发送的场景类别识别信息可以是即时进行的，即一旦服务器识别得到场景类别识别信息即进行发送，以使得终端接收；该接收服务器发送的场景类别识别信息也可以是间歇式进行的，由于车辆在每个场景中均存在一定的停留时间，为了节省信道资源，可以进行周期性发送场景类别识别信息。
304、若场景类别识别信息满足推送条件，则将待推送信息在车辆行驶场景中进行信息推送。
本实施例中，场景类别识别信息满足推送条件即在该场景下驾驶员处于空闲状态或等待状态，即非驾驶状态的场景，在上述场景下进行信息推送，即不会影响正常的驾驶过程，也可以保证信息对于用户的触达率，提高了推送效率。
具体的，场景类别识别信息可以包括行驶环境类别和天气环境类别，满足推送条件的场景类别识别信息包括行驶环境类别指示驾驶对象处于安全行驶的场景；或停止行驶的场景；或天气环境类别指示不影响驾驶对象行驶的天气场景。
可选的，除了单一的场景判断外，还可以结合当前具体的行驶过程进行推送的判定，即当场景类别满足推送条件时，则统计车辆在预设时长的行驶参数，例如过去5分钟内的平均车速，以避免瞬时速度变化对于推送条件判断的影响；然后基于行驶参数对应的数值波动情况确定推送参数，即在波动平缓，且处于低数值的平均车速时进行推送；进而根据推送参数生成推送序列表，以基于推送序列表对待推送信息进行信息推送，通过推送序列表的设定，实现了将推送信息的时机进行数值化，从而保证了推送时机的准确性。
在一种可能的场景中，模型输出的场景类别识别信息标识为地下停车场且行驶参数标识车速为零时，或模型输出场景类别识别信息标识为堵车且行驶参数标识一段时间内平均车速小于一定阈值等情况下为适合推送，模型输出场景类别识别信息标识为隧道且行驶参数标识车速较快时为不适合推送等。
具体的，如图5所示，图5为本申请实施例提供的另一种车辆行驶场景中信息推送的方法的场景示意图；图中示出了终端的导航界面，当用户点击开启推送控制A1后，即执行上述实施例的推送过程，并在导航界面上进行“当前场景是否推送信息的显示”，且用户可以在空闲时点击推送信息查看A2，进行推送信息列表的查看。
可以理解的是，基于上述实施例区分出了一些特别不适合推送的时机，即在不适合推送的时候不进行消息的推送，剩余时间均可推送。
进一步的，为了保证场景类别识别的准确性，还可以进行多帧结果的统计，即首先获取预设时间段内第一模型输出的场景类别集合，所述场景类别集合为在所述预设时间段内所述第一模型输出的多个历史场景类别识别信息。
该预设时间段可以是固定的一段时长，也可以是一定数量帧所对应的时长；然后统计 场景类别集合中所述多个历史场景类别识别信息所标识场景类别的类别分布信息，即不同帧所对应的类别集合；并基于类别分布信息中的极值项更新场景类别识别信息，例如更新为投票最多的帧所对应的场景类别；若更新后的场景类别识别信息满足推送条件，则统计车辆在预设时间段内的行驶参数，从而避免了瞬时场景的变化产生的场景类别识别结果的影响，增强了场景识别过程的鲁棒性。
在一种可能的场景中，可以对分类模型(第一模型、第二模型或预设模型)针对最近x帧的行驶场景图像信息的输出投票，按照多数原则确定当前场景等，从而达到防止离群点干扰的目的，保证场景识别的准确性。
在另一中可能的场景中，对于多个应用接收待推送信息的场景，可以将待推送信息缓存在推送池中，即响应于驾驶对象中待推送信息的接收，获取待推送信息对应的信息类型；若信息类型指示待推送信息为非即时信息，则将待推送信息输入推送池缓存；进而当场景类别满足推送条件，则基于推送池进行信息推送。
具体的，若结果显示推送则从推送池中取出全部消息并下发给用户，下发顺序可以为输入推送池的时间顺序，也可以是消息的优先级顺序等。
对于时间顺序推送的过程，即若场景类别满足推送条件，则获取待推送信息对应的时序信息，时序信息基于待推送信息输入推送池的时序设定；然后基于时序信息进行信息推送。
对于优先级顺序推送的过程，即若场景类别满足推送条件，则获取待推送信息对应的优先级信息，优先级信息基于待推送信息的类型设定；基于优先级信息进行信息推送，从而保证了信息推送的还原程度，以及信息的触达率。
具体的，如图6所示，图6为本申请实施例提供的另一种车辆行驶场景中信息推送的方法的场景示意图；图中示出了在场景适合信息推送B1，终端界面中按照优先级(有@提示)进行消息的排序推送，具体的推送可以是语音读取，也可以是自动的界面展示，还可以是图像的自动放大等，此处不做限定。
可以理解的是，为了节省终端的后台资源，还可以检测推送池对应的信息容量；若信息容量达到预设值，则立即推送待推送信息，从而保证了终端设备的正常运行。
结合上述实施例可知，若接收到车辆行驶场景中的待推送信息，获取车载图像采集设备采集的行驶场景图像信息，该行驶场景图像信息用于记录车辆在车辆行驶场景中对应的环境信息，由此可以获取基于所述行驶场景图像信息识别得到的场景类别识别信息。由于该场景类别识别信息用于指示所述环境信息的类别，若场景类别识别信息满足推送条件，则将待推送信息在车辆行驶场景中进行信息推送。从而实现对驾驶对象所处的场景进行识别，进而在合适的场景进行信息推送，保证了信息推送时用户可以进行准确的感知，使得信息推送的效率大大提升，基于信息推送所分散的注意力也不会影响驾驶过程中的安全性。
上述的一些实施例中介绍了终端设备中搭载车载视觉助手并结合服务器进行场景识别的过程，而在实际场景中，模型的识别过程也可以是在终端侧进行的，即可以是在移动终端，也可以是在车载终端。
在一种可能的场景中，车载设备(可以是移动终端或车载终端)不具备进行模型运算 的条件，如CPU性能较弱等，此时如果车载设备可以使用某些硬件与网络后台的服务器进行通讯，则可将采集到的行驶场景图像信息发送至服务器处理。
另外，若车载设备有进行模型运算的条件，或者无硬件或无条件(如流量耗尽)与服务器进行通讯，则在车载设备上运行第二模型对行驶场景图像信息进行场景类别识别信息的识别。
对应的，若车载设备有运行模型的条件，也能够通过网络与服务器通讯。此时可根据预设规则动态决定识别该场景类别识别信息的设备。例如当流量充足延迟较小时，可尽量多的将行驶场景图像信息上传到云端的服务器处理，并接收处理结果。当流量不足或对模型输出准确度要求不高，可在车载设备上运行第二模型减少流量消耗。
可以理解的是，对于移动终端与车载的终端设备之间可通过蓝牙、无线热点连接等方式判断手机在已知车载终端附近，并在移动终端或车载终端中展示图5所示的识别场景。
下面对车载终端与移动终端分别执行本申请的信息推送方案的场景进行说明，其中，作为前述终端设备的车载终端对应于模型侧的图像采集与识别，而移动终端可以与车载终端连接，对应于推送侧的信息接收与推送，下面对该场景进行说明。请参阅图7，图7为本申请实施例提供的一种车辆行驶场景中信息推送的方法的流程图，本申请实施例至少包括以下步骤：
701、图像采集。
本实施例中，车载终端可以通过车载图像采集设备进行图像采集，车载图像采集设备即车载图像采集设备，该设备以一定帧率采集车辆附近环境图像。这里要求图像采集设备需要能够拍摄到清晰度足够的路面上的彩色图像，视角不能过于倾斜向下或向上，同时具有尽量有较短的焦距，以获取更广范围的路面信息。
702、图像预处理。
本实施例中，车载终端对采集到的图像进行的预处理的过程可以包括：如缩放采集到的图像，以使用更小的更适合车载的模型等；或者对采集到的图像进行裁剪，如裁掉发动机盖等和车外空气质量无关的区域等；亦或对采集到的图像进行增强，如增大对比度，进行直方图均衡化或归一化等操作。
具体的，上述图像预处理的过程可以包括数字化、几何变换、归一化、平滑、复原和增强的步骤。
对于数字化的过程。由于一幅图像的灰度值是空间变量(位置的连续值)的连续函数，故可以在M×N点阵上对图像灰度采样并加以量化(归为2b个灰度等级之一)，可以得到计算机能够处理的数字图像。为了使数字图像能重建原来的图像，对M、N和b值的大小就有一定的要求。在接收装置的空间和灰度分辨能力范围内，M、N和b的数值越大，重建图像的质量就越好。当取样周期等于或小于原始图像中最小细节周期的一半时，重建图像的频谱等于原始图像的频谱，因此重建图像与原始图像可以完全相同。由于M、N和b三者的乘积决定一幅图像在计算机中的存储量，因此在存储量一定的条件下需要根据图像的不同性质选择合适的M、N和b值，以获取最好的处理效果。
对于几何变换的过程。用于改正图像采集系统的系统误差和仪器位置的随机误差所进 行的变换。对于卫星图像的系统误差，如地球自转、扫描镜速度和地图投影等因素所造成的畸变，可以用模型表示，并通过几何变换来消除。随机误差如飞行器姿态和高度变化引起的误差，难以用模型表示出来，所以一般是在系统误差被纠正后，通过把被观测的图和已知正确几何位置的图相比较，用图中一定数量的地面控制点解双变量多项式函数组而达到变换的目的。
对于归一化的过程。即使图像的某些特征在给定变换下具有不变性质的一种图像标准形式。图像的某些性质，例如物体的面积和周长，本来对于坐标旋转来说就具有不变的性质。在一般情况下，某些因素或变换对图像一些性质的影响可通过归一化处理得到消除或减弱，从而可以被选作测量图像的依据。例如对于光照不可控的遥感图片，灰度直方图的归一化对于图像分析是十分必要的。灰度归一化、几何归一化和变换归一化是获取图像不变性质的三种归一化方法。
对于平滑的过程。即一种消除图像中随机噪声的技术，对平滑技术的基本要求是在消去噪声的同时不使图像轮廓或线条变得模糊不清。常用的平滑方法有中值法、局部求平均法和k近邻平均法。局部区域大小可以是固定的，也可以是逐点随灰度值大小变化的。此外，有时应用空间频率域带通滤波方法。
对于复原的过程。即校正各种原因所造成的图像退化，使重建或估计得到的图像尽可能逼近于理想无退化的像场。在实际应用中常常发生图像退化现象。例如大气流的扰动，光学系统的像差，相机和物体的相对运动都会使遥感图像发生退化。基本的复原技术是把获取的退化图像g(x，y)看成是退化函数h(x，y)和理想图像f(x，y)的卷积。它们的傅里叶变换存在关系G(u，v＝H(u，v)F(u，v)。根据退化机理确定退化函数后，就可从此关系式求出F(u，v)，再用傅里叶反变换求出f(x，y)。通常把复原的过程称为反向滤波器。实际应用时，由于H(u，v)随离开uv平面原点的距离增加而迅速下降，为了避免高频范围内噪声的强化，当u2+v2大于某一界限值W时，使M(u，v)等于1。W0的选择应使H(u，v)在u2+v2≤W范围内不会出现零点。图像复原的代数方法是以最小二乘法最佳准则为基础。寻求一估值弮，使优度准则函数值最小。这种方法比较简单，可推导出最小二乘法维纳滤波器。当不存在噪声时，维纳滤波器成为理想的反向滤波器。
对于增强的过程。即对图像中的信息有选择地加强和抑制，以改善图像的视觉效果，或将图像转变为更适合于机器处理的形式，以便于数据抽取或识别。例如一个图像增强系统可以通过高通滤波器来突出图像的轮廓线，从而使机器能够测量轮廓线的形状和周长。图像增强技术有多种方法，反差展宽、对数变换、密度分层和直方图均衡等都可用于改变图像灰调和突出细节。实际应用时往往要用不同的方法，反复进行试验才能达到满意的效果。
可以理解的是，对上述预处理的方式可以使用其中的一种或多种的组合，可以采用任意的次序执行，具体的次序以及类型因实际场景而定，此处不做限定。
通过上述图像预处理，得到用于输入模型的行驶场景图像信息。
703、模型计算。
本实施例中，车载终端通过本地的第二模型或者预设模型进行模型计算，模型计算的 过程即将图像输入上述模型，上述模型的输出内容为拍摄内容所表示的细分场景，类似堵车，阴天，雨天，隧道，地下停车场等，也可以仅仅只是一个二分结果(是否适合推送消息)。
具体的，对于模型训练的过程即将场景与对应的图像特征作为样本对进行预设模型的训练，还可以标记对应的推送结果以便于二分类过程的进行。
另外，对于模型训练的过程，由于车辆行驶过程中的图像之间存在一定的关联性，故在训练时可以进行训练数据增强。
具体的，即为了保证了训练数据的全面性，可以进行数据增强，即通过增加你数据集中相关数据的数据量，保证样本的分布够均匀。这是由于道路场景可能存在于不同的条件，比如在不同的方向、位置、缩放比例、亮度等。故通过额外合成的数据来训练目标识别模型来解决这些情况。
具体的数据增强过程可以采用线下增强(offline augmentation)。这种方式适用于较小的数据集(smaller dataset)。最终会增加一定的倍数的数据集，这个倍数等于转换的个数。比如翻转所有图片，数据集相当于乘以2。
另外，数据增强也可以采用线上增强(online augmentation)或在飞行中增强(augmentation on the fly)。这种方式更适用于较大的数据集(larger datasets)，即避免无法承受爆炸性增加的规模。另外，会在输入模型之前进行小批量的转换，例如机器学习框架支持在线增强的，可以在gpu上加速。
704、辅助判断。
本实施例中，辅助判断的过程可以结合行驶参数进行，例如车速、档位、手刹状态等。
705、推送判断。
本实施例中，根据上述场景的识别以及行驶参数的综合判断，对于非驾驶场景可以进行信息推送。
706、获取消息。
本实施例中，移动终端获取的消息可以为多个应用发送的消息，可以是所有可能对驾驶过程产生影响的消息的集合。
707、消息分类。
本实施例中，消息分类的过程包括进行即时推送与非即时推送的判断，而需要即时推送的信息可以是导航信息或安全提示信息，具体的信息类型还可以因用户的设定的调整。
708、立即推送。
本实施例中，对于即时推送的消息需要立即进行推送，保证驾驶过程的正常进行。
709、推送池。
本实施例中，将非即时推送的消息即可以延后推送的消息存入推送池。如果推送池中消息过多，或有消息的存储时间超过设定值，则立即推送多出或超时的消息。
710、推送消息。
本实施例中，通过车载终端进行消息的推送例如车载终端的扬声器或界面显示。
通过上述实施例用户在驾驶负担较重时不会被推送时效性不强的消息(非即时推送的 消息)，而且这类消息被推送时用户的驾驶负担较轻，用户更有机会注意并浏览推送内容，推送更加有效。
在一种可能的场景中，车载场景中每个虚拟应用的启动均会与终端设备进行关联，从而进行应用的交互，下面对该场景进行说明。请参阅图8，图8为本申请实施例提供的一种虚拟应用的信息推送方法的流程图，本申请实施例至少包括以下步骤：
801、响应于目标操作在驾驶对象中启动目标应用。
本实施例中，目标操作可以是在车载导航上的启动操作，如图9所示，图9为本申请实施例提供的一种虚拟应用的信息推送方法的场景示意图；图中示出了车载导航上述启动按键C1，在点击启动按键C1后，即会进行场景信息的识别，并在交互界面中显示相关的识别结果。
另外，目标应用启动时，其中具有信息播放的虚拟元素的功能与后台场景识别的过程相关联，即播放权限被占用。即首先响应于目标操作启动目标应用对应的应用界面；然后将应用界面中用于信息播放的虚拟元素与驾驶对象中的终端设备进行关联，以启动目标应用。
具体的，如图10所示，图10为本申请实施例提供的另一种虚拟应用的信息推送方法的场景示意图；图中示出了语音推送D1，但是该语音推送D1的播放功能被车载视觉助手占用，需要在车载视觉助手中进行场景识别，进而得到推送判定后才会唤起播放功能。
802、接收目标应用生成的推送信息。
本实施例中，目标应用正常运行，只是信息播放的功能被车载视觉助手占用。
803、基于场景信息执行信息推送。
本实施例中，执行信息推送的过程如图3或图7所示的实施例，此处不做赘述。
本申请中使用深度学习技术对车载摄像头拍摄到的图片进行识别以分类当前车辆行驶环境的方法，对相关应用的权限进行了管理，以控制应用于用户的交互过程，在驾驶过程中驾驶员有余力处理驾驶外信息的环境下集中推送消息与服务，以达到减轻驾驶员负担，增加消息与服务推送效率的目的。
为了更好的实施本申请实施例的上述方案，下面还提供用于实施上述方案的相关装置。请参阅图11，图11为本申请实施例提供的一种信息推送装置的结构示意图，信息推送装置1100包括：
接收单元1101，用于接收车辆行驶场景中的待推送信息；
所述接收单元1101，还用于获取车载图像采集设备采集的行驶场景图像信息，所述行驶场景图像信息用于记录车辆在所述车辆行驶场景中对应的环境信息；
所述接收单元1101，还用于获取基于所述行驶场景图像信息识别得到的场景类别识别信息，所述场景类别识别信息用于指示所述环境信息的类别；
推送单元1103，用于若所述场景类别识别信息满足推送条件，则将所述待推送信息在所述车辆行驶场景中进行信息推送。
可选的，在本申请一些可能的实现方式中，所述接收单元1101还用于：
向服务器发送所述行驶场景图像信息，以使所述服务器对所述行驶场景图像信息进行 识别得到场景类别识别信息；
接收所述服务器发送的所述场景类别识别信息，所述场景类别识别信息用于指示所述环境信息的类别。
可选的，在本申请一些可能的实现方式中，所述接收单元1101，具体用于响应于目标指令向所述车载图像采集设备发送采集参数，以使得所述车载图像采集设备根据所述采集参数进行图像采集，以得到采集图像，所述采集参数适配于不同的所述行驶场景；
所述接收单元1101，具体用于接收所述车载图像采集设备发送的采集图像，并对所述采集图像进行预处理，以确定所述采集图像中的驾驶对象元素；
所述接收单元1101，具体用于确定所述驾驶对象元素对应的像素范围，以对所述像素范围进行裁剪得到裁剪图像；
所述接收单元1101，具体用于基于所述行驶场景的场景特征对所述裁剪图像进行参数调节，以得到所述行驶场景图像信息。
可选的，在本申请一些可能的实现方式中，所述发送单元1102，具体用于实时获取网络资源信息；
所述发送单元1102，具体用于若所述网络状态信息满足预设条件，则向服务器发送所述行驶场景图像信息，以使所述服务器将所述行驶场景图像信息输入第一模型，并对所述行驶场景图像信息进行识别得到所述场景类别识别信息。
可选的，在本申请一些可能的实现方式中，所述场景类别信息是基于归属概率确定的，所述归属概率是根据用于指示多个预设场景的归属概率的场景分值序列中的目标项确定的，所述场景分值序列是所述服务器将所述行驶场景图像信息输入第一模型得到的。
可选的，在本申请一些可能的实现方式中，所述发送单元1102，具体用于若所述网络状态信息不满足预设条件，则调用第二模型对所述行驶场景图像信息进行局部特征提取，以得到特征元素，所述第二模型占用的计算资源小于所述第一模型；
所述发送单元1102，具体用于基于所述特征元素进行识别，以得到特征类别；
所述发送单元1102，具体用于根据所述特征类别对应的场景信息确定所述场景类别识别信息。
可选的，在本申请一些可能的实现方式中，所述推送单元1103，具体用于若所述场景类别识别信息满足所述推送条件，则统计所述车辆在预设时长的行驶参数；
所述推送单元1103，具体用于基于所述行驶参数对应的数值波动情况确定推送参数；
所述推送单元1103，具体用于根据所述推送参数生成推送序列表，以基于所述推送序列表对所述待推送信息进行信息推送。
可选的，在本申请一些可能的实现方式中，所述推送单元1103，具体用于获取预设时间段内所述第一模型输出的场景类别集合，所述场景类别集合为在所述预设时间段内所述第一模型输出的多个历史场景类别识别信息；
所述推送单元1103，具体用于统计所述场景类别集合中所述多个历史场景类别识别信息所标识场景类别的类别分布信息；
所述推送单元1103，具体用于基于所述类别分布信息中的极值项更新所述场景类别识 别信息。
可选的，在本申请一些可能的实现方式中，所述推送单元1103，具体用于响应于在所述车辆中所述待推送信息的接收，获取所述待推送信息对应的信息类型；
所述推送单元1103，具体用于若所述信息类型指示所述待推送信息为非即时信息，则将所述待推送信息输入推送池缓存；
所述推送单元1103，具体用于若所述场景类别识别信息满足所述推送条件，则基于所述推送池进行信息推送。
可选的，在本申请一些可能的实现方式中，所述推送单元1103，具体用于若所述场景类别识别信息满足所述推送条件，则获取所述待推送信息对应的时序信息，所述时序信息基于所述待推送信息输入所述推送池的时序设定；
所述推送单元1103，具体用于基于所述时序信息生成推送序列，以对所述待推送信息进行信息推送。
可选的，在本申请一些可能的实现方式中，所述推送单元1103，具体用于获取所述待推送信息对应的优先级信息，所述优先级信息基于所述待推送信息的类型设定；
所述推送单元1103，具体用于基于所述优先级信息对应的推送等级对所述推送序列进行更新；
所述推送单元1103还用于基于更新后的所述推送序列对所述待推送信息进行信息推送。
可选的，在本申请一些可能的实现方式中，所述推送单元1103，具体用于获取所述待推送信息对应的信息类型；
所述推送单元1103，具体用于若所述信息类型指示所述待推送信息为即时推送信息，则立即对所述待推送信息进行信息推送。
通过接收车辆行驶场景中的待推送信息；然后获取车载图像采集设备采集的行驶场景图像信息，该行驶场景图像信息用于记录车辆在车辆行驶场景中对应的环境信息；进一步的向服务器发送行驶场景图像信息，以使服务器对行驶场景图像信息进行识别得到场景类别识别信息；进而接收服务器发送的场景类别识别信息，场景类别识别信息用于指示环境信息的类别；若场景类别识别信息满足推送条件，则将待推送信息在车辆行驶场景中进行信息推送。从而实现基于行驶环境的智能信息推送的过程，由于采用预设模型对驾驶对象所处的场景进行识别，进而在合适的场景进行信息推送，保证了信息推送时用户可以进行准确的感知，使得信息推送的效率大大提升，且提高了驾驶过程中信息推送的安全性。
本申请实施例还提供了一种虚拟应用的信息推送装置，请参阅图12，图12为本申请实施例提供的一种虚拟应用的信息推送装置的结构示意图，交互装置1200包括：
接收单元1201，用于接收车辆行驶场景中的待推送信息；
所述接收单元1201，还用于获取车载图像采集设备采集的行驶场景图像信息；
输入单元1202，用于将所述行驶场景图像信息输入预设模型，以对所述行驶场景图像信息进行识别得到场景类别识别信息；
推送单元1203，用于若所述场景类别识别信息满足推送条件，则将所述待推送信息在所述车辆行驶场景中进行信息推送。
本申请实施例还提供了一种终端设备，如图13所示，是本申请实施例提供的另一种终端设备的结构示意图，为了便于说明，仅示出了与本申请实施例相关的部分，具体技术细节未揭示的，请参照本申请实施例方法部分。该终端可以为包括手机、平板电脑、个人数字助理(personal digital assistant，PDA)、销售终端(point of sales，POS)、车载电脑等任意终端设备，以终端为手机为例：
图13示出的是与本申请实施例提供的终端相关的手机的部分结构的框图。参考图13，手机包括：射频(radio frequency，RF)电路1310、存储器1320、输入单元1330、显示单元1340、传感器1350、音频电路1360、无线保真(wireless fidelity，WiFi)模块1370、处理器1380、以及电源1390等部件。本领域技术人员可以理解，图13中示出的手机结构并不构成对手机的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
下面结合图13对手机的各个构成部件进行具体的介绍：
RF电路1310可用于收发信息或通话过程中，信号的接收和发送，特别地，将基站的下行信息接收后，给处理器1380处理；另外，将设计上行的数据发送给基站。
存储器1320可用于存储软件程序以及模块，处理器1380通过运行存储在存储器1320的软件程序以及模块，从而执行手机的各种功能应用以及数据处理。
输入单元1330可用于接收输入的数字或字符信息，以及产生与手机的用户设置以及功能控制有关的键信号输入。具体地，输入单元1330可包括触控面板1331以及其他输入设备1332。
显示单元1340可用于显示由用户输入的信息或提供给用户的信息以及手机的各种菜单。显示单元1340可包括显示面板1341，可选的，可以采用液晶显示器(liquid crystal display，LCD)、有机发光二极管(organic light-emitting diode，OLED)等形式来配置显示面板1341。
手机还可包括至少一种传感器1350，比如光传感器、运动传感器以及其他传感器。
音频电路1360、扬声器1361，传声器1362可提供用户与手机之间的音频接口。音频电路1360可将接收到的音频数据转换后的电信号，传输到扬声器1361，由扬声器1361转换为声音信号输出；另一方面，传声器1362将收集的声音信号转换为电信号，由音频电路1360接收后转换为音频数据，再将音频数据输出处理器1380处理后，经RF电路1310以发送给比如另一手机，或者将音频数据输出至存储器1320以便进一步处理。
WiFi属于短距离无线传输技术，手机通过WiFi模块1370可以帮助用户收发电子邮件、浏览网页和访问流式媒体等，它为用户提供了无线的宽带互联网访问。
处理器1380是手机的控制中心，利用各种接口和线路连接整个手机的各个部分，通过运行或执行存储在存储器1320内的软件程序和/或模块，以及调用存储在存储器1320内的数据，执行手机的各种功能和处理数据。
在本申请实施例中，该终端所包括的处理器1380还具有执行如上述车辆行驶场景中信息推送的方法的各个步骤的功能。
本申请实施例还提供了一种服务器，请参阅图14，图14是本申请实施例提供的一种服务器的结构示意图，该服务器1400可因配置或性能不同而产生比较大的差异，可以包括一 个或一个以上中央处理器(central processing units，CPU)1422(例如，一个或一个以上处理器)和存储器1432，一个或一个以上存储应用程序1442或数据1444的存储介质1430(例如一个或一个以上海量存储设备)。其中，存储器1432和存储介质1430可以是短暂存储或持久存储。存储在存储介质1430的程序可以包括一个或一个以上模块(图示没标出)，每个模块可以包括对服务器中的一系列指令操作。更进一步地，中央处理器1422可以设置为与存储介质1430通信，在服务器1400上执行存储介质1430中的一系列指令操作。
服务器1400还可以包括一个或一个以上电源1426，一个或一个以上有线或无线网络接口1450，一个或一个以上输入输出接口1458，和/或，一个或一个以上操作系统1441，例如Windows Server TM，Mac OSX TM，Unix TM，Linux TM，FreeBSD TM等等。
上述实施例中由信息推送装置所执行的步骤可以基于该图14所示的服务器结构。
本申请实施例中还提供一种计算机可读存储介质，该计算机可读存储介质中存储有车辆行驶场景中信息推送的指令，当其在计算机上运行时，使得计算机执行如前述图3至图10所示实施例描述的方法中信息推送装置所执行的步骤。
本申请实施例中还提供一种包括车辆行驶场景中信息推送的指令的计算机程序产品，当其在计算机上运行时，使得计算机执行如前述图3至图10所示实施例描述的方法中信息推送装置所执行的步骤。
本申请实施例还提供了一种车辆行驶场景中信息推送的系统，所述车辆行驶场景中信息推送的系统可以包含图11所描述实施例中的信息推送装置，或图12所描述实施例中的虚拟应用的信息推送装置，或图13所描述实施例中的终端设备，或者图14所描述的服务器。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可 以是个人计算机，信息推送装置，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read-only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。

Claims (17)

  1. 一种车辆行驶场景中信息推送的方法，所述方法由终端设备执行，所述方法包括：
    接收车辆行驶场景中的待推送信息；
    获取车载图像采集设备采集的行驶场景图像信息，所述行驶场景图像信息用于记录车辆在所述车辆行驶场景中对应的环境信息；
    获取基于所述行驶场景图像信息识别得到的场景类别识别信息，所述场景类别识别信息用于指示所述环境信息的类别；
    若所述场景类别识别信息满足推送条件，则将所述待推送信息在所述车辆行驶场景中进行信息推送。
  2. 根据权利要求1所述的方法，所述获取基于所述行驶场景图像信息识别得到的场景类别识别信息，所述场景类别识别信息用于指示所述环境信息的类别，包括：
    向服务器发送所述行驶场景图像信息，以使所述服务器对所述行驶场景图像信息进行识别得到场景类别识别信息；
    接收所述服务器发送的所述场景类别识别信息，所述场景类别识别信息用于指示所述环境信息的类别。
  3. 根据权利要求1所述的方法，所述获取车载图像采集设备采集的行驶场景图像信息，包括：
    响应于目标指令向所述车载图像采集设备发送采集参数，以使得所述车载图像采集设备根据所述采集参数进行图像采集，以得到采集图像，所述采集参数适配于不同的所述行驶场景；
    接收所述车载图像采集设备发送的采集图像，并对所述采集图像进行预处理，以确定所述采集图像中的驾驶对象元素；
    确定所述驾驶对象元素对应的像素范围，以对所述像素范围进行裁剪得到裁剪图像；
    基于所述行驶场景的场景特征对所述裁剪图像进行参数调节，以得到所述行驶场景图像信息。
  4. 根据权利要求2所述的方法，所述向服务器发送所述行驶场景图像信息，以使所述服务器对所述行驶场景图像信息进行识别得到场景类别识别信息，包括：
    实时获取网络资源信息；
    若所述网络状态信息满足预设条件，则向服务器发送所述行驶场景图像信息，以使所述服务器将所述行驶场景图像信息输入第一模型，并对所述行驶场景图像信息进行识别得到所述场景类别识别信息。
  5. 根据权利要求4所述的方法，所述场景类别信息是基于归属概率确定的，所述归属概率是根据用于指示多个预设场景的归属概率的场景分值序列中的目标项确定的，所述场景分值序列是所述服务器将所述行驶场景图像信息输入第一模型得到的。
  6. 根据权利要求4所述的方法，所述方法还包括：
    若所述网络状态信息不满足预设条件，则调用第二模型对所述行驶场景图像信息进行局部特征提取，以得到特征元素，所述第二模型占用的计算资源小于所述第一模型；
    基于所述特征元素进行识别，以得到特征类别；
    根据所述特征类别对应的场景信息确定所述场景类别识别信息。
  7. 根据权利要求1所述的方法，所述若所述场景类别识别信息满足推送条件，则将所述待推送信息在所述车辆行驶场景中进行信息推送，包括：
    若所述场景类别识别信息满足所述推送条件，则统计所述车辆在预设时长的行驶参数；
    基于所述行驶参数对应的数值波动情况确定推送参数；
    根据所述推送参数生成推送序列表，以基于所述推送序列表对所述待推送信息进行信息推送。
  8. 根据权利要求1所述的方法，所述方法还包括：
    获取预设时间段内所述第一模型输出的场景类别集合，所述场景类别集合为在所述预设时间段内所述第一模型输出的多个历史场景类别识别信息；
    统计所述场景类别集合中所述多个历史场景类别识别信息所标识场景类别的类别分布信息；
    基于所述类别分布信息中的极值项更新所述场景类别识别信息。
  9. 根据权利要求1所述的方法，所述方法还包括：
    响应于在所述车辆行驶场景中所述待推送信息的接收，获取所述待推送信息对应的信息类型；
    若所述信息类型指示所述待推送信息为非即时信息，则将所述待推送信息输入推送池缓存；
    所述若所述场景类别识别信息满足推送条件，则将所述待推送信息在所述车辆行驶场景中进行信息推送，包括：
    若所述场景类别识别信息满足所述推送条件，则基于所述推送池进行信息推送。
  10. 根据权利要求9所述的方法，所述若所述场景类别满足所述推送条件，则基于所述推送池进行信息推送，包括：
    若所述场景类别识别信息满足所述推送条件，则获取所述待推送信息对应的时序信息，所述时序信息基于所述待推送信息输入所述推送池的时序设定；
    基于所述时序信息生成推送序列，以对所述待推送信息进行信息推送。
  11. 根据权利要求10所述的方法，所述基于所述时序信息生成推送序列，以对所述待推送信息进行信息推送，包括：
    获取所述待推送信息对应的优先级信息，所述优先级信息基于所述待推送信息的类型设定；
    基于所述优先级信息对应的推送等级对所述推送序列进行更新；
    基于更新后的所述推送序列对所述待推送信息进行信息推送。
  12. 根据权利要求1-11任一项所述的方法，所述方法还包括：
    获取所述待推送信息对应的信息类型；
    若所述信息类型指示所述待推送信息为即时推送信息，则立即对所述待推送信息进行信息推送。
  13. 根据权利要求1所述的方法，所述场景类别识别信息包括行驶环境类别和天气环境类别，满足所述推送条件的场景类别识别信息包括所述行驶环境类别指示车辆处于安全行驶的场景或所述天气环境类别指示不影响车辆行驶的天气场景。
  14. 一种车辆行驶场景中信息推送的装置，包括：
    接收单元，用于接收车辆行驶场景中的待推送信息；
    所述接收单元，还用于获取车载图像采集设备采集的行驶场景图像信息，所述行驶场景图像信息用于记录车辆在所述车辆行驶场景中对应的环境信息；
    发送单元，用于向服务器发送所述行驶场景图像信息，以使所述服务器对所述行驶场景图像信息进行识别得到场景类别识别信息；
    所述接收单元，还用于接收所述服务器发送的所述场景类别识别信息，所述场景类别识别信息用于指示所述环境信息的类别；
    推送单元，用于若所述场景类别识别信息满足推送条件，则将所述待推送信息在所述车辆行驶场景中进行信息推送。
  15. 一种计算机设备，所述计算机设备包括处理器以及存储器：
    所述存储器用于存储程序代码；所述处理器用于根据所述程序代码中的指令执行权利要求1至13任一项所述的车辆行驶场景中信息推送的方法。
  16. 一种计算机可读存储介质，所述计算机可读存储介质中存储有指令，当其在计算机上运行时，使得计算机执行上述权利要求1至13任一项所述的车辆行驶场景中信息推送的方法。
  17. 一种包括指令的计算机程序产品，当其在计算机上运行时，使得所述计算机执行权利要求1至13任一项所述的车辆行驶场景中信息推送的方法。
PCT/CN2022/073344 2021-02-04 2022-01-24 一种车辆行驶场景中信息推送的方法以及相关装置 WO2022166625A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/952,466 US20230013451A1 (en) 2021-02-04 2022-09-26 Information pushing method in vehicle driving scene and related apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110155620.2 2021-02-04
CN202110155620.2A CN114936330A (zh) 2021-02-04 2021-02-04 一种车辆行驶场景中信息推送的方法以及相关装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/952,466 Continuation US20230013451A1 (en) 2021-02-04 2022-09-26 Information pushing method in vehicle driving scene and related apparatus

Publications (1)

Publication Number Publication Date
WO2022166625A1 true WO2022166625A1 (zh) 2022-08-11

Family

ID=82741957

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/073344 WO2022166625A1 (zh) 2021-02-04 2022-01-24 一种车辆行驶场景中信息推送的方法以及相关装置

Country Status (3)

Country Link
US (1) US20230013451A1 (zh)
CN (1) CN114936330A (zh)
WO (1) WO2022166625A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115391054A (zh) * 2022-10-27 2022-11-25 宁波均联智行科技股份有限公司 车机系统的资源分配方法及车机系统
CN115665699A (zh) * 2022-12-27 2023-01-31 博信通信股份有限公司 一种多场景的信号覆盖优化方法、装置、设备及介质

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116055553A (zh) * 2023-01-09 2023-05-02 中国第一汽车股份有限公司 车辆推送信息的处理方法、系统和装置
CN115861974B (zh) * 2023-02-24 2023-05-09 成都宜泊信息科技有限公司 停车场无牌车通行管理方法、系统、电子设备及存储介质
CN116434601B (zh) * 2023-04-25 2023-12-29 广州市单元信息科技有限公司 一种消息推送的智能化处理方法及系统
CN116256721B (zh) * 2023-05-11 2023-07-28 南京隼眼电子科技有限公司 通道场景模式判断方法、系统、电子设备及介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107306281A (zh) * 2016-04-20 2017-10-31 斑马网络技术有限公司 消息的推送方法、装置和设备
CN109936664A (zh) * 2017-12-15 2019-06-25 腾讯科技（深圳）有限公司 消息提醒方法、装置及存储介质
CN110060477A (zh) * 2019-04-28 2019-07-26 上海掌门科技有限公司 用于推送信息的方法和装置
US20200143175A1 (en) * 2018-11-06 2020-05-07 Fujitsu Limited Scenario detection apparatus and method
CN111131422A (zh) * 2019-12-17 2020-05-08 上海博泰悦臻电子设备制造有限公司 一种推送消息提示方法、车载终端及计算机存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107306281A (zh) * 2016-04-20 2017-10-31 斑马网络技术有限公司 消息的推送方法、装置和设备
CN109936664A (zh) * 2017-12-15 2019-06-25 腾讯科技（深圳）有限公司 消息提醒方法、装置及存储介质
US20200143175A1 (en) * 2018-11-06 2020-05-07 Fujitsu Limited Scenario detection apparatus and method
CN110060477A (zh) * 2019-04-28 2019-07-26 上海掌门科技有限公司 用于推送信息的方法和装置
CN111131422A (zh) * 2019-12-17 2020-05-08 上海博泰悦臻电子设备制造有限公司 一种推送消息提示方法、车载终端及计算机存储介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115391054A (zh) * 2022-10-27 2022-11-25 宁波均联智行科技股份有限公司 车机系统的资源分配方法及车机系统
CN115665699A (zh) * 2022-12-27 2023-01-31 博信通信股份有限公司 一种多场景的信号覆盖优化方法、装置、设备及介质

Also Published As

Publication number Publication date
US20230013451A1 (en) 2023-01-19
CN114936330A (zh) 2022-08-23

Similar Documents

Publication Publication Date Title
WO2022166625A1 (zh) 一种车辆行驶场景中信息推送的方法以及相关装置
US20220351511A1 (en) Systems and methods for augmented reality navigation
CN113762252B (zh) 无人机智能跟随目标确定方法、无人机和遥控器
CN112329725B (zh) 一种道路场景的要素识别方法、装置、设备以及存储介质
US20190057244A1 (en) Method for determining target through intelligent following of unmanned aerial vehicle, unmanned aerial vehicle and remote control
US11443438B2 (en) Network module and distribution method and apparatus, electronic device, and storage medium
CN111444826A (zh) 视频检测方法、装置、存储介质及计算机设备
CN110942071A (zh) 一种基于车牌分类和lstm的车牌识别方法
CN111310770A (zh) 目标检测方法和装置
JP2020536332A (ja) キーフレームスケジューリング方法及び装置、電子機器、プログラム並びに媒体
CN113033677A (zh) 视频分类方法、装置、电子设备和存储介质
CN114092920B (zh) 一种模型训练的方法、图像分类的方法、装置及存储介质
CN110956483A (zh) 广告信息推送方法、系统、服务器及车辆
CN113076889B (zh) 集装箱铅封识别方法、装置、电子设备和存储介质
CN113688839A (zh) 视频处理方法及装置、电子设备、计算机可读存储介质
CN111967332B (zh) 用于自动驾驶的能见度信息生成方法和装置
CN116823884A (zh) 多目标跟踪方法、系统、计算机设备及存储介质
CN113673527B (zh) 一种车牌识别方法及系统
CN114359789A (zh) 视频图像的目标检测方法、装置、设备及介质
CN115439692A (zh) 一种图像处理方法、装置、电子设备及介质
CN113469250A (zh) 图像拍摄方法、图像分类模型训练方法、装置及电子设备
CN114283087A (zh) 一种图像去噪方法及相关设备
CN115049895B (zh) 一种图像属性识别方法、属性识别模型训练方法及装置
CN115131562B (zh) 三维场景分割方法、模型训练方法、装置和电子设备
CN114723611B (zh) 图像重建模型训练方法、重建方法、装置、设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22748904

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 11.12.2023)