CN112348879A - Vehicle operation control method and device, electronic equipment and storage medium - Google Patents

Vehicle operation control method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN112348879A
CN112348879A CN202011197188.5A CN202011197188A CN112348879A CN 112348879 A CN112348879 A CN 112348879A CN 202011197188 A CN202011197188 A CN 202011197188A CN 112348879 A CN112348879 A CN 112348879A
Authority
CN
China
Prior art keywords
image
vehicle
target
preset
color space
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011197188.5A
Other languages
Chinese (zh)
Other versions
CN112348879B (en
Inventor
刘业鹏
赵雨佳
程骏
庞建新
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ubtech Robotics Corp
Original Assignee
Ubtech Robotics Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ubtech Robotics Corp filed Critical Ubtech Robotics Corp
Priority to CN202011197188.5A priority Critical patent/CN112348879B/en
Priority to CN202311422306.1A priority patent/CN117409070A/en
Publication of CN112348879A publication Critical patent/CN112348879A/en
Application granted granted Critical
Publication of CN112348879B publication Critical patent/CN112348879B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R16/00—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for
    • B60R16/02—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
    • B60W40/04—Traffic conditions
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B63—SHIPS OR OTHER WATERBORNE VESSELS; RELATED EQUIPMENT
    • B63B—SHIPS OR OTHER WATERBORNE VESSELS; EQUIPMENT FOR SHIPPING 
    • B63B49/00—Arrangements of nautical instruments or navigational aids
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64D—EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
    • B64D43/00—Arrangements or adaptations of instruments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Mechanical Engineering (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Automation & Control Theory (AREA)
  • Ocean & Marine Engineering (AREA)
  • Combustion & Propulsion (AREA)
  • Chemical & Material Sciences (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Transportation (AREA)
  • Mathematical Physics (AREA)
  • Geometry (AREA)
  • Traffic Control Systems (AREA)

Abstract

The invention is suitable for the technical field of intelligent transportation, and provides a transportation vehicle operation control method, a transportation vehicle operation control device, electronic equipment and a storage medium, wherein images in a preset visual field range in the transportation vehicle operation direction are acquired; preprocessing the image to obtain size information and state information of a target in the image; detecting whether the distance between the vehicle and the target is within a preset safe distance range or not according to the size information of the target in the image; when the distance between the vehicle and the target is within a preset safe distance range, the running state of the vehicle at the next moment is controlled according to the current running state of the vehicle and the state information of the target in the image, and the vehicle can be controlled to run safely only by acquiring the image in the running direction of the vehicle and processing the image, so that the vehicle can be provided with only a camera and a processor, the hardware structure is simple, and the calculation amount is small.

Description

Vehicle operation control method and device, electronic equipment and storage medium
Technical Field
The invention belongs to the technical field of intelligent transportation, and particularly relates to a transportation operation control method, a transportation operation control device, electronic equipment and a storage medium.
Background
The Intelligent Transportation (Intelligent Transportation) is an advanced scientific technology such as information technology, computer technology, data communication technology, sensor technology, electronic control technology, automatic control theory, operational research, artificial intelligence and the like, is effectively and comprehensively applied to the fields of Transportation, service control and vehicle manufacturing to strengthen the relation among Intelligent Transportation tools, roads and users, thereby forming a comprehensive Transportation system which ensures safety, improves efficiency, improves environment and saves energy. The existing intelligent transportation tool usually needs to carry various hardware devices such as a distance measuring sensor, an obstacle avoidance sensor, a camera and a processor, and adopts a corresponding algorithm to control the safe operation of the intelligent transportation tool, so that the hardware structure of the intelligent transportation tool is complex and a large amount of software computing resources are consumed.
Disclosure of Invention
In view of this, embodiments of the present invention provide a vehicle, an operation control method and apparatus thereof, and a storage medium, so as to solve the problems in the prior art that an intelligent vehicle has a complex hardware structure and needs to consume a large amount of software computing resources.
A first aspect of an embodiment of the present invention provides a vehicle operation control method, including:
acquiring an image in a preset visual field range in the running direction of a vehicle;
preprocessing the image to obtain size information and state information of a target in the image;
detecting whether the distance between the vehicle and the target is within a preset safe distance range or not according to the size information of the target in the image;
and when the distance between the vehicle and the target is within a preset safe distance range, controlling the running state of the vehicle at the next moment according to the current running state of the vehicle and the state information of the target in the image.
A second aspect of an embodiment of the present invention provides a vehicle operation control device including:
the image acquisition module is used for acquiring images in a preset visual field range in the running direction of the vehicle;
the image preprocessing module is used for preprocessing the image to obtain the size information and the state information of the target in the image;
the distance detection module is used for detecting whether the distance between the vehicle and the target is within a preset safe distance range according to the size information of the target in the image;
and the state control module is used for controlling the running state of the vehicle at the next moment according to the current running state of the vehicle and the state information of the target in the image when the distance between the vehicle and the target is within a preset safe distance range.
A third aspect of the embodiments of the present invention provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and further includes a camera or is connected to the camera in communication, and the processor implements the steps of the vehicle operation control method according to the first aspect of the embodiments of the present invention when executing the computer program.
A fourth aspect of embodiments of the present invention provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the steps of the vehicle operation control method according to the first aspect of embodiments of the present invention.
According to the vehicle operation control method provided by the first aspect of the embodiment of the invention, images in a preset visual field range in the vehicle operation direction are acquired; preprocessing the image to obtain size information and state information of a target in the image; detecting whether the distance between the vehicle and the target is within a preset safe distance range or not according to the size information of the target in the image; when the distance between the vehicle and the target is within a preset safe distance range, the running state of the vehicle at the next moment is controlled according to the current running state of the vehicle and the state information of the target in the image, and the vehicle can be controlled to run safely only by acquiring the image in the running direction of the vehicle and processing the image, so that the vehicle can be provided with only a camera and a processor, the hardware structure is simple, and the calculation amount is small.
It is understood that the beneficial effects of the second to fourth aspects can be seen from the description of the first aspect, and are not described herein again.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed for the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a first schematic flow chart of a vehicle operation control method according to an embodiment of the present invention;
FIG. 2 is a second flowchart of a vehicle operation control method according to an embodiment of the present invention;
FIG. 3 is a third flowchart illustrating a method for controlling the operation of a vehicle according to an embodiment of the present invention;
FIG. 4 is a dimensional relationship between an image and a first target area and a traffic light in the image provided by an embodiment of the present invention;
FIG. 5 is a fourth flowchart illustrating a vehicle operation control method according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a vehicle operation control device according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to" determining "or" in response to detecting ". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".
Furthermore, in the description of the present invention and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.
Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present invention. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise.
The vehicle operation control method provided by the embodiment of the invention is applied to electronic equipment which can be arranged on a vehicle. The Vehicle may be an automobile (e.g., car, bus, motorcycle, etc.), an Automated Guided Vehicle (AGV), a bicycle, an electric Vehicle (e.g., electric car, electric motorcycle, electric scooter), a boat, an unmanned aerial Vehicle, a train, etc. Depending on the type of vehicle, the electronic device may be an onboard device, a portable device, a marine device, etc., and the electronic device may include a control board, such as an Advanced reduced instruction set processor (ARM) development board. The embodiments of the present invention do not set any limit to the specific types of electronic devices and vehicles.
As shown in fig. 1, a vehicle operation control method according to an embodiment of the present invention includes the following steps S101 to S104:
s101, acquiring an image in a preset view field range in the running direction of a vehicle;
step S102, preprocessing the image to obtain size information and state information of a target in the image;
step S103, detecting whether the distance between the vehicle and the target is within a preset safe distance range according to the size information of the target in the image;
and step S104, when the distance between the vehicle and the target is within a preset safe distance range, controlling the running state of the vehicle at the next moment according to the current running state of the vehicle and the state information of the target in the image.
In application, the preset visual field range may include any angle range within a 0-720 ° visual field range centered on the vehicle; the preset visual field range can also comprise a 0-720-degree visual field range taking the vehicle as a center, and each obtained frame of image is a 720-degree panoramic image around the vehicle; the preset visual field range can also comprise a 0-360-degree visual field range taking the vehicle as a center, and each acquired image is a 360-degree panoramic image surrounding the vehicle. The preset visual field range is such that when the distance between the vehicle and the target is within the preset safe distance range, each obtained image comprises the complete target, namely when the distance between the vehicle and the target is within the preset safe distance range, the length and the width of each obtained image are respectively greater than or equal to the length and the width of the target in the image. The preset visual field range can be set according to the position of the target relative to the vehicle and the size of the target, and the preset visual field range can be adjusted according to different targets.
In one embodiment, before step S101, the method includes:
setting the size of a preset visual field range according to the position of a target relative to a vehicle and the size of the target, so that when the distance between the vehicle and the target is within a preset safety distance range, the target is located within the preset visual field range.
In application, the target may be any moving target or static target located on the running route of the vehicle, the moving target refers to a target whose position on the running route of the vehicle changes with time, the static target refers to a target whose position on the running route of the vehicle is fixed, for example, other vehicles, pedestrians, etc. are moving targets, and traffic lights, obstacles, etc. are static targets.
In application, the preprocessing of the image mainly comprises the steps of carrying out target detection on the image, and acquiring size information and state information of a target in the image when the target is detected. The size information mainly includes the length, width, and area of the object in the image, and the coordinates of the boundary point on the object in the image, and the like, and the length, width, and area can be obtained based on the coordinates. For the moving object, the state information includes moving parameters such as the running state (e.g., acceleration running, deceleration running, stop running), running direction, and the like of the moving object. For a static object, the state information includes characteristic parameters such as color and form of the static object itself. For a moving target, multiple frames of images need to be continuously acquired, and then each frame of image is preprocessed respectively, so as to obtain state information of the target according to the change condition of the size information of the moving target in the multiple frames of images. For a static target, only one frame of image may be acquired and preprocessed to obtain the state information of the static target, or multiple frames of images may be continuously acquired, and then each frame of image is preprocessed to obtain multiple pieces of initial state information of the static target, and the actual state information of the static target is determined according to the multiple pieces of initial state information, so as to improve the accuracy.
In application, the type of the image may be different according to the different state information that needs to be obtained, for example, when the state information includes color, the image at least needs to include RGB image, and on this basis, the image may also include depth image, grayscale image, and other images that are not related to color; when the state information does not include a color, the image may include at least one of an RGB image, a depth image, a grayscale image, and the like.
In one embodiment, the image is an RGB image, so that the vehicle operation control method can be applied to any moving object or stationary object located on the operation route of the vehicle, and since only the RGB camera needs to be equipped on the electronic device, the structure of the electronic device can be simplified and the calculation amount is small.
In application, by setting a preset safe distance range, the response time for the next operation action of the vehicle can be reserved, and the vehicle is prevented from prematurely making the next operation action. When the distance between the vehicle and the target is within the preset safety distance range, the running state of the vehicle is changed, no potential safety hazard exists, and the vehicle can run safely. For example, when the target is a moving target and the distance between the vehicle and the moving target is within a first preset safe distance range, the running state of the vehicle is changed until the running state is changed, and the vehicle can still keep a certain distance from the moving target, so that the occurrence of rubbing or collision accidents is avoided; when the target is a static target and the distance between the vehicle and the static target is within the second preset safety distance range, the running state of the vehicle is changed until the running state is changed, and the vehicle can still keep a certain distance from the static target, so that the vehicle is prevented from making wrong running actions on the static target.
In application, the operation state of the vehicle may include operation and stop operation, wherein the operation may be subdivided into uniform speed operation, deceleration operation and acceleration operation, and the stop operation may be subdivided into deceleration to 0, emergency braking, shutdown and standby.
In one embodiment, after step S104, the method includes: the process returns to step S101.
In the application, after controlling the running state of the vehicle at the next moment, the step S101 is returned to be executed to repeatedly execute the vehicle running control method, so as to realize continuous control of the running state of the vehicle.
As shown in fig. 2, in one embodiment, step S102 includes the following steps S201 to S204:
step S201, performing a zoom operation on the image to obtain an image with a preset size.
In the application, after the image is acquired, a zoom (resize) operation such as zoom-in, zoom-out, and cropping is performed on the image to zoom the image to a preset size that can be processed by the electronic device, and when a subsequent operation can directly process the image of the size, the zoom operation may not be performed, which is equivalent to setting the zoom ratio to 1.
Step S202, converting the image with the preset size into a preset color space to obtain an image in the preset color space.
In an application, after a scaling operation is performed on an image, the image is converted to an image format that can be processed by the electronic device, for example, converting the image from a BGR color space to an RGB color space. When the subsequent operation can directly process the image in the format, the format conversion operation may not be performed, which is equivalent to setting the format conversion operation as an invalid operation or a null operation.
Step S203, detecting the target in the image under the preset color space through the trained deep learning model, and obtaining the category and size information of the target in the image under the preset color space.
In application, a deep learning model is trained in advance through a large number of images containing targets, so that the deep learning model has functions of classifying state information of the targets in the images and detecting size information of the targets in the images. The category of the object mainly refers to the category of each parameter in the state information of the object. The deep learning module may be any model that can classify and size-detect objects in an image, for example, a Single Shot multi box Detector (SSD) model.
And S204, acquiring the state information of the target according to the category of the target in the image in the preset color space.
In the application, since the category of each parameter in the state information is known, it is possible to determine which parameter the state information of the object is specific to according to the category of the object.
In one embodiment, the target is a traffic light;
step S203, including:
detecting a traffic signal lamp in the image under the preset color space through the trained first deep learning model, and obtaining the color category and size information of the traffic signal lamp in the image under the preset color space;
step S204, comprising:
and acquiring the lighting color of the traffic signal lamp according to the color category of the traffic signal lamp in the image in the preset color space.
In application, when the target is a traffic light, the category is the color category of the traffic light, and the lighting color of the traffic light can be obtained based on the color category.
As shown in fig. 3, in one embodiment, based on the above embodiment related to the traffic signal, step S103 includes the following steps S301 to S303:
step S301, detecting whether the traffic signal lamp in the image in the preset color space is in a first target area according to the size information of the traffic signal lamp in the image in the preset color space;
step S302, when a traffic signal lamp in an image in a preset color space is in a first target area, detecting whether the height of the traffic signal lamp in the image in the preset color space is greater than a first preset multiple of the height of the image in the preset color space, wherein the first preset multiple is less than 1;
step S303, when the height of a traffic signal lamp in the image in the preset color space is greater than a first preset multiple of the height of the image in the preset color space, determining that the distance between the vehicle and the traffic signal lamp is within a first preset safety distance range.
In application, the first target area is a sub-area in the image, and since a certain safety distance needs to be kept between the vehicle and the traffic light, the size of the first target area is necessarily smaller than the size of the image, that is, the image contains the first target area. The traffic light is located in this subregion, which is a prerequisite for a distance between the vehicle and the traffic light within a first preset safety distance range. In order to improve the accuracy, it is further necessary to ensure that another precondition is satisfied, that is, the height of the traffic light in the image is greater than a first preset multiple of the height of the image. The first preset multiple is a critical value which enables the distance between the vehicle and the traffic light to be within a first preset safety distance range, and when the height of the traffic light in the image is smaller than or equal to the first preset multiple of the height of the image, the distance between the vehicle and the traffic light is larger than the upper limit value of the first preset safety distance range, namely the vehicle is too far away from the traffic light. The first preset multiple may be set according to actual needs, for example, 0.2.
Fig. 4 exemplarily shows the size relationship between the image 1 and the first target area 2 and the traffic signal 3 in the image 1.
In one embodiment, based on the above embodiments related to a traffic signal, the traffic signal is a traffic light;
step S104, comprising:
when the vehicle is currently running and the number of times that the lightening color of the traffic light is red is greater than the preset number of times, controlling the vehicle to stop running at the next moment;
when the vehicle is currently running and the number of times that the lightening color of the traffic light is yellow is greater than the preset number of times, controlling the vehicle to continue running at the next moment;
when the vehicle stops running currently and the number of times that the traffic light is green is greater than the preset number of times, controlling the vehicle to start running at the next moment;
and when the vehicle stops running currently and the number of times that the lightening color of the traffic light is yellow is larger than the preset number of times, controlling the vehicle to continue to stop running at the next moment.
In applications, the traffic signal is typically a traffic light comprising three colors of red, green and yellow signal light, without excluding that in other applications the traffic signal may also comprise other colors of signal light. The preset number of times is greater than or equal to 1, the preset number of times is determined by the frequency of acquiring the images, the preset number of times is in direct proportion to the frequency of acquiring the images, and when the frequency of acquiring the images is high (for example, 10 frames per second), the preset number of times should be set to be greater than 1 (for example, 3) so as to avoid the situation that the lighting color of the traffic light is detected to be one color at the beginning, and the lighting color of the traffic light is changed to other colors at the next moment, so that the vehicle runs wrongly at the next moment. The shutdown may be a shutdown and a standby or shutdown state, the startup may be a startup in a standby or shutdown state, the rapid startup may be facilitated by entering the standby state when the shutdown is stopped, and the power consumption may be reduced by entering the shutdown state when the shutdown is stopped.
In one embodiment, based on the above embodiment related to the traffic signal, step S104 further includes:
counting the number of times that the lighting color of the traffic light is red and yellow, respectively, when the vehicle is currently running;
when the vehicle stops operating at present, the number of times that the lighting color of the traffic light is green and yellow is counted, respectively.
In application, the number of times that the lighting colors of the traffic lights are red, yellow and green can be counted by different counters respectively. When the vehicle is controlled to stop running at the next moment, the counting times of the red light which is the lighting color of the traffic light is set to be 0, namely the counting value of the red light counter is set to be 0; when the vehicle is controlled to continue to operate at the next moment, the counting number of times that the lightening color of the traffic light is yellow is set to be 0, namely the counting value of the yellow light counter is set to be 0; when the vehicle is controlled to start running at the next moment, the counting times of the green lighting color of the traffic light is set to be 0, namely the counting value of the green light counter is set to be 0; when the vehicle is controlled to continue to stop operating at the next moment, the count number of times that the lighting color of the traffic light is yellow is set to 0, that is, the count value of the yellow light counter is set to 0.
In one embodiment, the target is a moving target;
step S101, including:
continuously acquiring a plurality of frames of images within a preset visual field range in the running direction of the vehicle;
step S203, including:
respectively detecting moving targets in each frame of image under the preset color space through a trained second deep learning model, and obtaining size information of the moving targets in each frame of image under the preset color space;
obtaining the size change rule of the moving target in the multi-frame image in the preset color space according to the size information of the moving target in each frame image in the preset color space;
obtaining the motion state category of the moving target according to the size change rule of the moving target in the multi-frame image in the preset color space;
step S204, comprising:
and obtaining the motion state of the motion target according to the motion state category of the motion target.
In application, when the target is a moving target, the category is a moving state category of the moving target, and the moving state of the moving target can be obtained based on the moving state category. The size change rule of the moving object can be obtained according to the size information of the moving object in each frame of image. If the vehicle is running at a constant speed currently, according to the time sequence of continuously acquiring the multi-frame images, when the size of the moving target in the continuously acquired multi-frame images is reduced at a constant speed along with the change of time, determining that the moving state of the moving target is in constant speed motion; when the size of a moving target in continuously acquired multi-frame images is accelerated and reduced along with the change of time, determining that the moving state of the moving target is accelerated; when the size of the moving target in continuously acquired multi-frame images is increased with time, determining that the moving state of the moving target is deceleration movement; when the size of the moving target in continuously acquired multi-frame images is increased at a constant speed along with the change of time, determining that the moving state of the moving target is stopped.
As shown in fig. 5, in one embodiment, based on the above embodiment related to the moving object, step S103 includes the following steps S501 to S503:
step S501, detecting whether a moving target in a last frame image in the preset color space is in a second target area according to size information of the moving target in the last frame image in the preset color space;
step S502, when the moving target in the last frame image in the preset color space is in a second target area, detecting whether the height of the moving target in the last frame image in the preset color space is greater than a second preset multiple of the height of any frame image in the preset color space, wherein the second preset multiple is less than 1;
step S503, when the height of the moving target in the last frame image in the preset color space is larger than a second preset multiple of the height of any frame image in the preset color space, determining that the distance between the vehicle and the moving target is within a second preset safety distance range.
In application, the second target area is a sub-area in the last frame of image of the multiple frames of images, and since a certain safety distance needs to be kept between the vehicle and the moving object, the size of the second target area is necessarily smaller than that of the last frame of image, that is, the last frame of image contains the second target area. The moving object is located in the sub-area, which is a prerequisite for the distance between the vehicle and the moving object to be within a second preset safe distance range. In order to improve the accuracy, it is further required to ensure that another precondition is satisfied, that is, the height of the moving object in the last frame of image is greater than a second preset multiple of the height of the last frame of image. The second preset multiple is a critical value which enables the distance between the vehicle and the moving target to be within a second preset safety distance range, and when the height of the moving target in the last frame of image is smaller than or equal to the second preset multiple of the height of the last frame of image, the distance between the vehicle and the moving target is larger than the upper limit value of the second preset safety distance range, namely the vehicle is too far away from the moving target. The second preset multiple may be set according to actual needs, for example, 0.5.
In one embodiment, based on the above embodiment related to the moving object, step S104 includes:
when the vehicle is currently running at a constant speed and the moving object moves at a constant speed or in an accelerated manner, controlling the vehicle to continue running at the next moment;
when the vehicle is running at a constant speed and the moving target moves in a decelerating or stopping manner at present, controlling the vehicle to operate in a decelerating or stopping manner at the next moment;
when the vehicle stops running at present and the moving target moves at a constant speed or in an accelerated manner, controlling the vehicle to start running at the next moment;
and when the vehicle stops running currently and the moving target moves in a decelerating way or stops moving, controlling the vehicle to continue to stop running at the next moment.
In the application, the operation stop may be operation stop and enter a standby or shutdown state, the operation start may be operation start in the standby or shutdown state, the operation start may be facilitated by entering the standby state when the operation stop is performed, and the power consumption may be reduced by entering the shutdown state when the operation stop is performed.
According to the vehicle operation control method provided by the embodiment of the invention, the safe operation of the vehicle can be controlled only by acquiring the image in the vehicle operation direction and processing the image, so that the vehicle can be provided with only a camera and a processor, the hardware structure is simple, and the calculated amount is small.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
The embodiment of the invention also provides a vehicle operation control device which is used for executing the steps in the vehicle operation control method. The vehicle operation control device may be a virtual appliance (virtual application) in the electronic device, which is operated by a processor of the electronic device, or may be the electronic device itself.
As shown in fig. 6, a vehicle operation control device 100 according to an embodiment of the present invention includes:
the image acquisition module 101 is used for acquiring images within a preset visual field range in the running direction of the vehicle;
the image preprocessing module 102 is configured to preprocess the image to obtain size information and state information of the target in the image;
the distance detection module 103 is used for detecting whether the distance between the vehicle and the target is within a preset safe distance range according to the size information of the target in the image;
and the state control module 104 is configured to control the operation state of the vehicle at the next moment according to the current operation state of the vehicle and the state information of the target in the image when the distance between the vehicle and the target is within the preset safe distance range.
In one embodiment, the vehicle running control apparatus further includes:
the device comprises a setting module, a display module and a control module, wherein the setting module is used for setting the size of a preset visual field range according to the position of a target relative to a vehicle and the size of the target, so that the target is positioned in the preset visual field range when the distance between the vehicle and the target is in a preset safe distance range.
In one embodiment, the vehicle running control apparatus further includes:
and the return module is used for returning to the image acquisition module.
In application, each module in the vehicle operation control device may be a software program module, may be implemented by different logic circuits integrated in a processor, and may also be implemented by a plurality of distributed processors.
As shown in fig. 7, an embodiment of the present invention further provides an electronic device 200, including: at least one processor 201 (only one processor is shown in fig. 7), a memory 202, a computer program 203 stored in the memory 202 and executable on the at least one processor 201, and a camera 204, the steps in the various vehicle operation control method embodiments described above being implemented when the processor 201 executes the computer program 203.
In an application, the electronic device may include, but is not limited to, a memory, a processor, a camera. Those skilled in the art will appreciate that fig. 7 is merely an example of an electronic device, and does not constitute a limitation of the electronic device, and may include more or less components than those shown, or combine certain components, or different components, such as input output devices, network access devices, etc. The electronic device may not include a camera, but may be communicatively connected to a camera of an external device through a wired or wireless communication module.
In an Application, the Processor may include a Central Processing Unit (CPU) and a Graphics Processing Unit (GPU), and the Processor may be other general purpose processors, Digital Signal Processors (DSP), Application Specific Integrated Circuits (ASIC), Field Programmable Gate Arrays (FPGA) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or any conventional processor or the like.
In application, the storage may in some embodiments be an internal storage unit of the electronic device, such as a hard disk or a memory of the electronic device. The memory may also be an external storage device of the electronic device in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, provided on the electronic device. Further, the memory may also include both internal storage units and external storage devices of the electronic device. The memory is used for storing an operating system, an application program, a Boot Loader (Boot Loader), data, and other programs, such as program codes of computer programs. The memory may also be used to temporarily store data that has been output or is to be output.
It should be noted that, because the contents of information interaction, execution process, and the like between the above-mentioned devices/modules are based on the same concept as the method embodiment of the present invention, specific functions and technical effects thereof can be referred to specifically in the method embodiment section, and are not described herein again.
It will be clear to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional modules is merely illustrated, and in practical applications, the above function distribution may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to perform all or part of the above described functions. Each functional module in the embodiments may be integrated into one processing module, or each module may exist alone physically, or two or more modules are integrated into one module, and the integrated module may be implemented in a form of hardware, or in a form of software functional module. In addition, the specific names of the functional modules are only for convenience of distinguishing from each other and are not used for limiting the protection scope of the present invention. The specific working process of the modules in the system may refer to the corresponding process in the foregoing method embodiment, and is not described herein again.
The embodiment of the invention provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method for controlling the running of the vehicle is realized according to any one of the embodiments.
Embodiments of the present invention provide a computer program product, which, when running on an electronic device, causes the electronic device to execute the vehicle operation control method of any of the above embodiments.
The integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may be implemented by a computer program, which is stored in a computer readable storage medium and used for instructing related hardware to implement the steps of the embodiments of the method. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include at least: any entity or apparatus capable of carrying computer program code to an electronic device, a recording medium, computer Memory, Read-Only Memory (ROM), Random-Access Memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium. Such as a usb-disk, a removable hard disk, a magnetic or optical disk, etc.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative modules and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus, electronic device and method can be implemented in other ways. For example, the above-described embodiments of the apparatus and electronic device are merely illustrative, and for example, a division of modules is merely a division of logical functions, and an actual implementation may have another division, for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.

Claims (12)

1. A vehicle operation control method characterized by comprising:
acquiring an image in a preset visual field range in the running direction of a vehicle;
preprocessing the image to obtain size information and state information of a target in the image;
detecting whether the distance between the vehicle and the target is within a preset safe distance range or not according to the size information of the target in the image;
and when the distance between the vehicle and the target is within a preset safe distance range, controlling the running state of the vehicle at the next moment according to the current running state of the vehicle and the state information of the target in the image.
2. The vehicle operation control method according to claim 1, wherein the preprocessing the image to obtain size information and state information of the object in the image comprises:
zooming the image to obtain an image with a preset size;
converting the image with the preset size into a preset color space to obtain an image in the preset color space;
detecting a target in the image in the preset color space through the trained deep learning model, and obtaining the category and size information of the target in the image in the preset color space;
and acquiring the state information of the target according to the category of the target in the image in the preset color space.
3. The vehicle operation control method according to claim 2, wherein the target is a traffic signal lamp;
the detecting, by the trained deep learning model, the target in the image in the preset color space to obtain the category and size information of the target in the image in the preset color space includes:
detecting a traffic signal lamp in the image under the preset color space through the trained first deep learning model, and obtaining the color category and size information of the traffic signal lamp in the image under the preset color space;
the obtaining of the state information of the target according to the category of the target in the image in the preset color space includes:
and acquiring the lighting color of the traffic signal lamp according to the color category of the traffic signal lamp in the image in the preset color space.
4. The vehicle running control method according to claim 3, wherein the detecting whether the distance between the vehicle and the object is within a preset safe distance range according to the size information of the object in the image comprises:
detecting whether the traffic signal lamp in the image in the preset color space is in a first target area or not according to the size information of the traffic signal lamp in the image in the preset color space;
when the traffic signal lamp in the image in the preset color space is in a first target area, detecting whether the height of the traffic signal lamp in the image in the preset color space is greater than a first preset multiple of the height of the image in the preset color space, wherein the first preset multiple is less than 1;
and when the height of a traffic signal lamp in the image in the preset color space is greater than a first preset multiple of the height of the image in the preset color space, determining that the distance between the vehicle and the traffic signal lamp is within a first preset safety distance range.
5. The vehicle operation control method according to claim 3 or 4, wherein the traffic signal lamp is a traffic light;
the controlling the running state of the vehicle at the next moment according to the current running state of the vehicle and the state information of the target in the image comprises the following steps:
when the vehicle is currently running and the number of times that the lightening color of the traffic light is red is greater than the preset number of times, controlling the vehicle to stop running at the next moment;
when the vehicle is currently running and the number of times that the lightening color of the traffic light is yellow is greater than the preset number of times, controlling the vehicle to continue running at the next moment;
when the vehicle stops running currently and the number of times that the traffic light is green is greater than the preset number of times, controlling the vehicle to start running at the next moment;
and when the vehicle stops running currently and the number of times that the lightening color of the traffic light is yellow is larger than the preset number of times, controlling the vehicle to continue to stop running at the next moment.
6. The vehicle operation control method according to claim 5, wherein the controlling of the operation state of the vehicle at the next time based on the current operation state of the vehicle and the state information of the object in the image, further comprises:
counting the number of times that the lighting color of the traffic light is red and yellow, respectively, when the vehicle is currently running;
when the vehicle stops operating at present, the number of times that the lighting color of the traffic light is green and yellow is counted, respectively.
7. The vehicle running control method according to claim 2, wherein the target is a moving target;
the acquiring of the image in the preset view field range in the running direction of the vehicle comprises the following steps:
continuously acquiring a plurality of frames of images within a preset visual field range in the running direction of the vehicle;
the detecting, by the trained deep learning model, the target in the image in the preset color space to obtain the category and size information of the target in the image in the preset color space includes:
respectively detecting moving targets in each frame of image under the preset color space through a trained second deep learning model, and obtaining size information of the moving targets in each frame of image under the preset color space;
obtaining the size change rule of the moving target in the multi-frame image in the preset color space according to the size information of the moving target in each frame image in the preset color space;
obtaining the motion state category of the moving target according to the size change rule of the moving target in the multi-frame image in the preset color space;
the obtaining of the state information of the target according to the category of the target in the image in the preset color space includes:
and obtaining the motion state of the motion target according to the motion state category of the motion target.
8. The vehicle operation control method according to claim 7, wherein the detecting whether the distance between the vehicle and the object is within a preset safe distance range according to the size information of the object in the image includes:
detecting whether the moving target in the last frame image in the preset color space is in a second target area or not according to the size information of the moving target in the last frame image in the preset color space;
when a moving target in the last frame of image in the preset color space is in a second target area, detecting whether the height of the moving target in the last frame of image in the preset color space is greater than a second preset multiple of the height of any frame of image in the preset color space, wherein the second preset multiple is less than 1;
and when the height of the moving target in the last frame of image in the preset color space is greater than a second preset multiple of the height of any frame of image in the preset color space, determining that the distance between the vehicle and the moving target is within a second preset safety distance range.
9. The vehicle operation control method according to claim 7 or 8, wherein the controlling of the operation state of the vehicle at the next timing based on the current operation state of the vehicle and the state information of the object in the image includes:
when the vehicle is currently running at a constant speed and the moving object moves at a constant speed or in an accelerated manner, controlling the vehicle to continue running at the next moment;
when the vehicle is running at a constant speed and the moving target moves in a decelerating or stopping manner at present, controlling the vehicle to operate in a decelerating or stopping manner at the next moment;
when the vehicle stops running at present and the moving target moves at a constant speed or in an accelerated manner, controlling the vehicle to start running at the next moment;
and when the vehicle stops running currently and the moving target moves in a decelerating way or stops moving, controlling the vehicle to continue to stop running at the next moment.
10. A vehicle operation control device characterized by comprising:
the image acquisition module is used for acquiring images in a preset visual field range in the running direction of the vehicle;
the image preprocessing module is used for preprocessing the image to obtain the size information and the state information of the target in the image;
the distance detection module is used for detecting whether the distance between the vehicle and the target is within a preset safe distance range according to the size information of the target in the image;
and the state control module is used for controlling the running state of the vehicle at the next moment according to the current running state of the vehicle and the state information of the target in the image when the distance between the vehicle and the target is within a preset safe distance range.
11. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, further comprising or being in communication with a camera, the processor implementing the steps of the vehicle operation control method according to any one of claims 1 to 9 when executing the computer program.
12. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out the steps of a vehicle operation control method according to any one of claims 1 to 9.
CN202011197188.5A 2020-10-30 2020-10-30 Vehicle operation control method and device, electronic equipment and storage medium Active CN112348879B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202011197188.5A CN112348879B (en) 2020-10-30 2020-10-30 Vehicle operation control method and device, electronic equipment and storage medium
CN202311422306.1A CN117409070A (en) 2020-10-30 2020-10-30 Vehicle operation control method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011197188.5A CN112348879B (en) 2020-10-30 2020-10-30 Vehicle operation control method and device, electronic equipment and storage medium

Related Child Applications (1)

Application Number Title Priority Date Filing Date
CN202311422306.1A Division CN117409070A (en) 2020-10-30 2020-10-30 Vehicle operation control method and device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN112348879A true CN112348879A (en) 2021-02-09
CN112348879B CN112348879B (en) 2023-12-19

Family

ID=74357125

Family Applications (2)

Application Number Title Priority Date Filing Date
CN202311422306.1A Pending CN117409070A (en) 2020-10-30 2020-10-30 Vehicle operation control method and device, electronic equipment and storage medium
CN202011197188.5A Active CN112348879B (en) 2020-10-30 2020-10-30 Vehicle operation control method and device, electronic equipment and storage medium

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN202311422306.1A Pending CN117409070A (en) 2020-10-30 2020-10-30 Vehicle operation control method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (2) CN117409070A (en)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101734242A (en) * 2008-11-11 2010-06-16 上海迪哈大计算机科技有限公司 Control device and method for automatic braking in vehicle
CN103204123A (en) * 2013-03-25 2013-07-17 中国电子科技集团公司第三十八研究所 Vehicle-pedestrian detecting, tracking and early-warning device and method
US20170151027A1 (en) * 2015-11-30 2017-06-01 Hansen Medical, Inc. Robot-assisted driving systems and methods
CN107146440A (en) * 2017-06-26 2017-09-08 奇瑞汽车股份有限公司 Start-up and shut-down control method and apparatus for automatic driving vehicle
CN107527511A (en) * 2016-06-22 2017-12-29 杭州海康威视数字技术股份有限公司 A kind of intelligent vehicle driving based reminding method and device
KR20180011415A (en) * 2016-07-22 2018-02-01 주식회사 셈앤텍 Apparatus for assisting safty drive of train based on features of signal
CN110569602A (en) * 2019-09-10 2019-12-13 中国科学技术大学 Data acquisition method and system for unmanned vehicle
CN111428663A (en) * 2020-03-30 2020-07-17 北京百度网讯科技有限公司 Traffic light state identification method and device, electronic equipment and storage medium
CN111695546A (en) * 2020-06-28 2020-09-22 北京京东乾石科技有限公司 Traffic signal lamp identification method and device for unmanned vehicle

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101734242A (en) * 2008-11-11 2010-06-16 上海迪哈大计算机科技有限公司 Control device and method for automatic braking in vehicle
CN103204123A (en) * 2013-03-25 2013-07-17 中国电子科技集团公司第三十八研究所 Vehicle-pedestrian detecting, tracking and early-warning device and method
US20170151027A1 (en) * 2015-11-30 2017-06-01 Hansen Medical, Inc. Robot-assisted driving systems and methods
CN107527511A (en) * 2016-06-22 2017-12-29 杭州海康威视数字技术股份有限公司 A kind of intelligent vehicle driving based reminding method and device
KR20180011415A (en) * 2016-07-22 2018-02-01 주식회사 셈앤텍 Apparatus for assisting safty drive of train based on features of signal
CN107146440A (en) * 2017-06-26 2017-09-08 奇瑞汽车股份有限公司 Start-up and shut-down control method and apparatus for automatic driving vehicle
CN110569602A (en) * 2019-09-10 2019-12-13 中国科学技术大学 Data acquisition method and system for unmanned vehicle
CN111428663A (en) * 2020-03-30 2020-07-17 北京百度网讯科技有限公司 Traffic light state identification method and device, electronic equipment and storage medium
CN111695546A (en) * 2020-06-28 2020-09-22 北京京东乾石科技有限公司 Traffic signal lamp identification method and device for unmanned vehicle

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
李海霞;罗芳芳;: "汽车辅助驾驶系统交通信号灯识别", 电子技术与软件工程, no. 13 *

Also Published As

Publication number Publication date
CN117409070A (en) 2024-01-16
CN112348879B (en) 2023-12-19

Similar Documents

Publication Publication Date Title
CN110494863B (en) Determining drivable free space of an autonomous vehicle
JP7101255B2 (en) Methods, vehicle control methods and devices for predicting the direction of motion of a target object
CN111874006B (en) Route planning processing method and device
CN114008685A (en) Intersection region detection and classification for autonomous machine applications
CN114902295A (en) Three-dimensional intersection structure prediction for autonomous driving applications
CN111133447A (en) Object detection and detection confidence suitable for autonomous driving
CN111133448A (en) Controlling autonomous vehicles using safe arrival times
CN112347829A (en) Determining lane allocation of objects in an environment using obstacle and lane detection
GB2550262A (en) Pedestrian detection and motion prediction with rear-facing camera
CN108133484B (en) Automatic driving processing method and device based on scene segmentation and computing equipment
EP3945460A1 (en) Instance segmentation using sensor data having different dimensionalities
US11308357B2 (en) Training data generation apparatus
CN111339996B (en) Method, device, equipment and storage medium for detecting static obstacle
EP4211651A1 (en) Efficient three-dimensional object detection from point clouds
CN115136148A (en) Projecting images captured using a fisheye lens for feature detection in autonomous machine applications
WO2024001093A1 (en) Semantic segmentation method, environment perception method, apparatus, and unmanned vehicle
CN114693540A (en) Image processing method and device and intelligent automobile
CN114973050A (en) Deep neural network aware ground truth data generation in autonomous driving applications
US20230213945A1 (en) Obstacle to path assignment for autonomous systems and applications
US20230343083A1 (en) Training Method for Multi-Task Recognition Network Based on End-To-End, Prediction Method for Road Targets and Target Behaviors, Computer-Readable Storage Media, and Computer Device
CN112348879A (en) Vehicle operation control method and device, electronic equipment and storage medium
CN112970029B (en) Deep neural network processing for sensor blind detection in autonomous machine applications
CN113744304A (en) Target detection tracking method and device
CN117152718B (en) Traffic light response method, device, vehicle and computer readable storage medium
US20230252638A1 (en) Systems and methods for panoptic segmentation of images for autonomous driving

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant