WO2022204840A1 - Image sensor calibration method and apparatus, device, and storage medium - Google Patents

Image sensor calibration method and apparatus, device, and storage medium Download PDF

Info

Publication number
WO2022204840A1
WO2022204840A1 PCT/CN2021/083466 CN2021083466W WO2022204840A1 WO 2022204840 A1 WO2022204840 A1 WO 2022204840A1 CN 2021083466 W CN2021083466 W CN 2021083466W WO 2022204840 A1 WO2022204840 A1 WO 2022204840A1
Authority
WO
WIPO (PCT)
Prior art keywords
images
calibration
interfaces
image sensors
image sensor
Prior art date
Application number
PCT/CN2021/083466
Other languages
French (fr)
Chinese (zh)
Inventor
张兴新
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to CN202180001476.5A priority Critical patent/CN113196339A/en
Priority to PCT/CN2021/083466 priority patent/WO2022204840A1/en
Publication of WO2022204840A1 publication Critical patent/WO2022204840A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
    • G06V10/147—Details of sensors, e.g. sensor lenses
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle

Definitions

  • the present application relates to the field of communication technologies, and in particular, to a calibration method, apparatus, device, and storage medium for an image sensor.
  • multiple image sensors can be deployed on the vehicle, and multiple image sensors are connected to the computing device on the vehicle through cables.
  • multiple image sensors such as cameras, lidars, millimeter-wave radars, etc.
  • the computing device processes the multiple images by Image stitching, image recognition and other processing to achieve vehicle planning and control.
  • the computing device In order to realize the purpose of processing multiple images to plan and control the vehicle, the computing device needs to determine the corresponding relationship between the multiple acquired images and each image sensor. At present, it is necessary to manually insert the cables of multiple image sensors into corresponding interfaces of the computing device according to the preset corresponding plugging relationship, wherein the preset corresponding plugging relationship is multiple image sensors and multiple computing devices.
  • the corresponding plug-in relationship between the interfaces, and the corresponding plug-in relationship has been pre-stored in the computing device. Therefore, after the computing device receives an image, it can know which image the image is based on according to the pre-stored corresponding plug-in relationship. sent by the sensor.
  • the present application discloses a calibration method, device, equipment and storage medium for an image sensor, which avoids the defect of incorrect cable insertion caused by human factors and improves production efficiency.
  • the present application provides a method for calibrating an image sensor, comprising: acquiring multiple sets of calibration images collected by multiple image sensors through multiple interfaces, wherein the multiple interfaces, the multiple image sensors and the There is a one-to-one correspondence between the multiple sets of calibration images; identify multiple calibration objects included in the multiple sets of calibration images, and obtain the correspondence between the multiple calibration objects and the multiple interfaces; wherein, There is a one-to-one correspondence between the multiple calibration objects and the multiple sets of calibration images; the multiple images are determined according to the correspondence between the multiple calibration objects and the multiple interfaces and preset information The corresponding relationship between the sensor and the multiple interfaces; the preset information includes the corresponding relationship between the multiple calibration objects and the multiple image sensors.
  • the computing device first obtains multiple sets of calibration images collected by each image sensor through different interfaces, and then determines the corresponding relationship between each calibration object and each interface by identifying the calibration objects contained in each set of calibration images, and then according to The corresponding relationship and the preset information determine the corresponding relationship between each image sensor and each interface.
  • the cables and interfaces of each image sensor can be "blindly inserted", and then the machine can replace the labor to automatically complete the cable insertion work, which avoids cable insertion errors caused by manual negligence, saves manpower, and improves the efficiency.
  • the plurality of image sensors are respectively deployed in different directions of the vehicle.
  • the plurality of calibration objects satisfy one or more of conditions (i)-(iii): (i) at least two calibration objects in the plurality of calibration objects (ii) shapes of at least two calibration objects in the plurality of calibration objects are different from each other; (iii) patterns of at least two calibration objects in the plurality of calibration objects are different from each other.
  • the calibration objects can be identified by at least one of color, shape and pattern, or in other words, the calibration objects located in front of each image sensor can be different in color, shape or pattern.
  • the method further includes: acquiring the data collected by the plurality of image sensors on the vehicle According to the corresponding relationship between the multiple image sensors and the multiple interfaces, the environment image is processed to obtain a processing result; the vehicle is controlled according to the processing result.
  • the computing device can use the correspondence to process the environmental images collected by the respective image sensors, and accurately control the vehicle according to the processing results.
  • the present application provides a method for calibrating an image sensor, including: acquiring images collected by a plurality of image sensors through a plurality of interfaces at a plurality of times; wherein, acquiring images at each of the plurality of times Among the images, at least one image contains the features of the calibration object; there is a one-to-one correspondence between the multiple interfaces and the multiple image sensors; the calibration object features include the color, pattern, and shape of the calibration object.
  • One or more items identify the images containing the features of the calibration object in the images obtained at each time, so as to determine the relationship between the features of the calibration object or the images containing the features of the calibration object at different times and the multiple interfaces
  • the acquisition relationship between the plurality of image sensors and the plurality of interfaces is determined according to the acquisition relationship and the preset information; the preset information includes the time sequence in which the features of the calibration object appear.
  • the computing device can determine the corresponding relationship between each image sensor and each interface by identifying the time sequence in which the features of the calibration object appear. Implementing the present application can save manpower and improve the efficiency of cable plugging.
  • the calibration object can be on-board equipment, such as the lighting system on the vehicle, including forward lighting, rear brake lights, left and right turn signals, etc., and the different colors of each lamp in working state can be used as different calibration objects ;
  • the calibration object can also be at least one of the pattern, color, and shape generated by the vehicle-mounted device.
  • the pattern projected by the lighting system can be used as the calibration object.
  • the calibration object can be different in pattern, or color, or The shape may be different; the corresponding relationship between each image sensor and each interface may also be determined by using the different timing sequences of the patterns projected by the lighting system.
  • the calibration object may further include: an off-vehicle device; or at least one of a pattern, color, and shape generated by the off-vehicle device.
  • the calibration object may be a two-dimensional code of the device outside the vehicle, and the positional relationship between the calibration object and the vehicle is not limited in the present application.
  • the present application provides a method for calibrating an image sensor, comprising: acquiring a plurality of identification information through a plurality of interfaces respectively; wherein, there is a one-to-one correspondence between the plurality of interfaces and the plurality of identification information ; each identification information includes parameters of each image sensor; according to the plurality of identification information and preset information, determine the corresponding relationship between the plurality of image sensors and the plurality of interfaces; wherein, the preset The information includes the correspondence between the plurality of image sensors and the plurality of identification information.
  • the computing device can use the different identification information of each image sensor to determine the corresponding relationship between each image sensor and each interface, wherein the identification The information includes parameters of the image sensor.
  • the computing device can determine the corresponding relationship between each image sensor and each interface by using the parameters of the image sensor.
  • the parameters of each image sensor include at least one of a brand, a frequency of capturing images, and a definition of capturing images.
  • the present application provides a method for calibrating an image sensor, which is applied to a computing device, where the computing device includes a user interaction interface, and the method includes: acquiring multiple images collected by multiple image sensors through multiple interfaces, and displaying the multiple images in the user interaction interface; wherein, there is a one-to-one correspondence between the multiple interfaces, the multiple image sensors, and the multiple images; in response to the user's An operation is performed to determine and save the correspondence between the plurality of image sensors and the plurality of interfaces; wherein, the operation of the user is used to correspond the plurality of images to the plurality of image sensors.
  • the method for the computing device to determine the correspondence between the multiple image sensors and the multiple interfaces may be to obtain different image sensors through different interfaces.
  • the user interface is used to display multiple images, and the user operates on the user interface to determine the correspondence between the multiple images and the multiple image sensors, and the computing device responds to the user's operation to determine multiple images. Correspondence between image sensors and multiple interfaces.
  • the present application provides a method for calibrating an image sensor, which is applied to a computing device, where the computing device is connected to a display system, and the method includes: the computing device acquires data collected by multiple image sensors through multiple interfaces. multiple images, and send the multiple images to the display system; wherein, there is a one-to-one correspondence between the multiple interfaces, the multiple image sensors, and the multiple images; the display The system receives the user's operation, and obtains the corresponding relationship between the plurality of images and the plurality of image sensors; the display system sends the corresponding relationship between the plurality of images and the plurality of image sensors to the plurality of image sensors.
  • the computing device the computing device determines the correspondence between the plurality of image sensors and the plurality of interfaces.
  • the method for the computing device to determine the correspondence between the multiple image sensors and the multiple interfaces may be:
  • the multiple images collected by the sensor are sent to the display system, the display system receives and displays the multiple images, the user operates on the display system, the display system receives and responds to the user's operation, and obtains the relationship between multiple images and multiple image sensors.
  • the corresponding relationship is sent to the computing device, and the computing device determines the corresponding relationship between the multiple image sensors and the multiple interfaces according to the received corresponding relationship.
  • the present application provides an image sensor calibration device, comprising: an acquisition unit configured to acquire multiple sets of calibration images collected by multiple image sensors through multiple interfaces, wherein the multiple interfaces, the multiple There is a one-to-one correspondence between each image sensor and the multiple sets of calibration images; the identification unit is used to identify multiple calibration objects included in the multiple sets of calibration images, and obtain the multiple calibration objects and the multiple calibration objects.
  • the determining unit is configured to determine the relationship between the multiple calibration objects and the multiple interfaces according to the relationship between the multiple calibration objects and the multiple interfaces.
  • the corresponding relationship between the multiple image sensors and the multiple interfaces is determined, and the preset information includes the corresponding relationship between the multiple calibration objects and the multiple image sensors.
  • the multiple image sensors are respectively deployed in different directions of the vehicle.
  • the plurality of calibration objects satisfy one or more of the conditions (i)-(iii): (i) at least two calibration objects in the plurality of calibration objects (ii) shapes of at least two calibration objects in the plurality of calibration objects are different from each other; (iii) patterns of at least two calibration objects in the plurality of calibration objects are different from each other.
  • the acquiring unit is further configured to acquire environmental images collected by the multiple image sensors on the vehicle;
  • the apparatus further includes: a processing unit configured to The corresponding relationship between each image sensor and the plurality of interfaces is to process the environment image to obtain a processing result;
  • a control unit is used to control the vehicle according to the processing result.
  • Each functional unit in the apparatus of the sixth aspect is used to implement the method described in the first aspect and any implementation manner of the first aspect.
  • the present application provides an image sensor calibration device, comprising: an acquisition unit configured to acquire images collected by a plurality of image sensors through a plurality of interfaces at a plurality of times; wherein, among the plurality of times Among the images obtained at each moment of , at least one image contains the features of the calibration object; there is a one-to-one correspondence between the multiple interfaces and the multiple image sensors; the calibration object features include the color of the calibration object, One or more of patterns and shapes; an identification unit, configured to identify the images containing the features of the calibration objects in the images obtained at each moment, so as to determine the features of the calibration objects at different moments or the inclusion of the calibration objects the acquisition relationship between the image of the feature and the multiple interfaces; the determining unit is configured to determine the corresponding relationship between the multiple image sensors and the multiple interfaces according to the acquired relationship and preset information; the preset Let the information include the chronological order in which the calibrator features appear.
  • the calibration object includes: an in-vehicle device; or, at least one of a pattern, color, and shape generated by the in-vehicle device.
  • the calibration object may further include: an off-vehicle device; or at least one of a pattern, color, and shape generated by the off-vehicle device.
  • the calibration object may be a two-dimensional code of the device outside the vehicle, and the positional relationship between the calibration object and the vehicle is not limited in this application.
  • Each functional unit in the apparatus of the seventh aspect is used to implement the method described in the second aspect and any implementation manner of the second aspect.
  • the present application provides a calibration device for an image sensor, comprising: an acquisition unit configured to acquire a plurality of identification information through a plurality of interfaces respectively; wherein, between the plurality of interfaces and the plurality of identification information There is a one-to-one correspondence; each identification information includes parameters of each image sensor; a determining unit is configured to determine the relationship between the multiple image sensors and the multiple interfaces according to the multiple identification information and preset information The corresponding relationship between the plurality of image sensors and the plurality of identification information is included in the preset information.
  • the parameters of each image sensor include at least one of a brand, a frequency of capturing images, and a definition of capturing images.
  • the present application provides an image sensor calibration device, the device includes a user interaction interface, the device includes: an acquisition unit, configured to acquire multiple images collected by multiple image sensors through multiple interfaces, and displaying the multiple images in the user interaction interface; wherein, there is a one-to-one correspondence between the multiple interfaces, the multiple image sensors, and the multiple images; the response unit is used for In response to a user's operation, the correspondence between the plurality of image sensors and the plurality of interfaces is determined and saved; wherein the user's operation is used to perform a comparison between the plurality of images and the plurality of image sensors correspond.
  • the present application provides an image sensor calibration device, the device is connected to a display system, and the device includes: an acquisition unit, used for the computing device to acquire multiple data collected by multiple image sensors through multiple interfaces images, and send the multiple images to the display system; wherein, there is a one-to-one correspondence between the multiple interfaces, the multiple image sensors, and the multiple images; the communication unit, using receiving the correspondence between the plurality of images sent by the display system and the plurality of image sensors; a determining unit for determining the correspondence between the plurality of image sensors and the plurality of interfaces .
  • Each functional unit in the apparatus of the tenth aspect is configured to implement the method described in the fifth aspect and any implementation manner of the fifth aspect.
  • the present application provides a computing device, the computing device includes a memory and a processor, the memory is used for storing instructions, and the processor is used for calling the instructions stored in the memory to execute the above-mentioned first aspect and
  • the method described in any implementation manner of the first aspect, or the method described in any implementation manner of the second aspect and the second aspect, or the method described in any implementation manner of the third aspect and the third aspect A method, or the method described in any implementation manner of the fourth aspect and the fourth aspect, or the method described in any implementation manner of the fifth aspect and the fifth aspect.
  • the present application provides a computer storage medium, comprising program instructions, when the computer runs the program instructions, the computer causes the computer to execute the method described in the first aspect and any implementation manner of the first aspect, Or the method described in any implementation manner of the second aspect and the second aspect, or the method described in any implementation manner of the third aspect and the third aspect, or any implementation of the fourth aspect and the fourth aspect The method described in the manner, or the method described in any implementation manner of the fifth aspect and the fifth aspect.
  • the present application provides a computer program product, including program instructions, which, when the program product is executed by a computing device, cause the computing device to execute the first aspect and any implementation manner of the first aspect.
  • the computer program product can be a software installation package, which can be downloaded and executed on a computing device if the method provided by any of the possible designs of any of the preceding aspects needs to be used to A method as described in any possible implementation of any of the above aspects is implemented.
  • the present application provides a method for calibrating an image sensor.
  • the timing sequence of each image sensor, or the corresponding relationship between each image sensor and each interface is determined according to the different identification information of different image sensors; or, the user interaction interface or display system is used to determine the corresponding relationship between each image sensor and each interface. .
  • the accuracy and production efficiency of plugging are improved.
  • FIG. 1 is a schematic flowchart of a method for calibrating an image sensor according to an embodiment of the present application
  • FIG. 2 is a scene example diagram of a calibration method for an image sensor provided by an embodiment of the present application
  • Fig. 3a is an example diagram of a calibration object provided by the embodiment of the present application.
  • Fig. 3b is an example diagram of a calibration object provided by the embodiment of the application.
  • Fig. 3c is an example diagram of a calibration object provided by the embodiment of the present application.
  • FIG. 3d is an example diagram of a calibration object provided by the embodiment of the present application.
  • FIG. 4 is a schematic flowchart of another method for calibrating an image sensor provided by an embodiment of the present application.
  • FIG. 5 is a schematic flowchart of another method for calibrating an image sensor provided by an embodiment of the present application.
  • FIG. 6 is a schematic flowchart of another image sensor calibration method provided by an embodiment of the present application.
  • FIG. 7 is a schematic flowchart of another image sensor calibration method provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of a calibration device for an image sensor provided by an embodiment of the present application.
  • FIG. 9 is a schematic diagram of another image sensor calibration device provided by an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of a computing device according to an embodiment of the present application.
  • Image sensors are used to monitor the environment around the vehicle and capture images.
  • ADAS advanced driving assistance system
  • the total field of view of multiple image sensors covers the entire horizontal space where the vehicle is located.
  • the image sensor can be a camera, lidar, millimeter wave radar, etc.
  • the computing device is used to process the environmental images collected by multiple image sensors to realize route planning, control decision-making, and the like.
  • the computing device may be located in a multiple domain controller/mobile data center (MDC) on the vehicle, or may be located in a cockpit domain controller (CDC) on the vehicle.
  • MDC multiple domain controller/mobile data center
  • CDC cockpit domain controller
  • the device may also be located in the electronic control unit (ECU) of the vehicle.
  • the computing device of the vehicle is configured with multiple cable interfaces, and the multiple image sensors are respectively connected to the computing device through the multiple cable interfaces.
  • the corresponding connection relationship between each cable interface and each image sensor is preset in the computing device, wherein each cable interface corresponds to an image sensor in one orientation, and different cable interfaces correspond to image sensors in different orientations. Therefore, before the vehicle leaves the factory, the worker needs to insert the cable of each image sensor into the corresponding cable interface of the computing device.
  • the cables of each image sensor are generally designed to be of different colors. When plugging, workers plug the cables of different colors into the corresponding cable interfaces.
  • the cable ports of each image sensor can be designed in different shapes, and the worker plugs the cables of each image sensor to the corresponding cable interface according to the shape of the cable port, so as to realize the connection between each image sensor and the computing device. connection between the various cable ports on the
  • the image sensors in all directions collect the images in the corresponding directions, and transmit the collected images to the computing device through the corresponding cable interface. Route planning, vehicle control, etc., to achieve assisted driving or automatic driving functions.
  • cables and cable interfaces need special design, which increases costs; during vehicle assembly, workers are required to insert the cables of each image sensor into the cable interface of the computing device according to the color or the shape of the cable port. It is labor-intensive, and due to the negligence of the workers, there may be a problem of wrong plugging. Once the plugging error occurs, the vehicle needs to be disassembled and reassembled, which not only consumes manpower, material resources, etc., but also increases costs.
  • the embodiment of the present application provides a method for supporting blind plugging of the cable, that is, a calibration method for the image sensor. Before introducing the embodiment of the method, first The connection mode between each image sensor and the computing device in the embodiment of this method is described.
  • the cables of each image sensor on the vehicle can be the same, and there is no need to design ports of different colors or different shapes. It can be "blind-plugged" by machine or manually, that is, the cables of each image sensor and the computing device are many.
  • the interfaces can be arbitrarily plugged and connected, and after the plugging is completed, the corresponding connection relationship between each image sensor and each interface of the computing device is determined by the computing device.
  • FIG. 1 is a schematic flowchart of a method for calibrating an orientation of an image sensor according to an embodiment of the present application, and the method includes but is not limited to the description of the following contents.
  • a plurality of image sensors are respectively deployed in different directions of the vehicle, and each image sensor is used to collect images in respective directions.
  • the specific number of image sensors on the vehicle is not specifically limited in this application.
  • the total field of view of all image sensors on the vehicle covers the entire horizontal space where the vehicle is located, or in other words, multiple images collected by multiple image sensors on the vehicle. contains the environmental information of the space around the vehicle.
  • the image sensor can be a camera, lidar, millimeter wave radar, etc.
  • the number of images collected by each image sensor may be one or multiple, and the application does not specifically limit the number of images collected by each image sensor.
  • different calibration objects are placed in front of each image sensor (the front of each image sensor, which refers to the position within the field of view of each image sensor), and the different calibration objects may be calibrations of different colors lamp.
  • the 4 image sensors are deployed on the vehicle, and the 4 image sensors are located in different directions of the vehicle (probably located on the front side, rear side, and left side of the vehicle). , one on the right side), for convenience of description, they are referred to here as a front image sensor, a rear image sensor, a left image sensor, and a right image sensor, respectively.
  • Place calibration lights of different colors in front of the front image sensor, rear image sensor, left image sensor, and right image sensor are placed in front of the front image sensor, rear image sensor, left image sensor, and right image sensor. For example, place a white calibration light in front of the front image sensor, and place a red calibration light in front of the rear image sensor. Place a yellow calibration light in front of the left image sensor and a green calibration light in front of the right image sensor.
  • Each image sensor collects images, and the colors of the images collected by each image sensor are different.
  • the color of the image collected by the front image sensor is mainly white
  • the color of the image collected by the rear image sensor is mainly red
  • the color of the image collected by the left image sensor is mainly white.
  • the color is mainly yellow
  • the color of the image captured by the right image sensor is mainly green.
  • Each image sensor transmits the collected images to the computing device through multiple interfaces (cable interfaces), and correspondingly, the computing device obtains multiple images collected by multiple image sensors through multiple interfaces.
  • the cable of each image sensor is connected with the computing device through an interface. It should be noted that due to the "blind insertion" between each image sensor and the cable interface, the computing device cannot confirm how each cable interface is correspondingly connected to each image sensor at this time. The device cannot confirm which image sensor transmits the image received by each cable interface.
  • each image sensor may be deployed inside the vehicle body, or may be deployed outside the vehicle body, which is not limited in this application.
  • different calibration objects are respectively placed in front of each image sensor (the front of each image sensor, which refers to the position within the field of view of each image sensor), and the calibration objects may be of different patterns or have different patterned items.
  • replace the calibration lights of different colors in the example in Figure 2 with different patterns for example, replace the white calibration lights in front of the front image sensor in Figure 2 with the pattern shown in Figure 3a, and replace the red in front of the rear image sensor in Figure 2.
  • Replace the calibration light with the pattern shown in Figure 3b replace the yellow calibration light in front of the left image sensor in Figure 2 with the pattern shown in Figure 3c, and replace the green calibration light in front of the right image sensor in Figure 2 with the pattern shown in Figure 3d picture of.
  • Each image sensor performs image acquisition, and transmits the acquired image to the computing device through the connected cable interface, and correspondingly, the computing device receives the image transmitted by each image sensor.
  • the patterns of the images transmitted by the respective image sensors are different.
  • the calibration objects in front of each image sensor may be objects with different patterns or patterns, lamps of different colors or objects with different colors, or objects with different shapes. and many more.
  • the scene around the vehicle can also be used as the calibration object.
  • the premise that the scene around the vehicle can be used as the calibration object is that the scenes within the field of view of each image sensor are different.
  • the images acquired by each image sensor are not exactly the same.
  • the application does not specifically limit the calibration object, as long as the images acquired by each image sensor are not identical.
  • the calibration objects are generally deployed within the field of view of each image sensor, and within a range that does not overlap with the field of view of any other image sensor.
  • the calibration objects are arranged at the positions facing the respective image sensors. The distance between the calibration object and the image sensor is not limited.
  • S102 Identify multiple calibration objects included in the multiple calibration images, and obtain the correspondence between the multiple calibration objects and the multiple interfaces.
  • the multiple calibration objects may be different in color, shape or pattern.
  • calibration lights of different colors are arranged in front of each image sensor, and the colors of the calibration objects in the images obtained by the computing device are different, and the colors include white, red, yellow, and green.
  • the color of the calibration light here refers to the color of the calibration light when it is in a working state.
  • FIGS. 3 a to 3 d different patterns are set in front of each image sensor, and the patterns of the multiple calibration objects in the multiple images obtained by the computing device are different, wherein the patterns include FIG. 3 a , FIG. 3 b , and FIG. 3 c and the pattern shown in Figure 3d.
  • the computing device recognizes multiple calibration objects contained in multiple images, and can recognize the calibration objects included in the image by identifying the color in the image or the pattern in the image or the shape in the image, so as to know that each image is derived from the image.
  • Which interface is transmitted in that is, the corresponding relationship between each calibration object (each image) and each interface is obtained.
  • Table 1 shows the correspondence between the calibration objects with different colors and the interface in the example of Figure 2. It can be seen from Table 1 that the colors of the calibration objects included in the images obtained from interfaces 1, 2, 3, and 4 are respectively It's white, red, yellow, green.
  • Table 2 shows the correspondence between different calibration objects and interfaces. It can be seen from Table 2 that the calibration objects included in the images obtained from interfaces 1, 2, 3, and 4 are the calibration objects shown in Figure 3a, respectively. , the calibration object shown in Figure 3b, the calibration object shown in Figure 3c, and the calibration object shown in Figure 3d.
  • the recognition image can be recognized by extracting features, or it can be recognized by inputting the image into a pre-trained recognition model, where the recognition model can be a recurrent neural network model, a convolutional neural network model, etc.
  • the application does not limit the identification method.
  • the shape of the calibration object refers to the existence or manifestation of the calibration object, and the shape is usually formed by the outline or edge of an object or real object, such as rectangle, circle, ellipse, ring, etc.
  • the pattern of the calibration object refers to the design pattern carried on the entity or object with a certain shape, and the pattern in this application does not include color information.
  • the color of the calibration object is also carried on the entity or object with a certain shape, and the color of the calibration object may refer to the color of the shape of the calibration object or the color of the pattern of the calibration object.
  • S103 Determine the correspondence between the multiple image sensors and the multiple interfaces according to the correspondence between the multiple calibration objects and the multiple interfaces and the preset information.
  • the preset information includes the correspondence between the plurality of calibration objects and the plurality of image sensors.
  • Table 3 shows the preset information in the computing device in the example of FIG. 2 , where the preset information provides the corresponding relationship between each azimuth image sensor and calibration objects of different colors.
  • Table 4 shows the preset information in the computing device, and the preset information provides the corresponding relationship between the image sensors in various orientations and different calibration objects.
  • the corresponding connection relationship between the multiple image sensors on the vehicle and the multiple interfaces can be determined, wherein each image sensor corresponds to an interface, so as to determine the acquisition of each image sensor. which orientation is the image in.
  • each azimuth image sensor and each interface shown in Table 1 and the preset information in Table 3 it can be determined that the cable of the front image sensor is connected to the 1 interface of the computing device, and the rear The cable of the image sensor is connected to the 2 port of the computing device, the cable of the left image sensor is connected to the 3 port of the computing device, and the cable of the right image sensor is connected to the 4 port of the computing device.
  • Table 5 As shown in Table 5.
  • each azimuth image sensor and each interface shown in Table 2 and the preset information in Table 4 it can also be determined that the cable of the front image sensor is connected to the 1 interface of the computing device , the cable of the rear image sensor is connected to the 2 port of the computing device, the cable of the left image sensor is connected to the 3 port of the computing device, and the cable of the right image sensor is connected to the 4 port of the computing device of.
  • the correspondence is saved to the computing device, so that the correspondence can be used directly in future applications without the need to perform matching again.
  • the method of this embodiment is used to calibrate multiple image sensors, determine the correspondence between multiple image sensors and multiple interfaces, and save the correspondence.
  • the vehicle is used, maintained, and debugged, parameters are set for image sensors in different directions on the vehicle, for example, setting the exposure of the image sensor, setting the frequency of image acquisition by the image sensor, etc.
  • the computing device obtains each image sensor.
  • the collected environmental images are then stitched and processed according to the multiple collected images, so as to realize the control of the vehicle, for example, to realize the assisted driving or automatic driving function of the vehicle.
  • the vehicle in this application is not limited to an automobile, but can also be any other type of vehicle, even a bicycle, a tricycle, etc., as long as there are devices (computing devices) with computing capabilities, multiple image sensors and multiple With only one interface, the calibration of multiple image sensors can be realized.
  • the computing device determines the correspondence between each image sensor and each interface according to the calibration objects contained in the calibration images collected by each image sensor, and to implement the embodiments of the present application, each image sensor cable is connected to each interface.
  • “Blind insertion” avoids the disadvantages of wrong cable insertion due to worker's negligence, saves manpower, and improves insertion efficiency.
  • FIG. 4 is a schematic flowchart of a method for calibrating an image sensor. The method includes but is not limited to the following descriptions.
  • step S101 in the above-mentioned embodiment for the deployment orientation of each image sensor, the manner and quantity of image acquisition by each image sensor, and for the sake of brevity of the description, details are not repeated here.
  • a calibration object is required, and the calibration object can be a calibration light of a certain color or a calibration object with a certain pattern or shape, etc.
  • the calibration objects can be placed in front of each image sensor in a preset order or at a preset time.
  • the preset time can be that the calibration object is placed in front of the front image sensor at time t1, and the calibration object is placed in front of the right image sensor at time t2.
  • the calibration object is placed in front of the rear image sensor at time t3
  • the calibration object is placed in front of the left image sensor at time t4, wherein the time sequence is t1, t2, t3, t4.
  • the preset sequence can be, first place the calibration object in front of the front image sensor, then place the calibration object in front of the right image sensor, then place the calibration object in front of the rear image sensor, and finally place the calibration object in the left image front of the sensor.
  • Each image sensor collects images at different times, and transmits the collected images to the computing device through different interfaces.
  • the computing device receives the images collected by each image sensor at different times.
  • the images acquired by the computing device at each moment at least one image contains a calibration object, and other images do not contain it.
  • the calibration object is placed in front of the front image sensor (the calibration object is not placed in front of other image sensors), and the front image sensor, rear image sensor, left image sensor, and right image sensor all collect images, and pass through different interfaces. It is transmitted to the computing device, and the computing device receives the images collected by each image sensor through different interfaces at time t1; then, move the calibration object to the front of the right image sensor, the front image sensor, the rear image sensor, the left image sensor, and the right image sensor.
  • All images are collected and transmitted to the computing device through different interfaces, and the computing device receives the images collected by each image sensor through different interfaces at t2; secondly, move the calibration object to the front of the rear image sensor, the front image sensor, the rear The image sensor, the left image sensor, and the right image sensor all collect images and transmit them to the computing device through different interfaces.
  • the computing device receives the images collected by each image sensor through different interfaces; finally, move the calibration object to the left image In front of the sensor, the front image sensor, the rear image sensor, the left image sensor, and the right image sensor all collect images and transmit them to the computing device through different interfaces respectively.
  • the computing device receives the images collected by each image sensor through different interfaces.
  • calibration lights can also be set by the front image sensor, the rear image sensor, the left image sensor, and the right image sensor. , and make it light up at different times to achieve the effect of the calibration object appearing in front of different image sensors at different times.
  • S202 Identify the images containing the features of the calibration object in the images acquired at each time, so as to determine the acquisition relationship between the images containing the features of the calibration object at different times and multiple interfaces.
  • the feature of the calibration object may be the color of the calibration object, the pattern of the calibration object, or the shape of the calibration object, and the like.
  • the images acquired at each moment in each moment are identified. Identify color information, pattern information, or shape information, etc., to determine the image containing the characteristics of the target.
  • the computing device After recognizing the images acquired at each moment, acquires the source interface of the images containing the features of the calibration object at different moments. Alternatively, the computing device obtains the source interface of the calibrator features at different times. It can be understood that the purpose of this embodiment can be achieved by identifying an image containing the features of the calibration object in the images acquired at each moment, so as to determine the acquisition relationship between the features of the calibration object at different times and multiple interfaces.
  • the computing device recognizes the image obtained at time t1, identifies the image containing the yellow calibration light, and determines from which interface the image containing the yellow calibration light is obtained at time t1 For example, it is obtained from interface 1; then, the computing device recognizes the image obtained at time t2, identifies the image containing the yellow calibration light, and determines which interface the image containing the yellow calibration light comes from at time t2 Obtained, for example, obtained from interface 2; secondly, identify the image obtained at time t3, identify the image containing the yellow calibration light, and determine which interface the image containing the yellow calibration light is from at time t3 Obtained, for example, obtained from interface 3; finally, the image obtained at time t4 is identified, and the image containing the yellow calibration light is identified, which also determines which interface the image containing the yellow calibration light is from at time t4. Obtained, for example, obtained from the 4 interface. Therefore, the acquisition relationship between images containing calibration objects at different times
  • the preset information includes the chronological sequence in which the features of the calibration objects appear, that is, the temporal sequence in which the calibration objects are set in front of each image sensor, or the sequence in which the calibration objects appear in front of each image sensor.
  • the calibration object is first set in front of which image sensor, then in front of which image sensor, second in front of which image sensor, and finally in front of which image sensor.
  • Table 7 exemplarily shows a kind of preset information.
  • the preset information shows the time sequence (sequence) of setting the same calibration object in front of each image sensor.
  • the calibration object is located in front of the front image sensor, then it moves to the front of the right image sensor, secondly, it moves to the front of the rear image sensor, and finally, it moves to the front of the left image sensor, or in other words, the calibration
  • the moving order of the object is front image sensor, right image sensor, rear image sensor, left image sensor.
  • the corresponding relationship between the multiple image sensors and the multiple interfaces is determined according to the acquisition relationship and preset information between the images containing the features of the calibration object and the multiple interfaces at different times. For example, according to Table 6 and Table 7, it can be concluded that the front image sensor is the 1 interface corresponding to the computing device, the right image sensor is the 2 interface corresponding to the computing device, the rear image sensor is the 3 interface corresponding to the computing device, and the left image sensor is the Corresponding to the 4 interfaces of the computing device, that is, Table 8.
  • a calibration object is set, and a calibration object is moved according to the preset order or preset time in the preset information as an example for description.
  • multiple calibration objects may also be set, and the multiple calibration objects may be of different colors, different patterns or different shapes.
  • the calibration object 1 is placed in front of the front image sensor, and no calibration objects are placed in front of other image sensors.
  • Each image sensor collects images and transmits the collected images to the computing device through different interfaces.
  • the computing device receives the images collected by each image sensor at time t11; at time t22, the calibration object 2 is placed in front of the right image sensor, and no calibration objects are placed in front of other image sensors, each image sensor collects images, and the collected The image is transmitted to the computing device through different interfaces.
  • the computing device receives the image collected by each image sensor at time t22; at time t33, the calibration object 3 is placed in front of the rear image sensor, and no calibration object is placed in front of other image sensors.
  • each image sensor collects images, and transmits the collected images to the computing device through different interfaces.
  • the computing device receives the images collected by each image sensor at time t33; at time t44, the calibration object 4 is placed on the left image sensor. In the front, no calibration objects are placed in front of other image sensors.
  • Each image sensor collects images and transmits the collected images to the computing device through different interfaces.
  • the computing device receives the images collected by each image sensor at time t44.
  • the computing device recognizes the images received at each moment in each moment, and according to the color, pattern or shape of the calibration object, etc., identifies which moment the image contains the image of the calibration object 1, and which moment the image contains the calibration object.
  • t1/t2/t3/t4, t11/t22/t33/t44 is only a convenient way to describe, the essence of the embodiment of the present invention is to determine the connection between the sensor and the interface through the sequence of the appearance of the calibration objects relationship, the specific values of t1/t2/t3/t4 and t11/t22/t33/t44 are not the focus of this embodiment.
  • a vehicle lighting system can also be used, wherein the vehicle lighting system includes lights and signal lights on the vehicle, and the corresponding relationship between each interface and each image sensor is determined (the calibration object includes the lights and signal lights on the vehicle, the calibration object is characterized by the color emitted when the light or signal light is in operation).
  • the vehicle lighting system includes lights and signal lights on the vehicle, and the corresponding relationship between each interface and each image sensor is determined (the calibration object includes the lights and signal lights on the vehicle, the calibration object is characterized by the color emitted when the light or signal light is in operation).
  • 1) time-sharing control of the lights on the vehicle. First, turn on the front lights of the vehicle, such as low beams or high beams, where the light emitted by the front lights is approximately white light, and the front image sensor, rear image sensor, left image sensor, and right image sensor capture images respectively.
  • each lighting lamp or each signal lamp in the vehicle lighting system can also be used, or, in a possible implementation, the different projection patterns of each lighting lamp or each signal lamp on the ground can be used to determine different times.
  • the corresponding relationship between the collected images and the multiple interfaces is further determined, thereby further determining the corresponding relationship between the multiple image sensors and the multiple interfaces.
  • the method for the computing device to determine the correspondence between each image sensor and each interface can also be based on the sequential order of acquiring images collected by each image sensor, that is, the computing device.
  • the corresponding relationship between each image sensor and each interface is further determined by identifying the temporal sequence of the appearance of the features of the calibration object contained in the images obtained at different times.
  • FIG. 5 is a schematic flowchart of a method for calibrating an image sensor provided by an embodiment of the present application. , the method includes but is not limited to the description of the following contents.
  • Each identification information includes parameters of each image sensor, such as the brand, resolution, etc. of the image sensor.
  • the computing device obtains a plurality of pieces of identification information respectively through a plurality of interfaces. For example, the computing device obtains the identification information of one of the image sensors through interface A, and the identification information includes brand a and front-view ultra-high-definition camera; through interface B, obtains the identification information of the other image sensor, and the identification information includes brand b, forward-looking standard definition camera Obtain the identification information of another image sensor through interface C, the identification information includes brand c, front-view standard definition camera; Obtain the identification information of another image sensor through interface D, and the identification information includes brand b, rear-view standard-definition camera, as shown in Table 9 shown.
  • S302. Determine the correspondence between the multiple image sensors and the multiple interfaces according to the multiple identification information and the preset information.
  • the preset information includes the correspondence between the plurality of image sensors and the plurality of identification information.
  • the identification information corresponding to the front image sensor is brand a, front-view ultra-high-definition camera
  • the identification information corresponding to the rear image sensor is brand b, front-view SD camera
  • the identification information corresponding to the left image sensor is brand c, front-view SD camera
  • the identification information corresponding to the right image sensor is brand b, rear-view SD camera.
  • Image sensors in all directions identification information front image sensor Brand a, front-view ultra-high-definition camera rear image sensor Brand b, front-view standard definition camera left image sensor Brand c, front-view standard definition camera right image sensor Brand b, rear view standard definition camera
  • the computing device can also determine the correspondence between each image sensor and each interface according to the different identification information of each image sensor.
  • the requirements for computing devices are relatively low. ,Easy to implement.
  • FIG. 6 is a schematic flowchart of a method for calibrating an image sensor provided by an embodiment of the present application. Including but not limited to the following descriptions.
  • S401 Acquire multiple images collected by multiple image sensors through multiple interfaces, and display the multiple images in a user interaction interface.
  • the plurality of image sensors collect images in respective directions, and then transmit the collected images to the computing device.
  • the computing device obtains images transmitted by the image sensors in different orientations through different interfaces.
  • the images sent by the image sensors in different orientations obtained by the computing device are different.
  • calibration objects may be respectively set in front of each image sensor.
  • each calibration object is different.
  • Each image sensor collects images of the calibration object in front of each image sensor or the scene containing the calibration object.
  • the images acquired by the sensor are different.
  • the image sensors in different orientations may only collect one image, and correspondingly, the computing device obtains an image sent by the image sensors in different orientations through different interfaces.
  • the computing device can determine the source interface of a specific image, and does not know the pair of images transmitted from the source interface and the image collected. Correspondence between image sensors. For example, referring to Table 12, Table 12 shows an example in which a computing device obtains images from different interfaces. It can be seen from Table 12 that the computing device obtains a total of 4 images, of which the image shown in Figure 3a is obtained from Interface A is transmitted in, the image shown in Figure 3b is transmitted from interface B, the image shown in Figure 3c is transmitted from interface C, and the image shown in Figure 3d is transmitted from interface D.
  • the user interaction interface can be a touch screen, which can be used to display images, wherein the user interaction interface can simultaneously display images collected by image sensors in different orientations on one page, or not simultaneously; the user interaction interface can also be used for Receive the user's click, slide and other operations, the user can see the images collected by the image sensor in different orientations through the slide, click and other operations.
  • S402. Determine and save the correspondence between the multiple image sensors and the multiple interfaces in response to the user's operation.
  • the user's operation is used to correspond the multiple images to the multiple image sensors, and the computing device determines the relationship between the multiple image sensors and the multiple interfaces according to the correspondence between the multiple images and the multiple interfaces in the above step S401. Corresponding connections between them.
  • an image sensor is connected to an interface, and the user's operation may be a sliding, a click, or other operations. It should be noted that the user associates multiple images with multiple image sensors by viewing different calibration objects located in front of different image sensors or different scenes within the field of view of different image sensors.
  • the user's operation may be that the user clicks the obtained images in a preset order.
  • the user selects the images collected by the front
  • the order of the images collected by the image sensor and the images collected by the right image sensor click on the corresponding images in turn (the user knows that the image collected by the front image sensor is shown in Figure 3a, and the image collected by the rear image sensor is shown in Figure 3b.
  • the image captured by the left image sensor is shown in Figure 3c
  • the image captured by the right image sensor is shown in Figure 3d, so click the image shown in Figure 3a, the image shown in Figure 3b, the image shown in Figure 3c, image shown in Figure 3d).
  • the computing device receives the user's operation and obtains the correspondence between the multiple image sensors and the multiple images; and then according to the correspondence between the respective images and the respective interfaces obtained in Table 12, to determine the relationship between the respective image sensors and the multiple images. For the relationship between each interface, see Table 14. It should be noted that the preset sequence in this example is preset in the computing device, and the user may click on the corresponding images according to the preset sequence.
  • the computing device uses its own user interface to display the images collected by each image sensor, and the user operates on the user interface, and the computing device determines the correspondence between each image sensor and each interface in response to the user's operation. relationship, the operation of this embodiment of the present application is simple.
  • FIG. 7 is a schematic flowchart of a method for calibrating an image sensor. The method includes but is not limited to the following descriptions.
  • the computing device acquires multiple images collected by multiple image sensors through multiple interfaces.
  • step S401 reference may be made to the description of the content of "acquiring multiple images collected by multiple image sensors through multiple interfaces" in step S401 , which is not repeated here for the sake of brevity of the description.
  • the computing device sends the multiple images to the display system, and accordingly, the display system receives the multiple images.
  • the display system can be used to display images and related content, etc., and can also be used to receive user operations, such as sliding operations, click operations.
  • the display system can be a system on the vehicle that can realize the function of displaying images and can receive user operations, or can be a display system external to the vehicle.
  • the computing device sends the obtained multiple images to the display system, and accordingly, the display system receives the multiple images sent by the computing device and displays the multiple images.
  • the display system can display images collected by image sensors in different orientations on one page at the same time, or can not display them at the same time. By clicking, sliding and other operations, the user can see the images collected by the image sensors in different orientations.
  • the display system receives the user's operation, and obtains the correspondence between the multiple images and the multiple image sensors.
  • the user's operation is the user's click or slide operation.
  • the user associates multiple images with multiple image sensors by viewing different calibration objects in front of different image sensors or different scenes within the field of view of different image sensors.
  • the user's operation may be that the user clicks the obtained images in a preset order.
  • the order of the images collected by the image sensor and the images collected by the right image sensor click on the corresponding images in turn (the user knows that the image collected by the front image sensor is shown in Figure 3a, and the image collected by the rear image sensor is shown in Figure 3b.
  • the image captured by the left image sensor is shown in Figure 3c, and the image captured by the right image sensor is shown in Figure 3d, so click the image shown in Figure 3a, the image shown in Figure 3b, the image shown in Figure 3c, image shown in Figure 3d). Therefore, the display system obtains the correspondence between the plurality of images and the plurality of image sensors.
  • the preset sequence is preset in the display system, and the user can click the corresponding images according to the preset sequence.
  • the display system sends the correspondence between the multiple images and the multiple image sensors to the computing device, and accordingly, the computing device receives the correspondence between the multiple images and the multiple image sensors.
  • the computing device determines the corresponding connection relationship between the multiple image sensors and the multiple interfaces.
  • the computing device After obtaining the correspondence between the multiple images and the multiple image sensors, the computing device determines the multiple image sensors according to the correspondence between each image and each interface in step S501 (for example, the correspondence shown in Table 12). Corresponding connection relationship with multiple interfaces (for example, the corresponding relationship shown in Table 14).
  • the computing device can also use the external display system to display the images collected by each image sensor, and the user operates on the display system, and the display system determines the relationship between each image sensor and each image in response to the user's operation.
  • the corresponding relationship between each image sensor and each interface is determined, and the corresponding relationship is sent to the computing device, so that the computing device determines the corresponding relationship between each image sensor and each interface.
  • the computing device only needs to have the functions of sending, receiving and storing.
  • FIG. 8 is a schematic structural diagram of an image sensor calibration device 800 provided by an embodiment of the present application.
  • the device 800 includes: an acquisition unit 801, configured to acquire images collected by multiple image sensors from multiple interfaces, specifically can be used to perform step S101 in the embodiment of FIG. 1, and can also be used to perform step S201 in the embodiment of FIG. 4; the identification unit 802 is used to identify the obtained image, and specifically can be used to perform the embodiment of FIG. 1
  • the step S102 in the FIG. 4 can also be used to execute the step S202 in the embodiment of FIG. 4 ; the determining unit 803 is used to determine the corresponding relationship between the multiple image sensors and the multiple interfaces, which can be specifically used to execute the step S202 in the embodiment of FIG. 1 .
  • the step S103 of FIG. 4 may also be used to execute the step S203 in the embodiment of FIG. 4 .
  • the image sensor calibration device 800 corresponds to the computing device in the method embodiment, and each unit in the image sensor calibration device 800 is used to implement various steps and methods implemented by the computing device in the method embodiment of FIG. 1 or FIG. 4 ,
  • each unit in the image sensor calibration device 800 is used to implement various steps and methods implemented by the computing device in the method embodiment of FIG. 1 or FIG. 4 .
  • the image sensor calibration device 800 calibrates the image sensor
  • only the division of the above functional units is used as an example.
  • the internal structure of the calibration device 800 is divided into different functional units to implement all or part of the functions described in the method embodiments.
  • FIG. 9 is a schematic structural diagram of another image sensor calibration device 900 provided by an embodiment of the present application.
  • the device 900 includes: an acquisition unit 901, which can be used to acquire images collected by multiple image sensors from multiple interfaces, For example, it can be used to perform the action of acquiring an image in step S401 in the embodiment of FIG. 6 , or to perform step S501 in the embodiment of FIG. 7 ; the acquiring unit 901 can also be used to acquire the identification information of the image sensor from multiple interfaces, such as It can be used to perform step S301 in the embodiment of FIG. 5; the determining unit 902 is used to determine the correspondence between multiple image sensors and multiple interfaces, for example, it can be used to perform step S302 in the embodiment of FIG. 5, and it can also be used to perform The action of determining and saving the correspondence between multiple image sensors and multiple interfaces in step S402 in the embodiment of FIG. 6 may also be used to execute step S505 in the embodiment of FIG. 7 .
  • the apparatus 900 may further include an interaction unit.
  • the interaction unit may be used to perform step S401 in the embodiment of FIG. 6 to display multiple images in the user interaction interface, and the interaction unit may also be used to perform the embodiment of FIG. 6 .
  • step S402 the operation of the user is responded to.
  • the device 900 may further include a communication unit, for example, the communication unit may be used to receive information sent by an external device or an external unit, or send information to an external device or an external unit, and specifically may be used to perform the implementation of the embodiment in FIG. 7 .
  • the image sensor calibration device 900 corresponds to the computing device in the method embodiment, and each unit in the image sensor calibration device 900 is used to implement various steps implemented by the computing device in the method embodiment of FIG. 5 or FIG. 6 or FIG. 7 . and method, for details, please refer to the description of the above method, which is not repeated here for the sake of brevity of the description.
  • the image sensor calibration device 900 calibrates the image sensor, only the division of the above functional units is used as an example.
  • the internal structure of the calibration device 900 is divided into different functional units to implement all or part of the functions described in the method embodiments.
  • FIG. 10 is a schematic structural diagram of a computing device 1000 according to an embodiment of the present application.
  • the computing device 1000 includes a processor 1010 , a communication interface 1020 , and a memory 1030 .
  • the processor 1010, the communication interface 1020 and the memory 1030 can be connected to each other through the internal bus 1040, and can also communicate through other means such as wireless transmission.
  • the bus 1040 may be a PCI bus or an EISA bus or the like.
  • the bus 1040 can be divided into an address bus, a data bus, a control bus, and the like. For ease of presentation, only one thick line is used in FIG. 10, but it does not mean that there is only one bus or one type of bus.
  • the processor 1010 may be constituted by at least one general-purpose processor, such as a CPU, or a combination of a CPU and a hardware chip.
  • the above-mentioned hardware chip may be an ASIC, a PLD or a combination thereof.
  • the above PLD can be CPLD, FPGA, GAL or any combination thereof.
  • Processor 1010 executes various types of digitally stored instructions, such as software or firmware programs stored in memory 1030, that enable computing device 1000 to provide a wide variety of services.
  • the memory 1030 is used to store program codes, and is controlled and executed by the processor 1010 to execute the steps described in the above-mentioned embodiments of FIG. 1 or FIG. 4 or FIG. 5 or FIG. 6 or FIG. Related descriptions are not repeated here.
  • the memory 1030 may include volatile memory, such as RAM; the memory 1030 may also include non-volatile memory, such as ROM, flash memory (Flash Memory); the memory 1030 may also include a combination of the above types.
  • volatile memory such as RAM
  • non-volatile memory such as ROM, flash memory (Flash Memory)
  • flash memory Flash Memory
  • the communication interface 1020 can be a wired interface (such as an Ethernet interface), an internal interface (such as a high-speed serial computer expansion bus (Peripheral Component Interconnect express, PCIE) bus interface), a wired interface (such as an Ethernet interface), or a wireless interface (such as a cellular network interface or using a wireless local area network interface) to communicate with other devices or modules.
  • a wired interface such as an Ethernet interface
  • an internal interface such as a high-speed serial computer expansion bus (Peripheral Component Interconnect express, PCIE) bus interface
  • PCIE Peripheral Component Interconnect express
  • Ethernet interface such as an Ethernet interface
  • a wireless interface such as a cellular network interface or using a wireless local area network interface
  • the computing device 1000 may further include a user interaction interface, and the user interaction interface may be used to display images, and may also be used to receive or respond to operations such as clicking, sliding, and the like by the user.
  • the user interaction interface may be used to display images, and may also be used to receive or respond to operations such as clicking, sliding, and the like by the user.
  • FIG. 10 is only a possible implementation manner of the embodiment of the present application.
  • the computing device may further include more or less components, which is not limited here.
  • An embodiment of the present application provides a computer program product, which, when the computer program product runs on a computing device, enables the computing device to execute the method described in FIG. 1 or FIG. 4 or FIG. 5 or FIG. 6 or FIG. 7 in the present application. example.
  • An embodiment of the present application provides a computer-readable storage medium, including program instructions.
  • the computer runs the program instructions, the computer executes the method embodiments described in FIG. 1 or FIG. 4 or FIG. 5 or FIG. 6 or FIG. 7 .
  • the disclosed systems, devices and methods may be implemented in other manners.
  • the apparatus embodiments described above are only illustrative.
  • the division of the unit is only a logical function division.
  • there may be other division methods for example, multiple units or components may be combined or Integration into another system, or some features can be ignored, or not implemented.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may also be electrical, mechanical or other forms of connection.
  • the unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solutions of the embodiments of the present application.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer-readable storage medium.
  • the technical solutions of the present application are essentially or part of contributions to the prior art, or all or part of the technical solutions can be embodied in the form of software products, and the computer software products are stored in a storage medium , including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program codes .
  • the above-mentioned embodiments it may be implemented in whole or in part by software, hardware, firmware or any combination thereof.
  • software it can be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer program instructions.
  • the computer program instructions When the computer program instructions are loaded and executed on a computer, the procedures or functions according to the embodiments of the present application are generated in whole or in part.
  • the computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable device.
  • the computer instructions may be stored in or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer program instructions may be transmitted from a website site, computer, server or data center via Wired or wireless transmission to another website site, computer, server or data center.
  • the computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes one or more available media integrated.
  • the available media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, digital video discs (DVDs), or semiconductor media (eg, solid state drives), and the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Vascular Medicine (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)

Abstract

The present application provides an image sensor calibration method and apparatus, a device, and a storage medium. The method comprises: obtaining, by means of a plurality of interfaces, a plurality of groups of calibration images acquired by a plurality of image sensors, wherein the plurality of interfaces, the plurality of image sensors, and the plurality of groups of calibration images respectively have one-to-one correspondence to each other; identifying a plurality of calibration objects comprised in the plurality of groups of calibration images to obtain a correspondence between the plurality of calibration objects and the plurality of interfaces, wherein the plurality of calibration objects and the plurality of groups of calibration images have one-to-one correspondence to each other; and determining a correspondence between the plurality of image sensors and the plurality of interfaces according to the correspondence between the plurality of calibration objects and the plurality of interfaces and preset information, wherein the preset information comprises a correspondence between the plurality of calibration objects and the plurality of image sensors. By implementing the present application, cable insertion errors caused by manual negligence are avoided, and the production efficiency is improved.

Description

一种图像传感器的标定方法、装置、设备及存储介质An image sensor calibration method, device, equipment and storage medium 技术领域technical field
本申请涉及通信技术领域，尤其涉及一种图像传感器的标定方法、装置、设备及存储介质。The present application relates to the field of communication technologies, and in particular, to a calibration method, apparatus, device, and storage medium for an image sensor.
背景技术Background technique
在自动驾驶系统中，车辆上可以部署多个图像传感器，多个图像传感器通过线缆与车辆上的计算设备相连接。其中，多个图像传感器(例如摄像头、激光雷达、毫米波雷达等)用于采集车辆周围环境的图像，然后将采集的多个图像通过线缆发送至计算设备，计算设备通过对多个图像进行图像拼接、图像识别等处理，实现车辆的规划和控制。In an autonomous driving system, multiple image sensors can be deployed on the vehicle, and multiple image sensors are connected to the computing device on the vehicle through cables. Among them, multiple image sensors (such as cameras, lidars, millimeter-wave radars, etc.) are used to collect images of the surrounding environment of the vehicle, and then the multiple collected images are sent to the computing device through cables, and the computing device processes the multiple images by Image stitching, image recognition and other processing to achieve vehicle planning and control.
为实现对多个图像进行处理以规划、控制车辆的目的，计算设备需要确定所获取的多个图像与各个图像传感器的对应关系。目前，需要人工根据预先设置的对应插接关系，将多个图像传感器的线缆分别对应插入计算设备相应的接口处，其中预先设置的对应插接关系为多个图像传感器与计算设备的多个接口之间的对应插接关系，且该对应插接关系已预先存储在计算设备中，因此，计算设备在接收到图像后，根据预先存储的对应插接关系就可知道该图像是由哪个图像传感器发送的。In order to realize the purpose of processing multiple images to plan and control the vehicle, the computing device needs to determine the corresponding relationship between the multiple acquired images and each image sensor. At present, it is necessary to manually insert the cables of multiple image sensors into corresponding interfaces of the computing device according to the preset corresponding plugging relationship, wherein the preset corresponding plugging relationship is multiple image sensors and multiple computing devices. The corresponding plug-in relationship between the interfaces, and the corresponding plug-in relationship has been pre-stored in the computing device. Therefore, after the computing device receives an image, it can know which image the image is based on according to the pre-stored corresponding plug-in relationship. sent by the sensor.
然而，在车辆装配过程中，由于工人疏忽等原因，容易出现线缆的插接错误，导致生产效率的降低和成本的增加。However, in the process of vehicle assembly, due to reasons such as worker's negligence, cable insertion errors are prone to occur, resulting in a decrease in production efficiency and an increase in cost.
发明内容SUMMARY OF THE INVENTION
本申请公开了一种图像传感器的标定方法、装置、设备及存储介质，避免了人为因素导致的线缆插接错误的缺陷，提高了生产效率。The present application discloses a calibration method, device, equipment and storage medium for an image sensor, which avoids the defect of incorrect cable insertion caused by human factors and improves production efficiency.
第一方面，本申请提供了一种图像传感器的标定方法，包括：通过多个接口获取多个图像传感器采集的多组标定图像，其中，所述多个接口、所述多个图像传感器以及所述多组标定图像之间分别存在一一对应关系；识别所述多组标定图像所包含的多个标定物，获得所述多个标定物与所述多个接口之间的对应关系；其中，所述多个标定物和所述多组标定图像之间存在一一对应关系；根据所述多个标定物与所述多个接口之间的对应关系以及预设信息，确定所述多个图像传感器与所述多个接口的对应关系；所述预设信息中包括所述多个标定物与所述多个图像传感器的对应关系。In a first aspect, the present application provides a method for calibrating an image sensor, comprising: acquiring multiple sets of calibration images collected by multiple image sensors through multiple interfaces, wherein the multiple interfaces, the multiple image sensors and the There is a one-to-one correspondence between the multiple sets of calibration images; identify multiple calibration objects included in the multiple sets of calibration images, and obtain the correspondence between the multiple calibration objects and the multiple interfaces; wherein, There is a one-to-one correspondence between the multiple calibration objects and the multiple sets of calibration images; the multiple images are determined according to the correspondence between the multiple calibration objects and the multiple interfaces and preset information The corresponding relationship between the sensor and the multiple interfaces; the preset information includes the corresponding relationship between the multiple calibration objects and the multiple image sensors.
可以看到，计算设备首先通过不同接口获取各个图像传感器采集到的多组标定图像，然后通过识别每组标定图像所包含的标定物，确定各个标定物与各个接口之间的对应关系，再根据该对应关系及预设信息确定各个图像传感器与各个接口之间的对应关系。实施本申请，各个图像传感器的线缆与接口可以“盲插”， 进而可由机器代替人工，自动完成线缆插接工作，避免了由于人工疏忽导致的线缆插接错误，节省了人力，提高了效率。It can be seen that the computing device first obtains multiple sets of calibration images collected by each image sensor through different interfaces, and then determines the corresponding relationship between each calibration object and each interface by identifying the calibration objects contained in each set of calibration images, and then according to The corresponding relationship and the preset information determine the corresponding relationship between each image sensor and each interface. In the implementation of the present application, the cables and interfaces of each image sensor can be "blindly inserted", and then the machine can replace the labor to automatically complete the cable insertion work, which avoids cable insertion errors caused by manual negligence, saves manpower, and improves the efficiency.
基于第一方面，在可能的实现方式中，所述多个图像传感器分别布署在车辆的不同方位上。Based on the first aspect, in a possible implementation manner, the plurality of image sensors are respectively deployed in different directions of the vehicle.
基于第一方面，在可能的实现方式中，所述多个标定物满足条件(i)-(iii)中的一项或多项：(i)所述多个标定物中至少两个标定物的颜色互不相同；(ii)所述多个标定物中至少两个标定物的形状互不相同；(iii)所述多个标定物中至少两个标定物的图案互不相同。Based on the first aspect, in a possible implementation, the plurality of calibration objects satisfy one or more of conditions (i)-(iii): (i) at least two calibration objects in the plurality of calibration objects (ii) shapes of at least two calibration objects in the plurality of calibration objects are different from each other; (iii) patterns of at least two calibration objects in the plurality of calibration objects are different from each other.
可以理解，可以通过颜色、形状和图案中的至少一项来识别标定物，或者说，位于各个图像传感器前方的标定物可以是颜色不同，也可以是形状不同，也可以是图案不同。It can be understood that the calibration objects can be identified by at least one of color, shape and pattern, or in other words, the calibration objects located in front of each image sensor can be different in color, shape or pattern.
基于第一方面，在可能的实现方式中，在所述确定所述多个图像传感器与所述多个接口的对应关系之后，所述方法还包括：获取车辆上的所述多个图像传感器采集的环境图像；根据所述多个图像传感器与所述多个接口的对应关系，对所述环境图像进行处理，获得处理结果；根据所述处理结果控制所述车辆。Based on the first aspect, in a possible implementation manner, after the determining the correspondence between the plurality of image sensors and the plurality of interfaces, the method further includes: acquiring the data collected by the plurality of image sensors on the vehicle According to the corresponding relationship between the multiple image sensors and the multiple interfaces, the environment image is processed to obtain a processing result; the vehicle is controlled according to the processing result.
可以理解，计算设备在确定了多个图像传感器与多个接口之间的对应关系后，可以利用该对应关系对各个图像传感器采集的环境图像进行处理，根据处理结果实现车辆的准确控制。It can be understood that, after determining the correspondence between the multiple image sensors and the multiple interfaces, the computing device can use the correspondence to process the environmental images collected by the respective image sensors, and accurately control the vehicle according to the processing results.
第二方面，本申请提供了一种图像传感器的标定方法，包括：在多个时刻，通过多个接口分别获取多个图像传感器采集的图像；其中，所述多个时刻中的每个时刻获取的所述图像中，至少一个图像包含标定物特征；所述多个接口与所述多个图像传感器之间存在一一对应关系；所述标定物特征包括标定物的颜色、图案、形状中的一项或多项；识别所述每个时刻获取的图像中包含所述标定物特征的图像，从而确定不同时刻所述标定物特征或包含所述标定物特征的图像与所述多个接口之间的获取关系；根据所述获取关系及预设信息，确定所述多个图像传感器与所述多个接口的对应关系；所述预设信息包括所述标定物特征出现的时序顺序。In a second aspect, the present application provides a method for calibrating an image sensor, including: acquiring images collected by a plurality of image sensors through a plurality of interfaces at a plurality of times; wherein, acquiring images at each of the plurality of times Among the images, at least one image contains the features of the calibration object; there is a one-to-one correspondence between the multiple interfaces and the multiple image sensors; the calibration object features include the color, pattern, and shape of the calibration object. One or more items; identify the images containing the features of the calibration object in the images obtained at each time, so as to determine the relationship between the features of the calibration object or the images containing the features of the calibration object at different times and the multiple interfaces The acquisition relationship between the plurality of image sensors and the plurality of interfaces is determined according to the acquisition relationship and the preset information; the preset information includes the time sequence in which the features of the calibration object appear.
可以看到，在将各个图像传感器线缆与各个接口“盲插”的情况下，计算设备可以通过识别标定物特征出现的时序顺序，来确定各个图像传感器与各个接口之间的对应关系。实施本申请，能够节省人力、提高线缆插接效率。It can be seen that when each image sensor cable is "blindly plugged" into each interface, the computing device can determine the corresponding relationship between each image sensor and each interface by identifying the time sequence in which the features of the calibration object appear. Implementing the present application can save manpower and improve the efficiency of cable plugging.
基于第一或第二方面，在可能的实现方式中，所述标定物包括：车载设备；或者，由车载设备产生的图案、颜色、形状中的至少一项。Based on the first or second aspect, in a possible implementation manner, the calibration object includes: an in-vehicle device; or at least one of a pattern, color, and shape generated by the in-vehicle device.
可以理解，标定物可以是车载设备，比如车辆上的灯光系统，包括前向照明灯、后向刹车灯、左右转向灯等，可以利用各个灯工作状态时的颜色不同，来作为不同的标定物；标定物还可以是车载设备产生的图案、颜色、形状中的至少一项，比如，可以是利用灯光系统投射出来的图案作为标定物，标定物可以是图案不同，也可以是颜色不同，也可以是形状不同；也可以利用灯光系统投射出的图案的时序顺序不同来确定各个图像传感器与各个接口之间的对应关系。It can be understood that the calibration object can be on-board equipment, such as the lighting system on the vehicle, including forward lighting, rear brake lights, left and right turn signals, etc., and the different colors of each lamp in working state can be used as different calibration objects ; The calibration object can also be at least one of the pattern, color, and shape generated by the vehicle-mounted device. For example, the pattern projected by the lighting system can be used as the calibration object. The calibration object can be different in pattern, or color, or The shape may be different; the corresponding relationship between each image sensor and each interface may also be determined by using the different timing sequences of the patterns projected by the lighting system.
基于第一或第二方面，在可能的实现方式中，所述标定物还可以包括：车外设备；或者，由车外设备产生的图案、颜色、形状中的至少一项。例如，标定物可以为设备在车辆外部的二维码，本申请标定物与车辆的位置关系不做限定。Based on the first or second aspect, in a possible implementation manner, the calibration object may further include: an off-vehicle device; or at least one of a pattern, color, and shape generated by the off-vehicle device. For example, the calibration object may be a two-dimensional code of the device outside the vehicle, and the positional relationship between the calibration object and the vehicle is not limited in the present application.
第三方面，本申请提供了一种图像传感器的标定方法，包括：通过多个接口分别获取多个标识信息；其中，所述多个接口与所述多个标识信息之间存在一一对应关系；每个标识信息中包括每个图像传感器的参数；根据所述多个标识信息和预设信息，确定出多个图像传感器与所述多个接口之间的对应关系；其中，所述预设信息中包括所述多个图像传感器与所述多个标识信息之间的对应关系。In a third aspect, the present application provides a method for calibrating an image sensor, comprising: acquiring a plurality of identification information through a plurality of interfaces respectively; wherein, there is a one-to-one correspondence between the plurality of interfaces and the plurality of identification information ; each identification information includes parameters of each image sensor; according to the plurality of identification information and preset information, determine the corresponding relationship between the plurality of image sensors and the plurality of interfaces; wherein, the preset The information includes the correspondence between the plurality of image sensors and the plurality of identification information.
可以看到，在将各个图像传感器线缆与各个接口“盲插”的情况下，计算设备可以利用各个图像传感器的不同标识信息，来确定各个图像传感器与各个接口之间的对应关系，其中标识信息包括图像传感器的参数。实施本申请，计算设备利用图像传感器的参数即可确定各个图像传感器与各个接口之间的对应关系，方案操作简单，计算设备计算量小，工作效率高。It can be seen that when each image sensor cable is "blindly plugged" with each interface, the computing device can use the different identification information of each image sensor to determine the corresponding relationship between each image sensor and each interface, wherein the identification The information includes parameters of the image sensor. In implementing the present application, the computing device can determine the corresponding relationship between each image sensor and each interface by using the parameters of the image sensor.
基于第三方面，在可能的实现方式中，所述每个图像传感器的参数包括品牌、采集图像的频率、采集图像的清晰度中的至少一项。Based on the third aspect, in a possible implementation manner, the parameters of each image sensor include at least one of a brand, a frequency of capturing images, and a definition of capturing images.
第四方面，本申请提供了一种图像传感器的标定方法，应用于计算设备，所述计算设备包括用户交互界面，所述方法包括：通过多个接口获取多个图像传感器采集的多个图像，并将所述多个图像在所述用户交互界面中显示出来；其中，所述多个接口、所述多个图像传感器以及所述多个图像之间分别存在一一对应关系；响应于用户的操作，确定并保存所述多个图像传感器与所述多个接口之间的对应关系；其中，所述用户的操作用于将所述多个图像与所述多个图像传感器进行对应。In a fourth aspect, the present application provides a method for calibrating an image sensor, which is applied to a computing device, where the computing device includes a user interaction interface, and the method includes: acquiring multiple images collected by multiple image sensors through multiple interfaces, and displaying the multiple images in the user interaction interface; wherein, there is a one-to-one correspondence between the multiple interfaces, the multiple image sensors, and the multiple images; in response to the user's An operation is performed to determine and save the correspondence between the plurality of image sensors and the plurality of interfaces; wherein, the operation of the user is used to correspond the plurality of images to the plurality of image sensors.
可以看到，在将各个图像传感器线缆与各个接口“盲插”的情况下，计算设备确定多个图像传感器与多个接口之间的对应关系的方法可以是，通过不同接口获取不同图像传感器采集的图像，利用用户交互界面将多个图像显示出来，由用户在用户交互界面上进行操作，确定多个图像与多个图像传感器的对应关系，计算设备响应于用户的操作，从而确定多个图像传感器与多个接口之间的对应关系。本方案对计算设备的要求较低，只需包括用户交互界面即可，操作简单，易于实现。It can be seen that in the case of "blindly plugging" each image sensor cable with each interface, the method for the computing device to determine the correspondence between the multiple image sensors and the multiple interfaces may be to obtain different image sensors through different interfaces. For the collected images, the user interface is used to display multiple images, and the user operates on the user interface to determine the correspondence between the multiple images and the multiple image sensors, and the computing device responds to the user's operation to determine multiple images. Correspondence between image sensors and multiple interfaces. This solution has relatively low requirements on computing equipment, and only needs to include a user interface, which is simple to operate and easy to implement.
第五方面，本申请提供了一种图像传感器的标定方法，应用于计算设备，所述计算设备与显示系统连接，所述方法包括：所述计算设备通过多个接口获取多个图像传感器采集的多个图像，并将所述多个图像发送至所述显示系统；其中，所述多个接口、所述多个图像传感器以及所述多个图像之间分别存在一一对应关系；所述显示系统接收用户的操作，获得所述多个图像与所述多个图像传感器之间的对应关系；所述显示系统将所述多个图像与所述多个图像传感器之间的对应关系发送至所述计算设备；所述计算设备确定出所述多个图像传感器与所述多个接口之间的对应关系。In a fifth aspect, the present application provides a method for calibrating an image sensor, which is applied to a computing device, where the computing device is connected to a display system, and the method includes: the computing device acquires data collected by multiple image sensors through multiple interfaces. multiple images, and send the multiple images to the display system; wherein, there is a one-to-one correspondence between the multiple interfaces, the multiple image sensors, and the multiple images; the display The system receives the user's operation, and obtains the corresponding relationship between the plurality of images and the plurality of image sensors; the display system sends the corresponding relationship between the plurality of images and the plurality of image sensors to the plurality of image sensors. The computing device; the computing device determines the correspondence between the plurality of image sensors and the plurality of interfaces.
可以看到，在将各个图像传感器线缆与各个接口“盲插”的情况下，计算设 备确定多个图像传感器与多个接口之间的对应关系的方法可以是，计算设备将获取的各个图像传感器采集的多个图像发送至显示系统，显示系统接收并显示多个图像，由用户在显示系统上进行操作，显示系统接收并响应于用户的操作，获得多个图像与多个图像传感器之间的对应关系，并将该对应关系发送至计算设备，计算设备根据接收到的对应关系确定出多个图像传感器与多个接口的对应关系。It can be seen that in the case of "blindly plugging" each image sensor cable with each interface, the method for the computing device to determine the correspondence between the multiple image sensors and the multiple interfaces may be: The multiple images collected by the sensor are sent to the display system, the display system receives and displays the multiple images, the user operates on the display system, the display system receives and responds to the user's operation, and obtains the relationship between multiple images and multiple image sensors. The corresponding relationship is sent to the computing device, and the computing device determines the corresponding relationship between the multiple image sensors and the multiple interfaces according to the received corresponding relationship.
第六方面，本申请提供了一种图像传感器的标定装置，包括：获取单元，用于通过多个接口获取多个图像传感器采集的多组标定图像，其中，所述多个接口、所述多个图像传感器以及所述多组标定图像之间分别存在一一对应关系；识别单元，用于识别所述多组标定图像所包含的多个标定物，获得所述多个标定物与所述多个接口之间的对应关系；其中，所述多个标定物和所述多组标定图像之间存在一一对应关系；确定单元，用于根据所述多个标定物与所述多个接口之间的对应关系以及预设信息，确定所述多个图像传感器与所述多个接口的对应关系；所述预设信息中包括所述多个标定物与所述多个图像传感器的对应关系。In a sixth aspect, the present application provides an image sensor calibration device, comprising: an acquisition unit configured to acquire multiple sets of calibration images collected by multiple image sensors through multiple interfaces, wherein the multiple interfaces, the multiple There is a one-to-one correspondence between each image sensor and the multiple sets of calibration images; the identification unit is used to identify multiple calibration objects included in the multiple sets of calibration images, and obtain the multiple calibration objects and the multiple calibration objects. The correspondence between the interfaces; wherein, there is a one-to-one correspondence between the multiple calibration objects and the multiple sets of calibration images; the determining unit is configured to determine the relationship between the multiple calibration objects and the multiple interfaces according to the relationship between the multiple calibration objects and the multiple interfaces. The corresponding relationship between the multiple image sensors and the multiple interfaces is determined, and the preset information includes the corresponding relationship between the multiple calibration objects and the multiple image sensors.
基于第六方面，在可能的实现方式中，所述多个图像传感器分别布署在车辆的不同方位上。Based on the sixth aspect, in a possible implementation manner, the multiple image sensors are respectively deployed in different directions of the vehicle.
基于第六方面，在可能的实现方式中，所述多个标定物满足条件(i)-(iii)中的一项或多项：(i)所述多个标定物中至少两个标定物的颜色互不相同；(ii)所述多个标定物中至少两个标定物的形状互不相同；(iii)所述多个标定物中至少两个标定物的图案互不相同。Based on the sixth aspect, in a possible implementation, the plurality of calibration objects satisfy one or more of the conditions (i)-(iii): (i) at least two calibration objects in the plurality of calibration objects (ii) shapes of at least two calibration objects in the plurality of calibration objects are different from each other; (iii) patterns of at least two calibration objects in the plurality of calibration objects are different from each other.
基于第六方面，在可能的实现方式中，所述获取单元还用于，获取车辆上的所述多个图像传感器采集的环境图像；所述装置还包括：处理单元，用于根据所述多个图像传感器与所述多个接口的对应关系，对所述环境图像进行处理，获得处理结果；控制单元，用于根据所述处理结果控制所述车辆。Based on the sixth aspect, in a possible implementation manner, the acquiring unit is further configured to acquire environmental images collected by the multiple image sensors on the vehicle; the apparatus further includes: a processing unit configured to The corresponding relationship between each image sensor and the plurality of interfaces is to process the environment image to obtain a processing result; a control unit is used to control the vehicle according to the processing result.
上述第六方面的装置中的各个功能单元用于实现上述第一方面及第一方面的任一实现方式所述的方法。Each functional unit in the apparatus of the sixth aspect is used to implement the method described in the first aspect and any implementation manner of the first aspect.
第七方面，本申请提供了一种图像传感器的标定装置，包括：获取单元，用于在多个时刻，通过多个接口分别获取多个图像传感器采集的图像；其中，所述多个时刻中的每个时刻获取的所述图像中，至少一个图像包含标定物特征；所述多个接口与所述多个图像传感器之间存在一一对应关系；所述标定物特征包括标定物的颜色、图案、形状中的一项或多项；识别单元，用于识别所述每个时刻获取的图像中包含所述标定物特征的图像，从而确定不同时刻所述标定物特征或包含所述标定物特征的图像与所述多个接口之间的获取关系；确定单元，用于根据所述获取关系及预设信息，确定所述多个图像传感器与所述多个接口的对应关系；所述预设信息包括所述标定物特征出现的时序顺序。In a seventh aspect, the present application provides an image sensor calibration device, comprising: an acquisition unit configured to acquire images collected by a plurality of image sensors through a plurality of interfaces at a plurality of times; wherein, among the plurality of times Among the images obtained at each moment of , at least one image contains the features of the calibration object; there is a one-to-one correspondence between the multiple interfaces and the multiple image sensors; the calibration object features include the color of the calibration object, One or more of patterns and shapes; an identification unit, configured to identify the images containing the features of the calibration objects in the images obtained at each moment, so as to determine the features of the calibration objects at different moments or the inclusion of the calibration objects the acquisition relationship between the image of the feature and the multiple interfaces; the determining unit is configured to determine the corresponding relationship between the multiple image sensors and the multiple interfaces according to the acquired relationship and preset information; the preset Let the information include the chronological order in which the calibrator features appear.
基于第六方面或第七方面，在可能的实现方式中，所述标定物包括：车载设备；或者，由车载设备产生的图案、颜色、形状中的至少一项。Based on the sixth aspect or the seventh aspect, in a possible implementation manner, the calibration object includes: an in-vehicle device; or, at least one of a pattern, color, and shape generated by the in-vehicle device.
基于第六方面或第七方面，在可能的实现方式中，所述标定物还可以包括：车外设备；或者，由车外设备产生的图案、颜色、形状中的至少一项。例如，标 定物可以为设备在车辆外部的二维码，本申请标定物与车辆的位置关系不做限定。Based on the sixth aspect or the seventh aspect, in a possible implementation manner, the calibration object may further include: an off-vehicle device; or at least one of a pattern, color, and shape generated by the off-vehicle device. For example, the calibration object may be a two-dimensional code of the device outside the vehicle, and the positional relationship between the calibration object and the vehicle is not limited in this application.
上述第七方面的装置中的各个功能单元用于实现上述第二方面及第二方面的任一实现方式所述的方法。Each functional unit in the apparatus of the seventh aspect is used to implement the method described in the second aspect and any implementation manner of the second aspect.
第八方面，本申请提供了一种图像传感器的标定装置，包括：获取单元，用于通过多个接口分别获取多个标识信息；其中，所述多个接口与所述多个标识信息之间存在一一对应关系；每个标识信息中包括每个图像传感器的参数；确定单元，用于根据所述多个标识信息和预设信息，确定出多个图像传感器与所述多个接口之间的对应关系；其中，所述预设信息中包括所述多个图像传感器与所述多个标识信息之间的对应关系。In an eighth aspect, the present application provides a calibration device for an image sensor, comprising: an acquisition unit configured to acquire a plurality of identification information through a plurality of interfaces respectively; wherein, between the plurality of interfaces and the plurality of identification information There is a one-to-one correspondence; each identification information includes parameters of each image sensor; a determining unit is configured to determine the relationship between the multiple image sensors and the multiple interfaces according to the multiple identification information and preset information The corresponding relationship between the plurality of image sensors and the plurality of identification information is included in the preset information.
基于第八方面，在可能的实现方式中，所述每个图像传感器的参数包括品牌、采集图像的频率、采集图像的清晰度中的至少一项。Based on the eighth aspect, in a possible implementation manner, the parameters of each image sensor include at least one of a brand, a frequency of capturing images, and a definition of capturing images.
上述第八方面的装置中的各个功能单元用于实现上述第三方面及第三方面的任一实现方式所述的方法。Each functional unit in the apparatus of the foregoing eighth aspect is used to implement the method described in the foregoing third aspect and any implementation manner of the third aspect.
第九方面，本申请提供了一种图像传感器的标定装置，所述装置包括用户交互界面，所述装置包括：获取单元，用于通过多个接口获取多个图像传感器采集的多个图像，并将所述多个图像在所述用户交互界面中显示出来；其中，所述多个接口、所述多个图像传感器以及所述多个图像之间分别存在一一对应关系；响应单元，用于响应于用户的操作，确定并保存所述多个图像传感器与所述多个接口之间的对应关系；其中，所述用户的操作用于将所述多个图像与所述多个图像传感器进行对应。In a ninth aspect, the present application provides an image sensor calibration device, the device includes a user interaction interface, the device includes: an acquisition unit, configured to acquire multiple images collected by multiple image sensors through multiple interfaces, and displaying the multiple images in the user interaction interface; wherein, there is a one-to-one correspondence between the multiple interfaces, the multiple image sensors, and the multiple images; the response unit is used for In response to a user's operation, the correspondence between the plurality of image sensors and the plurality of interfaces is determined and saved; wherein the user's operation is used to perform a comparison between the plurality of images and the plurality of image sensors correspond.
上述第九方面的装置中的各个功能单元用于实现上述第四方面及第四方面的任一实现方式所述的方法。Each functional unit in the apparatus of the ninth aspect above is used to implement the method described in the fourth aspect and any implementation manner of the fourth aspect.
第十方面，本申请提供了一种图像传感器的标定装置，所述装置与显示系统连接，所述装置包括：获取单元，用于所述计算设备通过多个接口获取多个图像传感器采集的多个图像，并将所述多个图像发送至所述显示系统；其中，所述多个接口、所述多个图像传感器以及所述多个图像之间分别存在一一对应关系；通信单元，用于接收所述显示系统发送的所述多个图像与所述多个图像传感器之间的对应关系；确定单元，用于确定出所述多个图像传感器与所述多个接口之间的对应关系。In a tenth aspect, the present application provides an image sensor calibration device, the device is connected to a display system, and the device includes: an acquisition unit, used for the computing device to acquire multiple data collected by multiple image sensors through multiple interfaces images, and send the multiple images to the display system; wherein, there is a one-to-one correspondence between the multiple interfaces, the multiple image sensors, and the multiple images; the communication unit, using receiving the correspondence between the plurality of images sent by the display system and the plurality of image sensors; a determining unit for determining the correspondence between the plurality of image sensors and the plurality of interfaces .
上述第十方面的装置中的各个功能单元用于实现上述第五方面及第五方面的任一实现方式所述的方法。Each functional unit in the apparatus of the tenth aspect is configured to implement the method described in the fifth aspect and any implementation manner of the fifth aspect.
第十一方面，本申请提供了一种计算设备，该计算设备包括存储器和处理器，所述存储器用于存储指令，所述处理器用于调用所述存储器中存储的指令执行上述第一方面及第一方面的任一实现方式中所述的方法，或者第二方面及第二方面的任一实现方式中所述的方法，或者第三方面及第三方面的任一实现方式中所述的方法，或者第四方面及第四方面的任一实现方式中所述的方法，或者第五方面及第五方面的任一实现方式中所述的方法。In an eleventh aspect, the present application provides a computing device, the computing device includes a memory and a processor, the memory is used for storing instructions, and the processor is used for calling the instructions stored in the memory to execute the above-mentioned first aspect and The method described in any implementation manner of the first aspect, or the method described in any implementation manner of the second aspect and the second aspect, or the method described in any implementation manner of the third aspect and the third aspect A method, or the method described in any implementation manner of the fourth aspect and the fourth aspect, or the method described in any implementation manner of the fifth aspect and the fifth aspect.
第十二方面，本申请提供了一种计算机存储介质，包括程序指令，当计算机 运行所述程序指令时，使得计算机执行上述第一方面及第一方面的任一实现方式中所述的方法，或者第二方面及第二方面的任一实现方式中所述的方法，或者第三方面及第三方面的任一实现方式中所述的方法，或者第四方面及第四方面的任一实现方式中所述的方法，或者第五方面及第五方面的任一实现方式中所述的方法。In a twelfth aspect, the present application provides a computer storage medium, comprising program instructions, when the computer runs the program instructions, the computer causes the computer to execute the method described in the first aspect and any implementation manner of the first aspect, Or the method described in any implementation manner of the second aspect and the second aspect, or the method described in any implementation manner of the third aspect and the third aspect, or any implementation of the fourth aspect and the fourth aspect The method described in the manner, or the method described in any implementation manner of the fifth aspect and the fifth aspect.
第十三方面，本申请提供了一种计算机程序产品，包括程序指令，当该程序产品被计算设备执行时，使得计算设备执行上述第一方面及第一方面的任一实现方式中所述的方法，或者第二方面及第二方面的任一实现方式中所述的方法，或者第三方面及第三方面的任一实现方式中所述的方法，或者第四方面及第四方面的任一实现方式中所述的方法，或者第五方面及第五方面的任一实现方式中所述的方法。该计算机程序产品可以为一个软件安装包，在需要使用前述任一方面的任一种可能的设计提供的方法的情况下，可以下载该计算机程序产品并在计算设备上执行该计算机程序产品，以实现上述任一方面的任一可能实现方式中所述的方法。According to a thirteenth aspect, the present application provides a computer program product, including program instructions, which, when the program product is executed by a computing device, cause the computing device to execute the first aspect and any implementation manner of the first aspect. The method, or the method described in any implementation manner of the second aspect and the second aspect, or the method described in any implementation manner of the third aspect and the third aspect, or the fourth aspect and the fourth aspect. The method described in an implementation manner, or the method described in any implementation manner of the fifth aspect and the fifth aspect. The computer program product can be a software installation package, which can be downloaded and executed on a computing device if the method provided by any of the possible designs of any of the preceding aspects needs to be used to A method as described in any possible implementation of any of the above aspects is implemented.
可以看到，为了避免由于人工疏忽导致的线缆插接错误，本申请提供了一种图像传感器的标定方法，根据各个图像传感器采集的图像中包括的标定物，或者根据图像中标定物特征出现的时序顺序，或者根据不同图像传感器的不同标识信息，来确定各个图像传感器与各个接口之间的对应关系；或者，利用用户交互界面或显示系统来确定各个图像传感器与各个接口之间的对应关系。实施本申请，提高了插接的准确率和生产效率。It can be seen that, in order to avoid cable insertion errors caused by human negligence, the present application provides a method for calibrating an image sensor. The timing sequence of each image sensor, or the corresponding relationship between each image sensor and each interface is determined according to the different identification information of different image sensors; or, the user interaction interface or display system is used to determine the corresponding relationship between each image sensor and each interface. . By implementing the present application, the accuracy and production efficiency of plugging are improved.
附图说明Description of drawings
为了更清楚地说明本申请实施例技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to explain the technical solutions of the embodiments of the present application more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. For those of ordinary skill, other drawings can also be obtained from these drawings without any creative effort.
图1为本申请实施例提供的一种图像传感器的标定方法的流程示意图；1 is a schematic flowchart of a method for calibrating an image sensor according to an embodiment of the present application;
图2为本申请实施例提供的一种图像传感器的标定方法的场景示例图；FIG. 2 is a scene example diagram of a calibration method for an image sensor provided by an embodiment of the present application;
图3a为本申请实施例提供的一种标定物示例图；Fig. 3a is an example diagram of a calibration object provided by the embodiment of the present application;
图3b为本申请实施例提供的一种标定物示例图；Fig. 3b is an example diagram of a calibration object provided by the embodiment of the application;
图3c为本申请实施例提供的一种标定物示例图；Fig. 3c is an example diagram of a calibration object provided by the embodiment of the present application;
图3d为本申请实施例提供的一种标定物示例图；FIG. 3d is an example diagram of a calibration object provided by the embodiment of the present application;
图4为本申请实施例提供的又一种图像传感器的标定方法的流程示意图；FIG. 4 is a schematic flowchart of another method for calibrating an image sensor provided by an embodiment of the present application;
图5为本申请实施例提供的又一种图像传感器的标定方法的流程示意图；FIG. 5 is a schematic flowchart of another method for calibrating an image sensor provided by an embodiment of the present application;
图6为本申请实施例提供的又一种图像传感器的标定方法的流程示意图；FIG. 6 is a schematic flowchart of another image sensor calibration method provided by an embodiment of the present application;
图7为本申请实施例提供的又一种图像传感器的标定方法的流程示意图；7 is a schematic flowchart of another image sensor calibration method provided by an embodiment of the present application;
图8为本申请实施例提供的一种图像传感器的标定装置示意图；8 is a schematic diagram of a calibration device for an image sensor provided by an embodiment of the present application;
图9为本申请实施例提供的又一种图像传感器的标定装置示意图；9 is a schematic diagram of another image sensor calibration device provided by an embodiment of the present application;
图10为本申请实施例提供的一种计算设备的结构示意图。FIG. 10 is a schematic structural diagram of a computing device according to an embodiment of the present application.
具体实施方式Detailed ways
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述。The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application.
图像传感器用于监测车辆周围的环境，采集图像。在具有高级驾驶辅助系统(advanced driving assistance system，ADAS)的车辆中，图像传感器一般为多个，多个图像传感器分别布署于车辆的不同方位上。在部分车型上，多个图像传感器总的视野范围覆盖车辆所在的整个水平空间。图像传感器可以是摄像头、激光雷达、毫米波雷达等。Image sensors are used to monitor the environment around the vehicle and capture images. In a vehicle with an advanced driving assistance system (ADAS), there are generally multiple image sensors, and the multiple image sensors are respectively deployed on different positions of the vehicle. On some models, the total field of view of multiple image sensors covers the entire horizontal space where the vehicle is located. The image sensor can be a camera, lidar, millimeter wave radar, etc.
计算设备用于对多个图像传感器采集的环境图像进行处理，实现路线规划、控制决策等。其中，计算设备可以位于车辆上的多域控制器/移动数据中心(multiple domain controller/mobile data center，MDC)内，也可以位于车辆上的座舱域控制器(cockpit domain controller，CDC)内，计算设备还可以位于车辆的电子控制单元(electronic control unit，ECU)中。The computing device is used to process the environmental images collected by multiple image sensors to realize route planning, control decision-making, and the like. The computing device may be located in a multiple domain controller/mobile data center (MDC) on the vehicle, or may be located in a cockpit domain controller (CDC) on the vehicle. The device may also be located in the electronic control unit (ECU) of the vehicle.
车辆的计算设备上配置有多个线缆接口，多个图像传感器分别通过多个线缆接口与计算设备连接。计算设备中预先设定了各个线缆接口与各个图像传感器的对应连接关系，其中，每个线缆接口对应连接一个方位上的图像传感器，不同线缆接口对应连接不同方位上的图像传感器。因此，在车辆出厂前，工人需要将各个图像传感器的线缆插入到计算设备对应的线缆接口中。为了简化产线难度，降低线缆插接出错的风险，一般将各个图像传感器的线缆设计为不同颜色，在插接时，工人将不同颜色的线缆分别对应插接到相应的线缆接口上，或者也可以将各个图像传感器的线缆端口设计为不同形状，工人根据线缆端口形状将各个图像传感器的线缆分别插接到对应的线缆接口上，从而实现各个图像传感器与计算设备上各个线缆接口之间的连接。The computing device of the vehicle is configured with multiple cable interfaces, and the multiple image sensors are respectively connected to the computing device through the multiple cable interfaces. The corresponding connection relationship between each cable interface and each image sensor is preset in the computing device, wherein each cable interface corresponds to an image sensor in one orientation, and different cable interfaces correspond to image sensors in different orientations. Therefore, before the vehicle leaves the factory, the worker needs to insert the cable of each image sensor into the corresponding cable interface of the computing device. In order to simplify the difficulty of the production line and reduce the risk of cable plugging errors, the cables of each image sensor are generally designed to be of different colors. When plugging, workers plug the cables of different colors into the corresponding cable interfaces. Alternatively, the cable ports of each image sensor can be designed in different shapes, and the worker plugs the cables of each image sensor to the corresponding cable interface according to the shape of the cable port, so as to realize the connection between each image sensor and the computing device. connection between the various cable ports on the
装配完成后，各个方位的图像传感器采集相应方向上的图像，并通过各自对应的线缆接口将采集的图像传输至计算设备中，计算设备对各个方向上的图像进行处理，通过分析、决策实现路线规划、车辆控制等，从而实现辅助驾驶或自动驾驶功能。After the assembly is completed, the image sensors in all directions collect the images in the corresponding directions, and transmit the collected images to the computing device through the corresponding cable interface. Route planning, vehicle control, etc., to achieve assisted driving or automatic driving functions.
但是，上述方法中，线缆和线缆接口需要特殊设计，增加成本；车辆装配过程中，需要工人根据颜色或线缆端口形状将各个图像传感器的线缆插入至计算设备的线缆接口中，耗费人力，且由于工人疏忽可能出现插接错误的问题，一旦出现插接错误，则需要车辆拆卸、重装，既耗费人力、物力等，又增加成本费用。However, in the above method, cables and cable interfaces need special design, which increases costs; during vehicle assembly, workers are required to insert the cables of each image sensor into the cable interface of the computing device according to the color or the shape of the cable port. It is labor-intensive, and due to the negligence of the workers, there may be a problem of wrong plugging. Once the plugging error occurs, the vehicle needs to be disassembled and reassembled, which not only consumes manpower, material resources, etc., but also increases costs.
为了克服上述方法中人工插接图像传感器线缆存在的问题，本申请实施例提供了一种支持线缆盲插的方法，即一种图像传感器的标定方法，在介绍本方法实施例之前，先说明本方法实施例中各个图像传感器与计算设备之间的连接方式。In order to overcome the problem of manually plugging the image sensor cable in the above method, the embodiment of the present application provides a method for supporting blind plugging of the cable, that is, a calibration method for the image sensor. Before introducing the embodiment of the method, first The connection mode between each image sensor and the computing device in the embodiment of this method is described.
车辆上各个图像传感器的线缆可以是相同的，无需设计为不同颜色或不同形 状的端口，可以通过机器或人工进行“盲插”，也就是说，各个图像传感器的线缆与计算设备的多个接口之间可以任意插接，在插接完成后，由计算设备确定各个图像传感器与计算设备的各个接口之间的对应连接关系。The cables of each image sensor on the vehicle can be the same, and there is no need to design ports of different colors or different shapes. It can be "blind-plugged" by machine or manually, that is, the cables of each image sensor and the computing device are many. The interfaces can be arbitrarily plugged and connected, and after the plugging is completed, the corresponding connection relationship between each image sensor and each interface of the computing device is determined by the computing device.
下面介绍一下本申请实施例提供的一种图像传感器的标定方法，该方法应用于车辆的计算设备，其中计算设备可以是多域控制器/移动数据中心MDC座舱域控制器CDC或电子控制单元ECU等。参见图1，图1为本申请实施例提供的一种图像传感器的方位标定方法的流程示意图，该方法包括但不限于以下内容的描述。The following describes a method for calibrating an image sensor provided by an embodiment of the present application. The method is applied to a computing device of a vehicle, where the computing device may be a multi-domain controller/mobile data center MDC cockpit domain controller CDC or electronic control unit ECU Wait. Referring to FIG. 1 , FIG. 1 is a schematic flowchart of a method for calibrating an orientation of an image sensor according to an embodiment of the present application, and the method includes but is not limited to the description of the following contents.
S101、通过多个接口获取多个图像传感器采集的多个标定图像。S101. Acquire multiple calibration images collected by multiple image sensors through multiple interfaces.
多个图像传感器分别布署于车辆的不同方位上，每个图像传感器用于采集各自方位上的图像。车辆上图像传感器的具体数量本申请不做具体限定，一般来讲，车辆上所有图像传感器总的视野范围覆盖了车辆所在的整个水平空间，或者说，车辆上多个图像传感器采集的多个图像中包含了车辆周围空间的环境信息。图像传感器可以是摄像头、激光雷达、毫米波雷达等。A plurality of image sensors are respectively deployed in different directions of the vehicle, and each image sensor is used to collect images in respective directions. The specific number of image sensors on the vehicle is not specifically limited in this application. Generally speaking, the total field of view of all image sensors on the vehicle covers the entire horizontal space where the vehicle is located, or in other words, multiple images collected by multiple image sensors on the vehicle. contains the environmental information of the space around the vehicle. The image sensor can be a camera, lidar, millimeter wave radar, etc.
每个图像传感器采集的图像数量可以是一张，也可以是多张，本申请对每个图像传感器采集的图像的数量不做具体限定。The number of images collected by each image sensor may be one or multiple, and the application does not specifically limit the number of images collected by each image sensor.
在一种示例中，在各个图像传感器的前方(各个图像传感器的前方，指的是各个图像传感器的视野范围内的位置)分别放置有不同的标定物，不同的标定物可以是不同颜色的标定灯。例如，参见图2所示的示例图，在图2中，车辆上布署了4个图像传感器，4个图像传感器分别位于车辆的不同方位上(大概位于车辆的前侧、后侧、左侧、右侧各一个)，为了便于描述，这里分别称为前图像传感器、后图像传感器、左图像传感器、右图像传感器。分别在前图像传感器、后图像传感器、左图像传感器、右图像传感器的前方放置不同颜色的标定灯，例如，在前图像传感器的前方放置白色标定灯，在后图像传感器的前方放置红色标定灯，在左图像传感器的前方放置黄色标定灯，在右图像传感器的前方放置绿色标定灯。In an example, different calibration objects are placed in front of each image sensor (the front of each image sensor, which refers to the position within the field of view of each image sensor), and the different calibration objects may be calibrations of different colors lamp. For example, referring to the example diagram shown in FIG. 2, in FIG. 2, 4 image sensors are deployed on the vehicle, and the 4 image sensors are located in different directions of the vehicle (probably located on the front side, rear side, and left side of the vehicle). , one on the right side), for convenience of description, they are referred to here as a front image sensor, a rear image sensor, a left image sensor, and a right image sensor, respectively. Place calibration lights of different colors in front of the front image sensor, rear image sensor, left image sensor, and right image sensor. For example, place a white calibration light in front of the front image sensor, and place a red calibration light in front of the rear image sensor. Place a yellow calibration light in front of the left image sensor and a green calibration light in front of the right image sensor.
各个图像传感器进行图像采集，各个图像传感器采集的图像的颜色不同，其中，前图像传感器采集的图像的颜色主要为白色，后图像传感器采集的图像的颜色主要为红色，左图像传感器采集的图像的颜色主要为黄色，右图像传感器采集的图像的颜色主要为绿色。Each image sensor collects images, and the colors of the images collected by each image sensor are different. The color of the image collected by the front image sensor is mainly white, the color of the image collected by the rear image sensor is mainly red, and the color of the image collected by the left image sensor is mainly white. The color is mainly yellow, and the color of the image captured by the right image sensor is mainly green.
各个图像传感器将采集的图像通过多个接口(线缆接口)传输至计算设备，相应的，计算设备通过多个接口获取多个图像传感器采集的多个图像。其中，每个图像传感器的线缆通过一个接口与计算设备相连接。需要说明的是，由于各个图像传感器与线缆接口之间是“盲插”的，所以计算设备此时无法确认各个线缆接口是与各个图像传感器之间是怎样对应连接的，或者说，计算设备无法确认各个线缆接口接收的图像是哪个图像传感器传输的。Each image sensor transmits the collected images to the computing device through multiple interfaces (cable interfaces), and correspondingly, the computing device obtains multiple images collected by multiple image sensors through multiple interfaces. Wherein, the cable of each image sensor is connected with the computing device through an interface. It should be noted that due to the "blind insertion" between each image sensor and the cable interface, the computing device cannot confirm how each cable interface is correspondingly connected to each image sensor at this time. The device cannot confirm which image sensor transmits the image received by each cable interface.
需要说明的是，各个图像传感器可以布署于车体的内部，也可以布署于车身的外部，本申请不做限定。It should be noted that each image sensor may be deployed inside the vehicle body, or may be deployed outside the vehicle body, which is not limited in this application.
在又一种示例中，在各个图像传感器的前方(各个图像传感器的前方，指的 是各个图像传感器的视野范围内的位置)分别放置有不同的标定物，标定物可以是不同图案或具有不同图案的物品。例如，将图2示例中不同颜色的标定灯替换为不同图案，比如，将图2中前图像传感器前方的白色标定灯替换为图3a所示的图案，将图2中后图像传感器前方的红色标定灯替换为图3b所示的图案，将图2中左图像传感器前方的黄色标定灯替换为图3c所示的图案，将图2中右图像传感器前方的绿色标定灯替换为图3d所示的图案。In yet another example, different calibration objects are respectively placed in front of each image sensor (the front of each image sensor, which refers to the position within the field of view of each image sensor), and the calibration objects may be of different patterns or have different patterned items. For example, replace the calibration lights of different colors in the example in Figure 2 with different patterns, for example, replace the white calibration lights in front of the front image sensor in Figure 2 with the pattern shown in Figure 3a, and replace the red in front of the rear image sensor in Figure 2. Replace the calibration light with the pattern shown in Figure 3b, replace the yellow calibration light in front of the left image sensor in Figure 2 with the pattern shown in Figure 3c, and replace the green calibration light in front of the right image sensor in Figure 2 with the pattern shown in Figure 3d picture of.
各个图像传感器进行图像采集，并将采集的图像通过相连的线缆接口传输至计算设备，相应的，计算设备接收到各个图像传感器传输的图像。其中各个图像传感器传输的图像的图案不同。Each image sensor performs image acquisition, and transmits the acquired image to the computing device through the connected cable interface, and correspondingly, the computing device receives the image transmitted by each image sensor. The patterns of the images transmitted by the respective image sensors are different.
需要说明的是，本实施例中，各个图像传感器前方的标定物可以是不同图案或具有不同图案的物品，还可以是不同颜色的灯或具有不同颜色的物品，还可以是不同形状的物品，等等。另外，还可以利用车辆周围所在的场景作为标定物，能够利用车辆周围所在的场景作为标定物的前提是，各个图像传感器所在视野范围内的场景有所区别，或者说，若利用各个图像传感器采集图像，各个图像传感器所采集得到的图像不完全相同。本申请对标定物不做具体限定，只要各个图像传感器采集所得到的图像不完全相同即可。It should be noted that, in this embodiment, the calibration objects in front of each image sensor may be objects with different patterns or patterns, lamps of different colors or objects with different colors, or objects with different shapes. and many more. In addition, the scene around the vehicle can also be used as the calibration object. The premise that the scene around the vehicle can be used as the calibration object is that the scenes within the field of view of each image sensor are different. The images acquired by each image sensor are not exactly the same. The application does not specifically limit the calibration object, as long as the images acquired by each image sensor are not identical.
需要说明的是，一般将标定物布署在各自图像传感器的视野范围内，且与其他任何一个图像传感器的视野范围不重合的范围内。优选的，将标定物布署在各自图像传感器正对的位置。标定物距离图像传感器的距离不做限定。It should be noted that the calibration objects are generally deployed within the field of view of each image sensor, and within a range that does not overlap with the field of view of any other image sensor. Preferably, the calibration objects are arranged at the positions facing the respective image sensors. The distance between the calibration object and the image sensor is not limited.
S102、识别多个标定图像中所包含的多个标定物，获得多个标定物与多个接口之间的对应关系。S102: Identify multiple calibration objects included in the multiple calibration images, and obtain the correspondence between the multiple calibration objects and the multiple interfaces.
多个标定物可以是颜色不同，也可以是形状不同，也可以是图案不同。例如，在图2示例中，各个图像传感器前方设置有不同颜色的标定灯，则计算设备获得的多个图像中的多个标定物的颜色不同，其中颜色包括白色、红色、黄色、绿色。需要说明的是，这里标定灯的颜色指的是标定灯处于工作状态时的颜色。又例如，在图3a至图3d示例中，各个图像传感器前方设置有不同图案，则计算设备获得的多个图像中的多个标定物的图案不同，其中图案包括图3a、图3b、图3c和图3d所示的图案。The multiple calibration objects may be different in color, shape or pattern. For example, in the example of FIG. 2 , calibration lights of different colors are arranged in front of each image sensor, and the colors of the calibration objects in the images obtained by the computing device are different, and the colors include white, red, yellow, and green. It should be noted that the color of the calibration light here refers to the color of the calibration light when it is in a working state. For another example, in the examples of FIGS. 3 a to 3 d , different patterns are set in front of each image sensor, and the patterns of the multiple calibration objects in the multiple images obtained by the computing device are different, wherein the patterns include FIG. 3 a , FIG. 3 b , and FIG. 3 c and the pattern shown in Figure 3d.
计算设备识别多个图像中所包含的多个标定物，可以通过识别图像中的颜色或图像中的图案或图像中的形状，来识别图像中所包含的标定物，从而知道了各个图像是从哪个接口传输进来的，也就是说，获得了各个标定物(各个图像)与各个接口之间的对应关系。例如，表1给出了图2示例中具有不同颜色的标定物与接口之间的对应关系，由表1可知，从接口1、2、3、4获取的图像所包含的标定物的颜色分别是白色、红色、黄色、绿色。The computing device recognizes multiple calibration objects contained in multiple images, and can recognize the calibration objects included in the image by identifying the color in the image or the pattern in the image or the shape in the image, so as to know that each image is derived from the image. Which interface is transmitted in, that is, the corresponding relationship between each calibration object (each image) and each interface is obtained. For example, Table 1 shows the correspondence between the calibration objects with different colors and the interface in the example of Figure 2. It can be seen from Table 1 that the colors of the calibration objects included in the images obtained from interfaces 1, 2, 3, and 4 are respectively It's white, red, yellow, green.
表1Table 1
接口interface 标定物的颜色Color of the calibrator
11 白色White
22 红色red
33 黄色yellow
44 绿色green
又例如，表2给出了不同标定物与接口之间的对应关系，由表2可知，从接口1、2、3、4获取的图像所包含的标定物分别是图3a所示的标定物、图3b所示的标定物、图3c所示的标定物和图3d所示的标定物。For another example, Table 2 shows the correspondence between different calibration objects and interfaces. It can be seen from Table 2 that the calibration objects included in the images obtained from interfaces 1, 2, 3, and 4 are the calibration objects shown in Figure 3a, respectively. , the calibration object shown in Figure 3b, the calibration object shown in Figure 3c, and the calibration object shown in Figure 3d.
表2Table 2
接口interface 标定物Calibrator
11 图3a所示的标定物The calibrator shown in Figure 3a
22 图3b所示的标定物The calibrator shown in Figure 3b
33 图3c所示的标定物The calibrator shown in Figure 3c
44 图3d所示的标定物The calibrator shown in Figure 3d
需要说明的是，识别图像可以是通过提取特征的方式识别，也可以是将图像输入预先训练好的识别模型中进行识别，其中识别模型可以是循环神经网络模型、卷积神经网络模型等，本申请对识别方式不做限定。It should be noted that the recognition image can be recognized by extracting features, or it can be recognized by inputting the image into a pre-trained recognition model, where the recognition model can be a recurrent neural network model, a convolutional neural network model, etc. The application does not limit the identification method.
为了更好地理解本申请，下面对标定物的形状、图案和颜色进行说明。For a better understanding of the present application, the shape, pattern and color of the calibration object are described below.
本申请中，标定物的形状指的是标定物的存在或表现形式，通常形状是由物体或实物的轮廓或边缘构成的，例如，长方形、圆形、椭圆形、环形等。In this application, the shape of the calibration object refers to the existence or manifestation of the calibration object, and the shape is usually formed by the outline or edge of an object or real object, such as rectangle, circle, ellipse, ring, etc.
标定物的图案指的是承载在具有一定形状的实体或实物之上的设计纹路，本申请中的图案不包括颜色信息。The pattern of the calibration object refers to the design pattern carried on the entity or object with a certain shape, and the pattern in this application does not include color information.
标定物的颜色也是承载在具有一定形状的实体或实物之上的，标定物的颜色指的可以是标定物的形状的颜色，也可以是标定物的图案的颜色。The color of the calibration object is also carried on the entity or object with a certain shape, and the color of the calibration object may refer to the color of the shape of the calibration object or the color of the pattern of the calibration object.
S103、根据多个标定物与多个接口之间的对应关系以及预设信息，确定多个图像传感器与多个接口的对应关系。S103: Determine the correspondence between the multiple image sensors and the multiple interfaces according to the correspondence between the multiple calibration objects and the multiple interfaces and the preset information.
预设信息中包括多个标定物与多个图像传感器之间的对应关系。例如，表3示出的是图2示例中计算设备中的预设信息，该预设信息给出了各方位图像传感器与不同颜色的标定物之间的对应关系。The preset information includes the correspondence between the plurality of calibration objects and the plurality of image sensors. For example, Table 3 shows the preset information in the computing device in the example of FIG. 2 , where the preset information provides the corresponding relationship between each azimuth image sensor and calibration objects of different colors.
表3table 3
Figure PCTCN2021083466-appb-000001
Figure PCTCN2021083466-appb-000001
表4示出的是计算设备中的预设信息，该预设信息给出了各方位图像传感器与不同标定物之间的对应关系。Table 4 shows the preset information in the computing device, and the preset information provides the corresponding relationship between the image sensors in various orientations and different calibration objects.
表4Table 4
各方位图像传感器All azimuth image sensors 标定物Calibrator
前图像传感器front image sensor 图3a所示的标定物The calibrator shown in Figure 3a
后图像传感器rear image sensor 图3b所示的标定物The calibrator shown in Figure 3b
左图像传感器left image sensor 图3c所示的标定物The calibrator shown in Figure 3c
右图像传感器right image sensor 图3d所示的标定物The calibrator shown in Figure 3d
根据多个图像的图像信息和预设信息，可以确定出车辆上的多个图像传感器与多个接口之间的对应连接关系，其中，每个图像传感器对应一个接口，从而确定出各个图像传感器采集的是哪个方位的图像。例如，根据表1示出的各方位图像传感器与各接口之间的关系以及表3中的预设信息，可以确定出：前图像传感器的线缆是与计算设备的1接口相连接的，后图像传感器的线缆是与计算设备的2接口相连接的，左图像传感器的线缆是与计算设备的3接口相连接的，右图像传感器的线缆是与计算设备的4接口相连接的，如表5所示。又例如，根据表2示出的各方位图像传感器与各接口之间的关系以及表4中的预设信息，同样可以确定出：前图像传感器的线缆是与计算设备的1接口相连接的，后图像传感器的线缆是与计算设备的2接口相连接的，左图像传感器的线缆是与计算设备的3接口相连接的，右图像传感器的线缆是与计算设备的4接口相连接的。According to the image information and preset information of the multiple images, the corresponding connection relationship between the multiple image sensors on the vehicle and the multiple interfaces can be determined, wherein each image sensor corresponds to an interface, so as to determine the acquisition of each image sensor. which orientation is the image in. For example, according to the relationship between each azimuth image sensor and each interface shown in Table 1 and the preset information in Table 3, it can be determined that the cable of the front image sensor is connected to the 1 interface of the computing device, and the rear The cable of the image sensor is connected to the 2 port of the computing device, the cable of the left image sensor is connected to the 3 port of the computing device, and the cable of the right image sensor is connected to the 4 port of the computing device. As shown in Table 5. For another example, according to the relationship between each azimuth image sensor and each interface shown in Table 2 and the preset information in Table 4, it can also be determined that the cable of the front image sensor is connected to the 1 interface of the computing device , the cable of the rear image sensor is connected to the 2 port of the computing device, the cable of the left image sensor is connected to the 3 port of the computing device, and the cable of the right image sensor is connected to the 4 port of the computing device of.
表5table 5
Figure PCTCN2021083466-appb-000002
Figure PCTCN2021083466-appb-000002
可选的，在确定出车辆上的多个图像传感器与多个接口的对应关系之后，将该对应关系保存至计算设备中，以便在以后的应用中，直接使用该对应关系，无需再次进行匹配。例如，在一种应用场景中，车辆装配时，使用本实施例的方法对多个图像传感器进行标定，确定出多个图像传感器与多个接口之间的对应关系，保存该对应关系。在车辆使用、维修、调试时，对车辆上不同方位的图像传感器进行参数设置，例如，设置图像传感器的曝光度、设置图像传感器采集图像的频率等，在参数设置后，计算设备获取各个图像传感器采集的环境图像，然后根据采集到的多个图像进行拼接、处理等，可以实现对车辆的控制，例如，实现车辆的辅助驾驶或自动驾驶功能。Optionally, after the correspondence between the multiple image sensors on the vehicle and the multiple interfaces is determined, the correspondence is saved to the computing device, so that the correspondence can be used directly in future applications without the need to perform matching again. . For example, in an application scenario, during vehicle assembly, the method of this embodiment is used to calibrate multiple image sensors, determine the correspondence between multiple image sensors and multiple interfaces, and save the correspondence. When the vehicle is used, maintained, and debugged, parameters are set for image sensors in different directions on the vehicle, for example, setting the exposure of the image sensor, setting the frequency of image acquisition by the image sensor, etc. After the parameters are set, the computing device obtains each image sensor. The collected environmental images are then stitched and processed according to the multiple collected images, so as to realize the control of the vehicle, for example, to realize the assisted driving or automatic driving function of the vehicle.
需要说明的是，本申请中的车辆不限于汽车，也可以是其他类型的任意车辆，甚至是自行车、三轮车等，只要车辆上存在具有运算能力的设备(计算设备)、多个图像传感器和多个接口，即可实现对多个图像传感器的标定。It should be noted that the vehicle in this application is not limited to an automobile, but can also be any other type of vehicle, even a bicycle, a tricycle, etc., as long as there are devices (computing devices) with computing capabilities, multiple image sensors and multiple With only one interface, the calibration of multiple image sensors can be realized.
可以看到，计算设备根据各个图像传感器采集的标定图像中所包含的标定物，来确定各个图像传感器与各个接口之间的对应关系，实施本申请实施例，各个图像传感器线缆与各个接口进行“盲插”，避免了由于工人疏忽导致线缆插接错误的弊端，且节省人力，提高了插接效率。It can be seen that the computing device determines the correspondence between each image sensor and each interface according to the calibration objects contained in the calibration images collected by each image sensor, and to implement the embodiments of the present application, each image sensor cable is connected to each interface. "Blind insertion" avoids the disadvantages of wrong cable insertion due to worker's negligence, saves manpower, and improves insertion efficiency.
本申请实施例还提供了一种图像传感器的标定方法，参见图4所示，图4为一种图像传感器的标定方法的流程示意图，该方法包括但不限于以下内容的描述。An embodiment of the present application further provides a method for calibrating an image sensor. Referring to FIG. 4 , FIG. 4 is a schematic flowchart of a method for calibrating an image sensor. The method includes but is not limited to the following descriptions.
S201、在多个时刻，通过多个接口分别获取多个图像传感器采集的图像。S201. Obtain images collected by multiple image sensors through multiple interfaces at multiple times.
本实施例中，各个图像传感器的布署方位、各个图像传感器采集图像的方式和数量参考上述实施例中步骤S101相关内容的描述，为了说明书的简洁，在此不再赘述。In this embodiment, reference to the description of the relevant content of step S101 in the above-mentioned embodiment for the deployment orientation of each image sensor, the manner and quantity of image acquisition by each image sensor, and for the sake of brevity of the description, details are not repeated here.
本示例中需要标定物，该标定物可以为某一颜色的标定灯或具有某图案或某形状的标定物等。标定物可以是按照预设顺序或预设时刻放置在各个图像传感器前方的，例如，预设时刻可以是，t1时刻标定物放置在前图像传感器的前方，t2时刻标定物放置在右图像传感器的前方，t3时刻标定物放置在后图像传感器的前方，t4时刻标定物放置在左图像传感器的前方，其中按时间先后顺序为t1、t2、t3、t4。预设顺序可以是，先将标定物放置在前图像传感器的前方，再将标定物放置在右图像传感器的前方，其次将标定物放置在后图像传感器的前方，最后将标定物放置在左图像传感器的前方。In this example, a calibration object is required, and the calibration object can be a calibration light of a certain color or a calibration object with a certain pattern or shape, etc. The calibration objects can be placed in front of each image sensor in a preset order or at a preset time. For example, the preset time can be that the calibration object is placed in front of the front image sensor at time t1, and the calibration object is placed in front of the right image sensor at time t2. In the front, the calibration object is placed in front of the rear image sensor at time t3, and the calibration object is placed in front of the left image sensor at time t4, wherein the time sequence is t1, t2, t3, t4. The preset sequence can be, first place the calibration object in front of the front image sensor, then place the calibration object in front of the right image sensor, then place the calibration object in front of the rear image sensor, and finally place the calibration object in the left image front of the sensor.
各个图像传感器在不同时刻采集图像，并将采集的图像通过不同的接口传输至计算设备，相应的，计算设备在不同时刻，接收到各个图像传感器采集的图像。其中计算设备每个时刻获取的图像中，至少一个图像中包含标定物，其他图像中不包含。Each image sensor collects images at different times, and transmits the collected images to the computing device through different interfaces. Correspondingly, the computing device receives the images collected by each image sensor at different times. Among the images acquired by the computing device at each moment, at least one image contains a calibration object, and other images do not contain it.
例如，首先，标定物放置在前图像传感器的前方(其他图像传感器的前方未放置标定物)，前图像传感器、后图像传感器、左图像传感器、右图像传感器均采集图像，并分别通过不同的接口传输至计算设备，在t1时刻计算设备通过不同的接口接收各个图像传感器采集的图像；然后，将标定物移动到右图像传感器的前方，前图像传感器、后图像传感器、左图像传感器、右图像传感器均采集图像，并分别通过不同的接口传输至计算设备，在t2时刻计算设备通过不同的接口接收各个图像传感器采集的图像；其次，将标定物移动到后图像传感器的前方，前图像传感器、后图像传感器、左图像传感器、右图像传感器均采集图像，并分别通过不同的接口传输至计算设备，t3时刻计算设备通过不同的接口接收各个图像传感器采集的图像；最后，将标定物移动到左图像传感器的前方，前图像传感器、后图像传感器、左图像传感器、右图像传感器均采集图像，并分别通过不同的接口传输至计算设备，t4时刻计算设备通过不同的接口接收各个图像传感器采集的图像。可以理解的，标定物在不同时刻出现在不同图像传感器前方位置的方式不仅限于移动标的物的方式，例如，也可以通过在前图像传感器、后图像传感器、左图像传感器、右图像传感器设置标定灯，并在不同时刻使其点亮的方式，达到标定物在不同时刻出现在不同图像传感器前方位置的效果。For example, first, the calibration object is placed in front of the front image sensor (the calibration object is not placed in front of other image sensors), and the front image sensor, rear image sensor, left image sensor, and right image sensor all collect images, and pass through different interfaces. It is transmitted to the computing device, and the computing device receives the images collected by each image sensor through different interfaces at time t1; then, move the calibration object to the front of the right image sensor, the front image sensor, the rear image sensor, the left image sensor, and the right image sensor. All images are collected and transmitted to the computing device through different interfaces, and the computing device receives the images collected by each image sensor through different interfaces at t2; secondly, move the calibration object to the front of the rear image sensor, the front image sensor, the rear The image sensor, the left image sensor, and the right image sensor all collect images and transmit them to the computing device through different interfaces. At t3, the computing device receives the images collected by each image sensor through different interfaces; finally, move the calibration object to the left image In front of the sensor, the front image sensor, the rear image sensor, the left image sensor, and the right image sensor all collect images and transmit them to the computing device through different interfaces respectively. At t4, the computing device receives the images collected by each image sensor through different interfaces. It can be understood that the way that the calibration object appears in front of different image sensors at different times is not limited to the way of moving the object. For example, calibration lights can also be set by the front image sensor, the rear image sensor, the left image sensor, and the right image sensor. , and make it light up at different times to achieve the effect of the calibration object appearing in front of different image sensors at different times.
S202、识别每个时刻获取的图像中包含标定物特征的图像，从而确定不同时刻包含标定物特征的图像与多个接口之间的获取关系。S202: Identify the images containing the features of the calibration object in the images acquired at each time, so as to determine the acquisition relationship between the images containing the features of the calibration object at different times and multiple interfaces.
每个时刻获取的图像中，一定存在特定图像中包含标定物特征，其他图像中 不包含标定物特征。其中，标定物特征可以是标定物的颜色或标定物的图案或标定物的形状等。In the images obtained at each moment, there must be certain images that contain the features of the calibration object, and other images that do not contain the features of the calibration objects. The feature of the calibration object may be the color of the calibration object, the pattern of the calibration object, or the shape of the calibration object, and the like.
对各个时刻中每个时刻获取的图像进行识别。识别颜色信息或图案信息或形状信息等，从而确定出包含标定物特征的图像。在对各个时刻获取的图像识别结束后，计算设备获取了在不同时刻包含标定物特征的图像的来源接口。或者，计算设备获取在不同时刻的标定物特征的来源接口。可以理解的，通过识别每个时刻获取的图像中包含标定物特征的图像，从而确定不同时刻标定物特征与多个接口之间的获取关系，可以实现本实施例的目的。The images acquired at each moment in each moment are identified. Identify color information, pattern information, or shape information, etc., to determine the image containing the characteristics of the target. After recognizing the images acquired at each moment, the computing device acquires the source interface of the images containing the features of the calibration object at different moments. Alternatively, the computing device obtains the source interface of the calibrator features at different times. It can be understood that the purpose of this embodiment can be achieved by identifying an image containing the features of the calibration object in the images acquired at each moment, so as to determine the acquisition relationship between the features of the calibration object at different times and multiple interfaces.
例如，标定物为黄色标定灯，首先，计算设备对t1时刻获取的图像进行识别，识别出包含黄色标定灯的图像，也就确定出了t1时刻包含黄色标定灯的图像是从哪个接口获取得到的，比如是从1接口获取得到的；然后，计算设备对t2时刻获取的图像进行识别，识别出包含黄色标定灯的图像，也就确定出了t2时刻包含黄色标定灯的图像是从哪个接口获取得到的，比如是从2接口获取得到的；其次，对t3时刻获取的图像进行识别，识别出包含黄色标定灯的图像，也就确定出了t3时刻包含黄色标定灯的图像是从哪个接口获取得到的，比如是从3接口获取得到的；最后，对t4时刻获取的图像进行识别，识别出包含黄色标定灯的图像，也就确定出了t4时刻包含黄色标定灯的图像是从哪个接口获取得到的，比如是从4接口获取得到的。因此，得到不同时刻包含标定物的图像与多个接口之间的获取关系，参见表6所示。For example, if the calibration object is a yellow calibration light, first, the computing device recognizes the image obtained at time t1, identifies the image containing the yellow calibration light, and determines from which interface the image containing the yellow calibration light is obtained at time t1 For example, it is obtained from interface 1; then, the computing device recognizes the image obtained at time t2, identifies the image containing the yellow calibration light, and determines which interface the image containing the yellow calibration light comes from at time t2 Obtained, for example, obtained from interface 2; secondly, identify the image obtained at time t3, identify the image containing the yellow calibration light, and determine which interface the image containing the yellow calibration light is from at time t3 Obtained, for example, obtained from interface 3; finally, the image obtained at time t4 is identified, and the image containing the yellow calibration light is identified, which also determines which interface the image containing the yellow calibration light is from at time t4. Obtained, for example, obtained from the 4 interface. Therefore, the acquisition relationship between images containing calibration objects at different times and multiple interfaces is obtained, as shown in Table 6.
表6Table 6
Figure PCTCN2021083466-appb-000003
Figure PCTCN2021083466-appb-000003
S203、根据获取关系及预设信息，确定多个图像传感器与多个接口的对应关系。S203. Determine the corresponding relationship between the multiple image sensors and the multiple interfaces according to the acquired relationship and the preset information.
预设信息中包括标定物特征出现的时序顺序，也就是，在各个图像传感器的前方设置所述标定物的时序顺序，或者说，标定物在各个图像传感器的前方出现的先后顺序。例如，标定物首先设置在哪个图像传感器的前方，然后设置在哪个图像传感器的前方，其次设置在哪个图像传感器的前方，最后设置在哪个图像传感器的前方。例如，参见表7所示，表7示例性地示出了一种预设信息，该预设信息中示出了在各个图像传感器的前方设置同一标定物的时序顺序(先后顺序)，由该表可知，首先，标定物是位于前图像传感器的前方的，然后，移动至右图像 传感器的前方，其次，移动至后图像传感器的前方，最后，移动至左图像传感器的前方，或者说，标定物的移动顺序是前图像传感器、右图像传感器、后图像传感器、左图像传感器。The preset information includes the chronological sequence in which the features of the calibration objects appear, that is, the temporal sequence in which the calibration objects are set in front of each image sensor, or the sequence in which the calibration objects appear in front of each image sensor. For example, the calibration object is first set in front of which image sensor, then in front of which image sensor, second in front of which image sensor, and finally in front of which image sensor. For example, referring to Table 7, Table 7 exemplarily shows a kind of preset information. The preset information shows the time sequence (sequence) of setting the same calibration object in front of each image sensor. It can be seen from the table that, first, the calibration object is located in front of the front image sensor, then it moves to the front of the right image sensor, secondly, it moves to the front of the rear image sensor, and finally, it moves to the front of the left image sensor, or in other words, the calibration The moving order of the object is front image sensor, right image sensor, rear image sensor, left image sensor.
表7Table 7
Figure PCTCN2021083466-appb-000004
Figure PCTCN2021083466-appb-000004
根据不同时刻包含标定物特征的图像与多个接口之间的获取关系及预设信息，确定出多个图像传感器与多个接口的对应关系。例如，根据表6和表7，可以得出，前图像传感器是对应计算设备的1接口，右图像传感器是对应计算设备的2接口，后图像传感器是对应计算设备的3接口，左图像传感器是对应计算设备的4接口，即表8。The corresponding relationship between the multiple image sensors and the multiple interfaces is determined according to the acquisition relationship and preset information between the images containing the features of the calibration object and the multiple interfaces at different times. For example, according to Table 6 and Table 7, it can be concluded that the front image sensor is the 1 interface corresponding to the computing device, the right image sensor is the 2 interface corresponding to the computing device, the rear image sensor is the 3 interface corresponding to the computing device, and the left image sensor is the Corresponding to the 4 interfaces of the computing device, that is, Table 8.
表8Table 8
Figure PCTCN2021083466-appb-000005
Figure PCTCN2021083466-appb-000005
需要说明的是，上述描述中，是以设置了一个标定物，将一个标定物按照预设信息中的预设顺序或预设时刻进行移动为例来说明的。本示例中还可以设置多个标定物，多个标定物可以是颜色不同、图案不同或形状不同。It should be noted that, in the above description, a calibration object is set, and a calibration object is moved according to the preset order or preset time in the preset information as an example for description. In this example, multiple calibration objects may also be set, and the multiple calibration objects may be of different colors, different patterns or different shapes.
例如，1)在t11时刻，标定物1放置在前图像传感器的前方，其他图像传感器的前方未放置标定物，各个图像传感器采集图像，并将采集的图像通过不同接口传输至计算设备，相应的，计算设备接收到各个图像传感器在t11时刻采集的图像；在t22时刻，标定物2放置在右图像传感器的前方，其他图像传感器的前方未放置标定物，各个图像传感器采集图像，并将采集的图像通过不同接口传输至计算设备，相应的，计算设备接收到各个图像传感器在t22时刻采集的图像；在t33时刻，标定物3放置在后图像传感器的前方，其他图像传感器的前方未放置标定物，各个图像传感器采集图像，并将采集的图像通过不同接口传输至计算设备，相应的，计算设备接收到各个图像传感器在t33时刻采集的图像；在t44时刻，标定物4放置在左图像传感器的前方，其他图像传感器的前方未放置标定物，各个图像传感器采集图像，并将采集的图像通过不同接口传输至计算设备，相应的，计算设备接收到各个图像传感器在t44时刻采集的图像。2)计算设备对各个时刻中每个时刻接收到的图像进行识别，根据标定物的颜色或图案或形状 等，识别出哪个时刻的图像中包含标定物1的图像，哪个时刻的图像中包含标定物2的图像，哪个时刻的图像中包含标定物3的图像，以及哪个时刻的图像中包含标定物4的图像，从而确定出分别包含标定物1、标定物2、标定物3、标定物4的图像是从哪个接口获取的。3)根据预设信息以及2)中的识别结果，进一步确定出各个接口与各个图像传感器之间的对应关系。For example, 1) At time t11, the calibration object 1 is placed in front of the front image sensor, and no calibration objects are placed in front of other image sensors. Each image sensor collects images and transmits the collected images to the computing device through different interfaces. The corresponding , the computing device receives the images collected by each image sensor at time t11; at time t22, the calibration object 2 is placed in front of the right image sensor, and no calibration objects are placed in front of other image sensors, each image sensor collects images, and the collected The image is transmitted to the computing device through different interfaces. Correspondingly, the computing device receives the image collected by each image sensor at time t22; at time t33, the calibration object 3 is placed in front of the rear image sensor, and no calibration object is placed in front of other image sensors. , each image sensor collects images, and transmits the collected images to the computing device through different interfaces. Correspondingly, the computing device receives the images collected by each image sensor at time t33; at time t44, the calibration object 4 is placed on the left image sensor. In the front, no calibration objects are placed in front of other image sensors. Each image sensor collects images and transmits the collected images to the computing device through different interfaces. Correspondingly, the computing device receives the images collected by each image sensor at time t44. 2) The computing device recognizes the images received at each moment in each moment, and according to the color, pattern or shape of the calibration object, etc., identifies which moment the image contains the image of the calibration object 1, and which moment the image contains the calibration object. The image of object 2, the image at which time contains the image of calibration object 3, and the image at which time contains the image of calibration object 4, so as to determine whether the image of calibration object 1, calibration object 2, calibration object 3, and calibration object 4 are included. The interface from which the image is obtained. 3) According to the preset information and the recognition result in 2), further determine the correspondence between each interface and each image sensor.
需要说明的是，t1/t2/t3/t4，t11/t22/t33/t44的仅仅是一个便于描述的方式，本发明实施例的本质是通过标定物出现的先后时序来确定传感器与接口连接的关系，t1/t2/t3/t4，t11/t22/t33/t44具体取值，不是本实施例的重点。It should be noted that t1/t2/t3/t4, t11/t22/t33/t44 is only a convenient way to describe, the essence of the embodiment of the present invention is to determine the connection between the sensor and the interface through the sequence of the appearance of the calibration objects relationship, the specific values of t1/t2/t3/t4 and t11/t22/t33/t44 are not the focus of this embodiment.
在一种示例中，还可以利用车辆照明系统，其中车辆照明系统包括车辆上的照明灯和信号灯，确定各个接口与各个图像传感器的对应关系(标定物包括车辆上的照明灯和信号灯，标定物的特征为照明灯或信号灯处于工作状态的情况下，发出的颜色)。例如，1)分时控制车辆上的灯。首先，打开车辆的前向照明灯，例如近光灯或远光灯，其中，前向照明灯发出的光近似为白光，前图像传感器、后图像传感器、左图像传感器、右图像传感器分别采集图像，发送图像至计算设备；然后，关闭前向照明灯，打开后向刹车灯，后向刹车灯发出的光近似为红色，前图像传感器、后图像传感器、左图像传感器、右图像传感器分别采集图像，发送图像至计算设备；其次，关闭后向刹车灯，打开左转向灯，左转向灯发出的光近似为黄光，前图像传感器、后图像传感器、左图像传感器、右图像传感器分别采集图像，发送图像至计算设备；最后，关闭左转向灯，打开右转向灯，右转向灯发出的光也近似为黄色，前图像传感器、后图像传感器、左图像传感器、右图像传感器分别采集图像，发送图像至计算设备。2)通过对不同批次/不同时刻接收到的图像进行识别，根据颜色识别图像，从而可以确定出第一批次/第一个接收时刻的图像中，包含白色光的图像是从哪个接口获取得到的，第二批次/第二个接收时刻的图像中，包含红色光的图像是从哪个接口获取得到的，第三批次/第三个接收时刻的图像中，包含黄色光的图像是从哪个接口获取得到的，第四个批次/第四个接收时刻的图像中，包含黄色光的图像是从哪个接口获取得到的。3)根据2)的结果以及预设信息中保存的灯的控制顺序，确定出各个接口与各个图像传感器之间的对应关系。In an example, a vehicle lighting system can also be used, wherein the vehicle lighting system includes lights and signal lights on the vehicle, and the corresponding relationship between each interface and each image sensor is determined (the calibration object includes the lights and signal lights on the vehicle, the calibration object is characterized by the color emitted when the light or signal light is in operation). For example, 1) time-sharing control of the lights on the vehicle. First, turn on the front lights of the vehicle, such as low beams or high beams, where the light emitted by the front lights is approximately white light, and the front image sensor, rear image sensor, left image sensor, and right image sensor capture images respectively. , send the image to the computing device; then, turn off the forward lighting, turn on the rear brake light, the light emitted by the rear brake light is approximately red, and the front image sensor, rear image sensor, left image sensor, and right image sensor collect images respectively. , send the image to the computing device; secondly, turn off the rear brake light, turn on the left turn signal, the light emitted by the left turn signal is approximately yellow light, and the front image sensor, rear image sensor, left image sensor, and right image sensor collect images respectively, Send the image to the computing device; finally, turn off the left turn signal, turn on the right turn signal, the light emitted by the right turn signal is also approximately yellow, the front image sensor, rear image sensor, left image sensor, right image sensor respectively collect images, and send the images to the computing device. 2) By identifying the images received in different batches/at different times, and identifying the images according to their colors, it is possible to determine from which interface the images containing white light are obtained in the images of the first batch/the first receiving moment. Obtained, in the image of the second batch/second receiving moment, the image containing red light is obtained from which interface, and in the image of the third batch/third receiving moment, the image containing yellow light is From which interface is obtained, and from which interface the image containing yellow light is obtained in the images of the fourth batch/fourth receiving moment. 3) According to the result of 2) and the control sequence of the lamps stored in the preset information, the corresponding relationship between each interface and each image sensor is determined.
需要说明的是，还可以利用车辆照明系统中各个照明灯或各个信号灯的闪烁频率不同，或者，在可能的实现中，利用各个照明灯或各个信号灯在地面的投影图案不同，来确定出不同时刻所采集的图像与多个接口之间的对应关系，从而进一步确定出多个图像传感器与多个接口之间的对应关系。It should be noted that the different flashing frequencies of each lighting lamp or each signal lamp in the vehicle lighting system can also be used, or, in a possible implementation, the different projection patterns of each lighting lamp or each signal lamp on the ground can be used to determine different times. The corresponding relationship between the collected images and the multiple interfaces is further determined, thereby further determining the corresponding relationship between the multiple image sensors and the multiple interfaces.
可以看到，在“盲插”的情况下，计算设备确定各个图像传感器与各个接口之间的对应关系的方法，还可以是根据获取各个图像传感器采集的图像的时序顺序，也就是，计算设备通过识别不同时刻获取的图像中包含的标定物特征出现的时序顺序，进一步确定各个图像传感器与各个接口之间的对应关系。It can be seen that in the case of "blind insertion", the method for the computing device to determine the correspondence between each image sensor and each interface can also be based on the sequential order of acquiring images collected by each image sensor, that is, the computing device. The corresponding relationship between each image sensor and each interface is further determined by identifying the temporal sequence of the appearance of the features of the calibration object contained in the images obtained at different times.
本申请实施例还提供了一种图像传感器的标定方法，该方法应用于车辆上的 计算设备，参见图5所示，图5为本申请实施例提供的一种图像传感器的标定方法的流程示意图，该方法包括但不限于以下内容的描述。An embodiment of the present application further provides a method for calibrating an image sensor, and the method is applied to a computing device on a vehicle. Referring to FIG. 5 , FIG. 5 is a schematic flowchart of a method for calibrating an image sensor provided by an embodiment of the present application. , the method includes but is not limited to the description of the following contents.
S301、通过多个接口分别获取多个标识信息。S301. Acquire multiple pieces of identification information respectively through multiple interfaces.
关于多个图像传感器的方位布署参考S101中相关内容的描述，为了说明书的简洁，在此不再赘述。For the azimuth arrangement of the multiple image sensors, refer to the description of the related content in S101 , which is not repeated here for the sake of brevity of the description.
每个标识信息中包括每个图像传感器的参数，例如图像传感器的品牌、分辨率等。Each identification information includes parameters of each image sensor, such as the brand, resolution, etc. of the image sensor.
计算设备通过多个接口分别获取多个标识信息。例如，计算设备通过接口A获取其中一个图像传感器的标识信息，标识信息包括品牌a、前视超高清摄像；通过接口B获取另一个图像传感器的标识信息，标识信息包括品牌b、前视标清摄像；通过接口C获取另一个图像传感器的标识信息，标识信息包括品牌c、前视标清摄像；通过接口D获取另一个图像传感器的标识信息，标识信息包括品牌b、后视标清摄像，如表9所示。The computing device obtains a plurality of pieces of identification information respectively through a plurality of interfaces. For example, the computing device obtains the identification information of one of the image sensors through interface A, and the identification information includes brand a and front-view ultra-high-definition camera; through interface B, obtains the identification information of the other image sensor, and the identification information includes brand b, forward-looking standard definition camera Obtain the identification information of another image sensor through interface C, the identification information includes brand c, front-view standard definition camera; Obtain the identification information of another image sensor through interface D, and the identification information includes brand b, rear-view standard-definition camera, as shown in Table 9 shown.
表9Table 9
计算设备的各接口Interfaces of computing devices 标识信息identification information
AA 品牌a、前视超高清摄像Brand a, front-view ultra-high-definition camera
BB 品牌b、前视标清摄像Brand b, front-view standard definition camera
CC 品牌c、前视标清摄像Brand c, front-view standard definition camera
DD 品牌b、后视标清摄像Brand b, rear view standard definition camera
S302、根据多个标识信息和预设信息，确定出多个图像传感器与多个接口之间的对应关系。S302. Determine the correspondence between the multiple image sensors and the multiple interfaces according to the multiple identification information and the preset information.
预设信息包括多个图像传感器与多个标识信息之间的对应关系。例如，参见表10所示，由表10的预设信息可知，前图像传感器对应的标识信息为品牌a、前视超高清摄像，后图像传感器对应的标识信息为品牌b、前视标清摄像，左图像传感器对应的标识信息为品牌c、前视标清摄像，右图像传感器对应的标识信息为品牌b、后视标清摄像。The preset information includes the correspondence between the plurality of image sensors and the plurality of identification information. For example, referring to Table 10, it can be seen from the preset information in Table 10 that the identification information corresponding to the front image sensor is brand a, front-view ultra-high-definition camera, and the identification information corresponding to the rear image sensor is brand b, front-view SD camera, The identification information corresponding to the left image sensor is brand c, front-view SD camera, and the identification information corresponding to the right image sensor is brand b, rear-view SD camera.
表10Table 10
各方位的图像传感器Image sensors in all directions 标识信息identification information
前图像传感器front image sensor 品牌a、前视超高清摄像Brand a, front-view ultra-high-definition camera
后图像传感器rear image sensor 品牌b、前视标清摄像Brand b, front-view standard definition camera
左图像传感器left image sensor 品牌c、前视标清摄像Brand c, front-view standard definition camera
右图像传感器right image sensor 品牌b、后视标清摄像Brand b, rear view standard definition camera
根据表9中从各个接口获得的标识信息和表10中各个方位的图像传感器的标识信息，可以确定出各个图像传感器与各个接口之间的对应关系，得到表11。According to the identification information obtained from each interface in Table 9 and the identification information of the image sensor in each orientation in Table 10, the correspondence between each image sensor and each interface can be determined, and Table 11 is obtained.
表11Table 11
各方位的图像传感器Image sensors in all directions 计算设备的各接口Interfaces of computing devices
前图像传感器front image sensor AA
后图像传感器rear image sensor BB
左图像传感器left image sensor CC
右图像传感器right image sensor DD
可以看到，在“盲插”的情况下，计算设备还可以根据各个图像传感器的标识信息不同，确定各个图像传感器与各个接口之间的对应关系，本申请中，对计算设备的要求较低，易于实现。It can be seen that in the case of "blind insertion", the computing device can also determine the correspondence between each image sensor and each interface according to the different identification information of each image sensor. In this application, the requirements for computing devices are relatively low. ,Easy to implement.
本申请实施例还提供了一种图像传感器的标定方法，该方法应用于计算设备，参见图6所示，图6为本申请实施例提供的一种图像传感器的标定方法的流程示意图，该方法包括但不限于以下内容的描述。An embodiment of the present application also provides a method for calibrating an image sensor, and the method is applied to a computing device. Referring to FIG. 6 , FIG. 6 is a schematic flowchart of a method for calibrating an image sensor provided by an embodiment of the present application. Including but not limited to the following descriptions.
S401、通过多个接口获取多个图像传感器采集的多个图像，并将多个图像在用户交互界面中显示出来。S401. Acquire multiple images collected by multiple image sensors through multiple interfaces, and display the multiple images in a user interaction interface.
关于多个图像传感器的方位布署参考上述步骤S101中相关内容的描述，为了说明书的简洁，在此不再赘述。Regarding the azimuth arrangement of the multiple image sensors, reference is made to the description of the relevant content in the foregoing step S101 , which is not repeated here for the sake of brevity of the description.
多个图像传感器采集各自方向上的图像，然后将采集的图像传输至计算设备，相应的，计算设备通过不同接口获得不同方位的图像传感器传输的图像。The plurality of image sensors collect images in respective directions, and then transmit the collected images to the computing device. Correspondingly, the computing device obtains images transmitted by the image sensors in different orientations through different interfaces.
计算设备获得的不同方位的图像传感器发送的图像是不同的。在一种示例中，在各个图像传感器的前方可分别设置有标定物，一般各个标定物是不同的，各个图像传感器对各自前方的标定物或包含标定物的场景进行图像采集，不同方位的图像传感器采集所得到的图像是不同的。本实施例中，各个方位的图像传感器可以只采集一张图像，相应的，计算设备分别通过不同接口获得不同方位的图像传感器发送的一张图像。The images sent by the image sensors in different orientations obtained by the computing device are different. In an example, calibration objects may be respectively set in front of each image sensor. Generally, each calibration object is different. Each image sensor collects images of the calibration object in front of each image sensor or the scene containing the calibration object. The images acquired by the sensor are different. In this embodiment, the image sensors in different orientations may only collect one image, and correspondingly, the computing device obtains an image sent by the image sensors in different orientations through different interfaces.
需要说明的是，由于各个图像传感器的线缆与各个接口之间是“盲插”的，计算设备可以确定特定图像的来源接口，并不知道从来源接口传输进来的图像与采集该图像对的图像传感器之间的对应关系。例如，参见表12所示，表12示出了一种计算设备从不同接口获取图像的例子，从表12可看出，计算设备一共获得了4张图像，其中图3a所示的图像是从接口A传输进来的，图3b所示的图像是从接口B传输进来的，图3c所示的图像是从接口C传输进来的，图3d所示的图像是从接口D传输进来的。It should be noted that since the cables of each image sensor and each interface are "blindly inserted", the computing device can determine the source interface of a specific image, and does not know the pair of images transmitted from the source interface and the image collected. Correspondence between image sensors. For example, referring to Table 12, Table 12 shows an example in which a computing device obtains images from different interfaces. It can be seen from Table 12 that the computing device obtains a total of 4 images, of which the image shown in Figure 3a is obtained from Interface A is transmitted in, the image shown in Figure 3b is transmitted from interface B, the image shown in Figure 3c is transmitted from interface C, and the image shown in Figure 3d is transmitted from interface D.
表12Table 12
计算设备获取的图像Image acquired by computing device 获取图像的对应接口Get the corresponding interface of the image
图3a所示的图像The image shown in Figure 3a AA
图3b所示的图像The image shown in Figure 3b BB
图3c所示的图像The image shown in Figure 3c CC
图3d所示的图像The image shown in Figure 3d DD
计算设备在获得不同方位的图像传感器发送的图像之后，通过用户交互界面显示出来。例如，用户交互界面可以是触摸屏，可以用于显示图像，其中，用户交互界面上可以在一个页面上同时显示不同方位的图像传感器采集的图像，也可 以不同时显示；用户交互界面也可以用于接收用户的点击、滑动等操作，用户通过滑动、点击等操作，可以看到不同方位的图像传感器采集的图像。After the computing device obtains the images sent by the image sensors in different orientations, the images are displayed through the user interaction interface. For example, the user interaction interface can be a touch screen, which can be used to display images, wherein the user interaction interface can simultaneously display images collected by image sensors in different orientations on one page, or not simultaneously; the user interaction interface can also be used for Receive the user's click, slide and other operations, the user can see the images collected by the image sensor in different orientations through the slide, click and other operations.
S402、响应于用户的操作，确定并保存多个图像传感器与多个接口之间的对应关系。S402. Determine and save the correspondence between the multiple image sensors and the multiple interfaces in response to the user's operation.
用户的操作用于将多个图像与多个图像传感器进行对应，计算设备再根据上述步骤S401中多个图像与多个接口之间的对应关系，从而确定出多个图像传感器与多个接口之间的对应连接关系。其中，一个图像传感器与一个接口相连接，用户的操作可以是滑动、点击等操作。需要说明的是，用户是通过查看位于不同图像传感器前方的不同标定物或不同图像传感器视野范围内的不同场景，来将多个图像与多个图像传感器对应起来的。The user's operation is used to correspond the multiple images to the multiple image sensors, and the computing device determines the relationship between the multiple image sensors and the multiple interfaces according to the correspondence between the multiple images and the multiple interfaces in the above step S401. Corresponding connections between them. Among them, an image sensor is connected to an interface, and the user's operation may be a sliding, a click, or other operations. It should be noted that the user associates multiple images with multiple image sensors by viewing different calibration objects located in front of different image sensors or different scenes within the field of view of different image sensors.
在一种示例中，用户的操作可以是，用户按照预设顺序依次点击获得的图像，比如：参见表13所示，用户按照前图像传感器所采集的图像、后图像传感器所采集的图像、左图像传感器所采集的图像、右图像传感器所采集的图像的顺序，依次点击对应的图像(用户通过查看，知道前图像传感器采集的图像为图3a所示，后图像传感器采集的图像为图3b所示，左图像传感器采集的图像为图3c所示，右图像传感器采集的图像为图3d所示，因此依次点击图3a所示的图像、图3b所示的图像、图3c所示的图像、图3d所示的图像)。则计算设备接收到用户的操作，获得了多个图像传感器与多个图像之间的对应关系；再根据表12中获得的各个图像与各个接口之间的对应关系，从而确定出各个图像传感器与各个接口之间的关系，参见表14所示。需要说明的是，本示例中预设顺序是计算设备中预先设定好的，用户按照预设顺序点击对应图像即可。In one example, the user's operation may be that the user clicks the obtained images in a preset order. For example, as shown in Table 13, the user selects the images collected by the front The order of the images collected by the image sensor and the images collected by the right image sensor, click on the corresponding images in turn (the user knows that the image collected by the front image sensor is shown in Figure 3a, and the image collected by the rear image sensor is shown in Figure 3b. The image captured by the left image sensor is shown in Figure 3c, and the image captured by the right image sensor is shown in Figure 3d, so click the image shown in Figure 3a, the image shown in Figure 3b, the image shown in Figure 3c, image shown in Figure 3d). Then the computing device receives the user's operation and obtains the correspondence between the multiple image sensors and the multiple images; and then according to the correspondence between the respective images and the respective interfaces obtained in Table 12, to determine the relationship between the respective image sensors and the multiple images. For the relationship between each interface, see Table 14. It should be noted that the preset sequence in this example is preset in the computing device, and the user may click on the corresponding images according to the preset sequence.
表13Table 13
Figure PCTCN2021083466-appb-000006
Figure PCTCN2021083466-appb-000006
表14Table 14
各方位的图像传感器Image sensors in all directions 计算设备的各接口Interfaces of computing devices
前图像传感器front image sensor AA
后图像传感器rear image sensor BB
左图像传感器left image sensor CC
右图像传感器right image sensor DD
可以看到，计算设备利用自身的用户交互界面将各个图像传感器采集的图像显示出来，由用户在用户交互界面上进行操作，计算设备响应于用户的操作，确定出各个图像传感器与各个接口的对应关系，本申请实施例操作简单。It can be seen that the computing device uses its own user interface to display the images collected by each image sensor, and the user operates on the user interface, and the computing device determines the correspondence between each image sensor and each interface in response to the user's operation. relationship, the operation of this embodiment of the present application is simple.
本申请实施例还提供了一种图像传感器的标定方法，参见图7所示，图7为一种图像传感器的标定方法的流程示意图，该方法包括但不限于以下内容的描述。An embodiment of the present application further provides a method for calibrating an image sensor. Referring to FIG. 7 , FIG. 7 is a schematic flowchart of a method for calibrating an image sensor. The method includes but is not limited to the following descriptions.
S501、计算设备通过多个接口获取多个图像传感器采集的多个图像。S501. The computing device acquires multiple images collected by multiple image sensors through multiple interfaces.
本步骤可参考步骤S401中关于“通过多个接口获取多个图像传感器采集的多个图像”内容的描述，为了说明书的简洁，在此不再赘述。For this step, reference may be made to the description of the content of "acquiring multiple images collected by multiple image sensors through multiple interfaces" in step S401 , which is not repeated here for the sake of brevity of the description.
S502、计算设备将多个图像发送至显示系统，相应的，显示系统接收多个图像。S502. The computing device sends the multiple images to the display system, and accordingly, the display system receives the multiple images.
显示系统可以用于显示图像及相关内容等，还可以用于接收用户的操作，例如，滑动操作、点击操作。显示系统可以为车辆上的能够实现显示图像功能且能接收用户操作的系统，也可以为车辆外接的显示系统。The display system can be used to display images and related content, etc., and can also be used to receive user operations, such as sliding operations, click operations. The display system can be a system on the vehicle that can realize the function of displaying images and can receive user operations, or can be a display system external to the vehicle.
计算设备将获得的多个图像发送至显示系统，相应的，显示系统接收到计算设备发送的多个图像，并将多个图像显示出来。其中，显示系统可以在一个页面上同时显示不同方位的图像传感器采集的图像，也可以不同时显示。用户通过点击、滑动等操作，可以看到不同方位的图像传感器采集的图像。The computing device sends the obtained multiple images to the display system, and accordingly, the display system receives the multiple images sent by the computing device and displays the multiple images. Among them, the display system can display images collected by image sensors in different orientations on one page at the same time, or can not display them at the same time. By clicking, sliding and other operations, the user can see the images collected by the image sensors in different orientations.
S503、显示系统接收用户的操作，获得多个图像与多个图像传感器之间的对应关系。S503. The display system receives the user's operation, and obtains the correspondence between the multiple images and the multiple image sensors.
用户的操作为用户的点击或滑动操作。用户通过查看位于不同图像传感器前方的不同标定物或不同图像传感器视野范围内的不同场景，将多个图像与多个图像传感器进行对应。The user's operation is the user's click or slide operation. The user associates multiple images with multiple image sensors by viewing different calibration objects in front of different image sensors or different scenes within the field of view of different image sensors.
在一种示例中，用户的操作可以是，用户按照预设顺序依次点击获得的图像，比如：上述表13所示，用户按照前图像传感器所采集的图像、后图像传感器所采集的图像、左图像传感器所采集的图像、右图像传感器所采集的图像的顺序，依次点击对应的图像(用户通过查看，知道前图像传感器采集的图像为图3a所示，后图像传感器采集的图像为图3b所示，左图像传感器采集的图像为图3c所示，右图像传感器采集的图像为图3d所示，因此依次点击图3a所示的图像、图3b所示的图像、图3c所示的图像、图3d所示的图像)。因此，显示系统获得多个图像与多个图像传感器之间的对应关系。需要说明的是，本示例中预设顺序是显示系统中预先设定好的，用户按照预设顺序点击对应图像即可。In one example, the user's operation may be that the user clicks the obtained images in a preset order. The order of the images collected by the image sensor and the images collected by the right image sensor, click on the corresponding images in turn (the user knows that the image collected by the front image sensor is shown in Figure 3a, and the image collected by the rear image sensor is shown in Figure 3b. The image captured by the left image sensor is shown in Figure 3c, and the image captured by the right image sensor is shown in Figure 3d, so click the image shown in Figure 3a, the image shown in Figure 3b, the image shown in Figure 3c, image shown in Figure 3d). Therefore, the display system obtains the correspondence between the plurality of images and the plurality of image sensors. It should be noted that, in this example, the preset sequence is preset in the display system, and the user can click the corresponding images according to the preset sequence.
S504、显示系统将多个图像与多个图像传感器之间的对应关系发送至计算设备，相应的，计算设备接收多个图像与多个图像传感器之间的对应关系。S504. The display system sends the correspondence between the multiple images and the multiple image sensors to the computing device, and accordingly, the computing device receives the correspondence between the multiple images and the multiple image sensors.
S505、计算设备确定出多个图像传感器与多个接口之间的对应连接关系。S505. The computing device determines the corresponding connection relationship between the multiple image sensors and the multiple interfaces.
计算设备获得多个图像与多个图像传感器之间的对应关系后，再根据步骤S501中各个图像与各个接口之间的对应关系(例如表12所示的对应关系)，确定出多个图像传感器与多个接口之间的对应连接关系(例如表14所示的对应关系)。After obtaining the correspondence between the multiple images and the multiple image sensors, the computing device determines the multiple image sensors according to the correspondence between each image and each interface in step S501 (for example, the correspondence shown in Table 12). Corresponding connection relationship with multiple interfaces (for example, the corresponding relationship shown in Table 14).
需要说明的是，本申请上述任意一个实施例所描述的方法，不仅应用于车辆，例如汽车，还可以用于其他的将多个图像传感器与多个接口相对应的应用场景中。It should be noted that the method described in any one of the above embodiments of the present application is not only applied to vehicles, such as automobiles, but also can be applied to other application scenarios in which multiple image sensors are corresponding to multiple interfaces.
可以看到，计算设备还可以利用外界的显示系统，将各个图像传感器采集的图像显示出来，由用户在显示系统上进行操作，显示系统响应于用户的操作，确定出各个图像传感器与各个图像之间的对应关系，并将该对应关系发送至计算设备，从而计算设备确定出各个图像传感器与各个接口之间的对应关系。本方案，计算设备只需具备发送、接收和存储功能即可。It can be seen that the computing device can also use the external display system to display the images collected by each image sensor, and the user operates on the display system, and the display system determines the relationship between each image sensor and each image in response to the user's operation. The corresponding relationship between each image sensor and each interface is determined, and the corresponding relationship is sent to the computing device, so that the computing device determines the corresponding relationship between each image sensor and each interface. In this solution, the computing device only needs to have the functions of sending, receiving and storing.
参见图8，图8为本申请实施例提供的一种图像传感器的标定装置800的结构示意图，该装置800包括：获取单元801，用于从多个接口获取多个图像传感器采集的图像，具体的可用于执行图1实施例中的步骤S101，还可以用于执行图4实施例中的步骤S201；识别单元802，用于对获取得到的图像进行识别，具体的可用于执行图1实施例中的步骤S102，还可以用于执行图4实施例中的步骤S202；确定单元803，用于确定多个图像传感器与多个接口之间的对应关系，具体的可用于执行图1实施例中的步骤S103，还可以用于执行图4实施例中的步骤S203。Referring to FIG. 8, FIG. 8 is a schematic structural diagram of an image sensor calibration device 800 provided by an embodiment of the present application. The device 800 includes: an acquisition unit 801, configured to acquire images collected by multiple image sensors from multiple interfaces, specifically can be used to perform step S101 in the embodiment of FIG. 1, and can also be used to perform step S201 in the embodiment of FIG. 4; the identification unit 802 is used to identify the obtained image, and specifically can be used to perform the embodiment of FIG. 1 The step S102 in the FIG. 4 can also be used to execute the step S202 in the embodiment of FIG. 4 ; the determining unit 803 is used to determine the corresponding relationship between the multiple image sensors and the multiple interfaces, which can be specifically used to execute the step S202 in the embodiment of FIG. 1 . The step S103 of FIG. 4 may also be used to execute the step S203 in the embodiment of FIG. 4 .
图像传感器的标定装置800对应于方法实施例中的计算设备，图像传感器的标定装置800中的各单元用于实现图1或图4方法实施例中的计算设备所实施的各种步骤和方法，具体细节可参见上述方法的描述，为了说明书的简洁，在此不再赘述。The image sensor calibration device 800 corresponds to the computing device in the method embodiment, and each unit in the image sensor calibration device 800 is used to implement various steps and methods implemented by the computing device in the method embodiment of FIG. 1 or FIG. 4 , For details, refer to the description of the above method, which is not repeated here for the sake of brevity of the description.
图像传感器的标定装置800在进行图像传感器的标定时，仅以上述各功能单元的划分进行举例说明，实际应用中，可以根据需要，将上述功能分配给不同的功能单元完成，即可以将图像传感器的标定装置800的内部结构划分成不同的功能单元，以实现方法实施例中描述的全部功能或者部分功能。When the image sensor calibration device 800 calibrates the image sensor, only the division of the above functional units is used as an example. The internal structure of the calibration device 800 is divided into different functional units to implement all or part of the functions described in the method embodiments.
参见图9，图9为本申请实施例提供的又一种图像传感器的标定装置900的结构示意图，该装置900包括：获取单元901，可用于从多个接口获取多个图像传感器采集的图像，比如可用于执行图6实施例中的步骤S401中的获取图像的动作，或者用于执行图7实施例中的步骤S501；获取单元901还可用于从多个接口获取图像传感器的标识信息，比如可用于执行图5实施例中的步骤S301；确定单元902，用于确定多个图像传感器与多个接口之间的对应关系，比如可用 于执行图5实施例中的步骤S302，还可用于执行图6实施例中的步骤S402中确定并保存多个图像传感器与多个接口之间的对应关系的动作，还可以用于执行图7实施例中的步骤S505。Referring to FIG. 9, FIG. 9 is a schematic structural diagram of another image sensor calibration device 900 provided by an embodiment of the present application. The device 900 includes: an acquisition unit 901, which can be used to acquire images collected by multiple image sensors from multiple interfaces, For example, it can be used to perform the action of acquiring an image in step S401 in the embodiment of FIG. 6 , or to perform step S501 in the embodiment of FIG. 7 ; the acquiring unit 901 can also be used to acquire the identification information of the image sensor from multiple interfaces, such as It can be used to perform step S301 in the embodiment of FIG. 5; the determining unit 902 is used to determine the correspondence between multiple image sensors and multiple interfaces, for example, it can be used to perform step S302 in the embodiment of FIG. 5, and it can also be used to perform The action of determining and saving the correspondence between multiple image sensors and multiple interfaces in step S402 in the embodiment of FIG. 6 may also be used to execute step S505 in the embodiment of FIG. 7 .
可选的，装置900还可以包括交互单元，比如，交互单元可用于执行图6实施例中的步骤S401中将多个图像在用户交互界面中显示出来，交互单元还可用于执行图6实施例中步骤S402中响应于用户的操作。Optionally, the apparatus 900 may further include an interaction unit. For example, the interaction unit may be used to perform step S401 in the embodiment of FIG. 6 to display multiple images in the user interaction interface, and the interaction unit may also be used to perform the embodiment of FIG. 6 . In step S402, the operation of the user is responded to.
可选的，装置900还可以包括通信单元，比如，通信单元可用于接收外部装置或外部单元所发送的信息，或者向外部装置或外部单元发送信息，具体的可用于执行图7实施例中的步骤S502和步骤S504。Optionally, the device 900 may further include a communication unit, for example, the communication unit may be used to receive information sent by an external device or an external unit, or send information to an external device or an external unit, and specifically may be used to perform the implementation of the embodiment in FIG. 7 . Step S502 and Step S504.
图像传感器的标定装置900对应于方法实施例中的计算设备，图像传感器的标定装置900中的各单元用于实现图5或图6或图7方法实施例中的计算设备所实施的各种步骤和方法，具体细节可参见上述方法的描述，为了说明书的简洁，在此不再赘述。The image sensor calibration device 900 corresponds to the computing device in the method embodiment, and each unit in the image sensor calibration device 900 is used to implement various steps implemented by the computing device in the method embodiment of FIG. 5 or FIG. 6 or FIG. 7 . and method, for details, please refer to the description of the above method, which is not repeated here for the sake of brevity of the description.
图像传感器的标定装置900在进行图像传感器的标定时，仅以上述各功能单元的划分进行举例说明，实际应用中，可以根据需要，将上述功能分配给不同的功能单元完成，即可以将图像传感器的标定装置900的内部结构划分成不同的功能单元，以实现方法实施例中描述的全部功能或者部分功能。When the image sensor calibration device 900 calibrates the image sensor, only the division of the above functional units is used as an example. The internal structure of the calibration device 900 is divided into different functional units to implement all or part of the functions described in the method embodiments.
参见图10，图10为本申请实施例提供的一种计算设备1000的结构示意图，计算设备1000包括：处理器1010、通信接口1020以及存储器1030。其中，处理器1010、通信接口1020以及存储器1030可以通过内部总线1040相互连接，也可通过无线传输等其他手段实现通信。Referring to FIG. 10 , FIG. 10 is a schematic structural diagram of a computing device 1000 according to an embodiment of the present application. The computing device 1000 includes a processor 1010 , a communication interface 1020 , and a memory 1030 . The processor 1010, the communication interface 1020 and the memory 1030 can be connected to each other through the internal bus 1040, and can also communicate through other means such as wireless transmission.
以通过总线1040连接为例，总线1040可以是PCI总线或EISA总线等。总线1040可以分为地址总线、数据总线、控制总线等。为便于表示，图10中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。Taking the connection through the bus 1040 as an example, the bus 1040 may be a PCI bus or an EISA bus or the like. The bus 1040 can be divided into an address bus, a data bus, a control bus, and the like. For ease of presentation, only one thick line is used in FIG. 10, but it does not mean that there is only one bus or one type of bus.
处理器1010可以由至少一个通用处理器构成，例如CPU，或者CPU和硬件芯片的组合。上述硬件芯片可以是ASIC、PLD或其组合。上述PLD可以是CPLD、FPGA、GAL或其任意组合。处理器1010执行各种类型的数字存储指令，例如存储在存储器1030中的软件或者固件程序，它能使计算设备1000提供较宽的多种服务。The processor 1010 may be constituted by at least one general-purpose processor, such as a CPU, or a combination of a CPU and a hardware chip. The above-mentioned hardware chip may be an ASIC, a PLD or a combination thereof. The above PLD can be CPLD, FPGA, GAL or any combination thereof. Processor 1010 executes various types of digitally stored instructions, such as software or firmware programs stored in memory 1030, that enable computing device 1000 to provide a wide variety of services.
存储器1030用于存储程序代码，并由处理器1010来控制执行，以执行上述图1或图4或图5或图6或图7实施例所述的步骤，具体可以参考上述所示实施例的相关描述，此处不再展开赘述。The memory 1030 is used to store program codes, and is controlled and executed by the processor 1010 to execute the steps described in the above-mentioned embodiments of FIG. 1 or FIG. 4 or FIG. 5 or FIG. 6 or FIG. Related descriptions are not repeated here.
存储器1030可以包括易失性存储器，例如RAM；存储器1030也可以包括非易失性存储器，例如ROM、快闪存储器(Flash Memory)；存储器1030还可以包括上述种类的组合。The memory 1030 may include volatile memory, such as RAM; the memory 1030 may also include non-volatile memory, such as ROM, flash memory (Flash Memory); the memory 1030 may also include a combination of the above types.
通信接口1020可以为有线接口(例如以太网接口)，可以为内部接口(例如高速串行计算机扩展总线(Peripheral Component Interconnect express,PCIE)总 线接口)、有线接口(例如以太网接口)或无线接口(例如蜂窝网络接口或使用无线局域网接口)，用于与与其他设备或模块进行通信。The communication interface 1020 can be a wired interface (such as an Ethernet interface), an internal interface (such as a high-speed serial computer expansion bus (Peripheral Component Interconnect express, PCIE) bus interface), a wired interface (such as an Ethernet interface), or a wireless interface ( such as a cellular network interface or using a wireless local area network interface) to communicate with other devices or modules.
可选的，计算设备1000还可以包括用户交互界面，用户交互界面可用于显示图像，还可以用于接收或响应用户的点击、滑动等操作。Optionally, the computing device 1000 may further include a user interaction interface, and the user interaction interface may be used to display images, and may also be used to receive or respond to operations such as clicking, sliding, and the like by the user.
需要说明的，图10仅仅是本申请实施例的一种可能的实现方式，实际应用中，计算设备还可以包括更多或更少的部件，这里不作限制。关于本申请实施例中未示出或未描述的内容，可参见前述方法所述实施例中的相关阐述，这里不再赘述。It should be noted that FIG. 10 is only a possible implementation manner of the embodiment of the present application. In practical applications, the computing device may further include more or less components, which is not limited here. For content not shown or described in the embodiments of the present application, reference may be made to the relevant descriptions in the embodiments of the foregoing methods, and details are not repeated here.
本申请实施例提供了一种计算机程序产品，当该计算机程序产品在计算设备上运行时，使得计算设备执行本申请上述图1或图4或图5或图6或图7所述的方法实施例。An embodiment of the present application provides a computer program product, which, when the computer program product runs on a computing device, enables the computing device to execute the method described in FIG. 1 or FIG. 4 or FIG. 5 or FIG. 6 or FIG. 7 in the present application. example.
本申请实施例提供了一种计算机可读存储介质，包括程序指令，当计算机运行程序指令时，计算机执行上述图1或图4或图5或图6或图7所述的方法实施例。An embodiment of the present application provides a computer-readable storage medium, including program instructions. When the computer runs the program instructions, the computer executes the method embodiments described in FIG. 1 or FIG. 4 or FIG. 5 or FIG. 6 or FIG. 7 .
本领域普通技术人员可以意识到，结合本文中所公开的实施例中描述的各方法步骤和单元，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各实施例的步骤及组成。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。本领域普通技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。Those of ordinary skill in the art can realize that, in combination with the method steps and units described in the embodiments disclosed herein, they can be implemented in electronic hardware, computer software, or a combination of the two. Interchangeability, the steps and components of the various embodiments have been generally described in terms of functions in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Persons of ordinary skill in the art may use different methods of implementing the described functionality for each particular application, but such implementations should not be considered beyond the scope of this application.
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参见前述方法实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the above-described systems, devices and units, reference may be made to the corresponding processes in the foregoing method embodiments, which are not repeated here.
在本申请所提供的几个实施例中，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，该单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另外，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口、装置或单元的间接耦合或通信连接，也可以是电的，机械的或其它的形式连接。In the several embodiments provided in this application, the disclosed systems, devices and methods may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be other division methods, for example, multiple units or components may be combined or Integration into another system, or some features can be ignored, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may also be electrical, mechanical or other forms of connection.
该作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本申请实施例方案的目的。The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solutions of the embodiments of the present application.
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以是两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.
该集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分，或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例中方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read-only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application are essentially or part of contributions to the prior art, or all or part of the technical solutions can be embodied in the form of software products, and the computer software products are stored in a storage medium , including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program codes .
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。该计算机程序产品包括一个或多个计算机程序指令。在计算机上加载和执行该计算机程序指令时，全部或部分地产生按照本申请实施例中的流程或功能。该计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。该计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，该计算机程序指令可以从一个网站站点、计算机、服务器或数据中心通过有线或无线方式向另一个网站站点、计算机、服务器或数据中心进行传输。该计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。该可用介质可以是磁性介质(例如软盘、硬盘、磁带)、光介质(例如，数字视频光盘(digital video disc，DVD)、或者半导体介质(例如固态硬盘)等。In the above-mentioned embodiments, it may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented in software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer program instructions. When the computer program instructions are loaded and executed on a computer, the procedures or functions according to the embodiments of the present application are generated in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable device. The computer instructions may be stored in or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer program instructions may be transmitted from a website site, computer, server or data center via Wired or wireless transmission to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes one or more available media integrated. The available media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, digital video discs (DVDs), or semiconductor media (eg, solid state drives), and the like.
以上描述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到各种等效的修改或替换，这些修改或替换都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以权利要求的保护范围为准。The above descriptions are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications within the technical scope disclosed in the present application. or replacement, these modifications or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (23)

  1. 一种图像传感器的标定方法，其特征在于，包括：A method for calibrating an image sensor, comprising:
    通过多个接口获取多个图像传感器采集的多组标定图像，其中，所述多个接口、所述多个图像传感器以及所述多组标定图像之间分别存在一一对应关系；Obtaining multiple sets of calibration images collected by multiple image sensors through multiple interfaces, wherein there is a one-to-one correspondence between the multiple interfaces, the multiple image sensors, and the multiple sets of calibration images;
    识别所述多组标定图像所包含的多个标定物，获得所述多个标定物与所述多个接口之间的对应关系；其中，所述多个标定物和所述多组标定图像之间存在一一对应关系；Identify multiple calibration objects included in the multiple sets of calibration images, and obtain the corresponding relationship between the multiple calibration objects and the multiple interfaces; wherein, the multiple calibration objects and the multiple sets of calibration images There is a one-to-one correspondence between them;
    根据所述多个标定物与所述多个接口之间的对应关系以及预设信息，确定所述多个图像传感器与所述多个接口的对应关系；所述预设信息中包括所述多个标定物与所述多个图像传感器的对应关系。Determine the correspondence between the multiple image sensors and the multiple interfaces according to the correspondence between the multiple calibration objects and the multiple interfaces and preset information; the preset information includes the multiple interfaces. The corresponding relationship between the calibration objects and the plurality of image sensors.
  2. 根据权利要求1所述的方法，其特征在于，所述多个图像传感器分别布署在车辆的不同方位上。The method according to claim 1, wherein the plurality of image sensors are respectively deployed in different directions of the vehicle.
  3. 根据权利要求1或2所述的方法，其特征在于，所述多个标定物满足条件(i)-(iii)中的一项或多项：The method according to claim 1 or 2, wherein the plurality of calibration objects satisfy one or more of conditions (i)-(iii):
    (i)所述多个标定物中至少两个标定物的颜色互不相同；(i) the colors of at least two calibration objects in the plurality of calibration objects are different from each other;
    (ii)所述多个标定物中至少两个标定物的形状互不相同；(ii) the shapes of at least two calibration objects in the plurality of calibration objects are different from each other;
    (iii)所述多个标定物中至少两个标定物的图案互不相同。(iii) The patterns of at least two calibration objects in the plurality of calibration objects are different from each other.
  4. 根据权利要求1-3任一项所述的方法，其特征在于，在所述确定所述多个图像传感器与所述多个接口的对应关系之后，所述方法还包括：The method according to any one of claims 1-3, wherein after the determining the correspondence between the plurality of image sensors and the plurality of interfaces, the method further comprises:
    获取车辆上的所述多个图像传感器采集的环境图像；acquiring environmental images collected by the plurality of image sensors on the vehicle;
    根据所述多个图像传感器与所述多个接口的对应关系，对所述环境图像进行处理，获得处理结果；According to the correspondence between the plurality of image sensors and the plurality of interfaces, process the environment image to obtain a processing result;
    根据所述处理结果控制所述车辆。The vehicle is controlled according to the processing result.
  5. 一种图像传感器的标定方法，其特征在于，包括：A method for calibrating an image sensor, comprising:
    在多个时刻，通过多个接口分别获取多个图像传感器采集的图像；其中，所述多个时刻中的每个时刻获取的所述图像中，至少一个图像包含标定物特征；所述多个接口与所述多个图像传感器之间存在一一对应关系；所述标定物特征包括标定物的颜色、图案、形状中的一项或多项；At multiple times, images collected by multiple image sensors are respectively acquired through multiple interfaces; wherein, among the images acquired at each of the multiple times, at least one image includes a feature of a calibration object; the multiple There is a one-to-one correspondence between the interface and the plurality of image sensors; the characteristics of the calibration object include one or more of the color, pattern, and shape of the calibration object;
    识别所述每个时刻获取的图像中包含所述标定物特征的图像，从而确定不同时刻所述标定物特征或包含所述标定物特征的图像与所述多个接口之间的获取关系；Identifying the images containing the features of the calibration object in the images obtained at each moment, so as to determine the acquisition relationship between the features of the calibration object or the images containing the features of the calibration objects at different times and the multiple interfaces;
    根据所述获取关系及预设信息，确定所述多个图像传感器与所述多个接口的对应关系；所述预设信息包括所述标定物特征出现的时序顺序。According to the acquired relationship and the preset information, the corresponding relationship between the multiple image sensors and the multiple interfaces is determined; the preset information includes the time sequence in which the features of the calibration object appear.
  6. 根据权利要求1-5任一项所述的方法，其特征在于，所述标定物包括：The method according to any one of claims 1-5, wherein the calibration object comprises:
    车载设备；Car Equipment;
    或者，由车载设备产生的图案、颜色、形状中的至少一项。Or, at least one of a pattern, color, and shape produced by the in-vehicle device.
  7. 一种图像传感器的标定方法，其特征在于，包括：A method for calibrating an image sensor, comprising:
    通过多个接口分别获取多个标识信息；其中，所述多个接口与所述多个标识信息之间存在一一对应关系；所述标识信息中包括图像传感器的参数；Obtain a plurality of identification information respectively through a plurality of interfaces; wherein, there is a one-to-one correspondence between the plurality of interfaces and the plurality of identification information; the identification information includes the parameters of the image sensor;
    根据所述多个标识信息和预设信息，确定出多个所述图像传感器与所述多个接口之间的对应关系；其中，所述预设信息中包括所述多个图像传感器与所述多个标识信息之间的对应关系。According to the plurality of identification information and preset information, the corresponding relationship between the plurality of image sensors and the plurality of interfaces is determined; wherein, the preset information includes the plurality of image sensors and the plurality of interfaces. Correspondence between multiple pieces of identification information.
  8. 根据权利要求7所述的方法，其特征在于，所述图像传感器的参数包括品牌、采集图像的频率、采集图像的清晰度中的至少一项。The method according to claim 7, wherein the parameters of the image sensor include at least one of a brand, a frequency of capturing images, and a definition of capturing images.
  9. 一种图像传感器的标定方法，其特征在于，应用于计算设备，所述计算设备包括用户交互界面，所述方法包括：A method for calibrating an image sensor, characterized in that it is applied to a computing device, wherein the computing device includes a user interface, and the method includes:
    通过多个接口获取多个图像传感器采集的多个图像，并将所述多个图像在所述用户交互界面中显示出来；其中，所述多个接口、所述多个图像传感器以及所述多个图像之间分别存在一一对应关系；Acquire multiple images collected by multiple image sensors through multiple interfaces, and display the multiple images in the user interaction interface; wherein, the multiple interfaces, the multiple image sensors, and the multiple image sensors There is a one-to-one correspondence between the images;
    响应于用户的操作，确定并保存所述多个图像传感器与所述多个接口之间的对应关系；其中，所述用户的操作用于将所述多个图像与所述多个图像传感器进行对应。In response to a user's operation, the correspondence between the plurality of image sensors and the plurality of interfaces is determined and saved; wherein the user's operation is used to perform a comparison between the plurality of images and the plurality of image sensors correspond.
  10. 一种图像传感器的标定方法，其特征在于，所述方法包括：A method for calibrating an image sensor, wherein the method comprises:
    计算设备通过多个接口获取多个图像传感器采集的多个图像，并将所述多个图像发送至显示系统；其中，所述多个接口、所述多个图像传感器以及所述多个图像之间分别存在一一对应关系；The computing device acquires multiple images collected by multiple image sensors through multiple interfaces, and sends the multiple images to the display system; wherein, the multiple interfaces, the multiple image sensors, and the multiple images are among the multiple images. There is a one-to-one correspondence between them;
    所述显示系统将所述多个图像显示出来；the display system displays the plurality of images;
    所述显示系统接收用户的操作，获得所述多个图像与所述多个图像传感器之间的对应关系；The display system receives the user's operation, and obtains the correspondence between the plurality of images and the plurality of image sensors;
    将所述多个图像与所述多个图像传感器之间的对应关系发送至所述计算设备；sending the correspondence between the plurality of images and the plurality of image sensors to the computing device;
    所述计算设备确定所述多个图像传感器与所述多个接口之间的对应关系。The computing device determines a correspondence between the plurality of image sensors and the plurality of interfaces.
  11. 一种图像传感器的标定装置，其特征在于，包括：A device for calibrating an image sensor, comprising:
    获取单元，用于通过多个接口获取多个图像传感器采集的多组标定图像，其中，所述多个接口、所述多个图像传感器以及所述多组标定图像之间分别存在一一对应关系；an acquisition unit, configured to acquire multiple sets of calibration images collected by multiple image sensors through multiple interfaces, wherein there is a one-to-one correspondence between the multiple interfaces, the multiple image sensors, and the multiple sets of calibration images respectively ;
    识别单元，用于识别所述多组标定图像所包含的多个标定物，获得所述多个标定物与所述多个接口之间的对应关系；其中，所述多个标定物和所述多组标定图像之间存在一一对应关系；an identification unit, configured to identify multiple calibration objects included in the multiple sets of calibration images, and obtain the correspondence between the multiple calibration objects and the multiple interfaces; wherein, the multiple calibration objects and the There is a one-to-one correspondence between multiple sets of calibration images;
    确定单元，用于根据所述多个标定物与所述多个接口之间的对应关系以及预设信息，确定所述多个图像传感器与所述多个接口的对应关系；所述预设信息中包括所述多个标定物与所述多个图像传感器的对应关系。a determining unit, configured to determine the corresponding relationship between the multiple image sensors and the multiple interfaces according to the corresponding relationship between the multiple calibration objects and the multiple interfaces and preset information; the preset information includes the correspondence between the plurality of calibration objects and the plurality of image sensors.
  12. 根据权利要求11所述的装置，其特征在于，所述多个图像传感器分别布署在车辆的不同方位上。The device according to claim 11, wherein the plurality of image sensors are respectively arranged in different directions of the vehicle.
  13. 根据权利要求11或12所述的装置，其特征在于，所述多个标定物满足条件(i)-(iii)中的一项或多项：The device according to claim 11 or 12, wherein the plurality of calibration objects satisfy one or more of conditions (i)-(iii):
    (i)所述多个标定物中至少两个标定物的颜色互不相同；(i) the colors of at least two calibration objects in the plurality of calibration objects are different from each other;
    (ii)所述多个标定物中至少两个标定物的形状互不相同；(ii) the shapes of at least two calibration objects in the plurality of calibration objects are different from each other;
    (iii)所述多个标定物中至少两个标定物的图案互不相同。(iii) The patterns of at least two calibration objects in the plurality of calibration objects are different from each other.
  14. 根据权利要求11-13任一项所述的装置，其特征在于，所述获取单元还用于，获取车辆上的所述多个图像传感器采集的环境图像；The device according to any one of claims 11 to 13, wherein the acquiring unit is further configured to acquire environmental images collected by the plurality of image sensors on the vehicle;
    所述装置还包括：The device also includes:
    处理单元，用于根据所述多个图像传感器与所述多个接口的对应关系，对所述环境图像进行处理，获得处理结果；a processing unit, configured to process the environment image according to the corresponding relationship between the plurality of image sensors and the plurality of interfaces, and obtain a processing result;
    控制单元，用于根据所述处理结果控制所述车辆。a control unit for controlling the vehicle according to the processing result.
  15. 一种图像传感器的标定装置，其特征在于，包括：A device for calibrating an image sensor, comprising:
    获取单元，用于在多个时刻，通过多个接口分别获取多个图像传感器采集的图像；其中，所述多个时刻中的每个时刻获取的所述图像中，至少一个图像包含标定物特征；所述多个接口与所述多个图像传感器之间存在一一对应关系；所述标定物特征包括标定物的颜色、图案、形状中的一项或多项；an acquisition unit, configured to acquire images collected by multiple image sensors through multiple interfaces at multiple times; wherein, among the images acquired at each of the multiple times, at least one image contains features of the calibration object ; There is a one-to-one correspondence between the plurality of interfaces and the plurality of image sensors; the characteristics of the calibration object include one or more of the color, pattern, and shape of the calibration object;
    识别单元，用于识别所述每个时刻获取的图像中包含所述标定物特征的图像，从而确定不同时刻所述标定物特征或包含所述标定物特征的图像与所述多个接口之间的获取关系；an identification unit, configured to identify an image including the feature of the calibration object in the images obtained at each moment, so as to determine the relationship between the feature of the calibration object or the image including the feature of the calibration object at different times and the multiple interfaces acquisition relationship;
    确定单元，用于根据所述获取关系及预设信息，确定所述多个图像传感器与所述多个接口的对应关系；所述预设信息包括所述标定物特征出现的时序顺序。and a determining unit, configured to determine the corresponding relationship between the plurality of image sensors and the plurality of interfaces according to the acquired relationship and preset information; the preset information includes the time sequence in which the features of the calibration object appear.
  16. 根据权利要求11-15任一项所述的装置，其特征在于，所述标定物包括：The device according to any one of claims 11-15, wherein the calibration object comprises:
    车载设备；Car Equipment;
    或者，由车载设备产生的图案、颜色、形状中的至少一项。Or, at least one of a pattern, color, and shape produced by the in-vehicle device.
  17. 一种图像传感器的标定装置，其特征在于，包括：A device for calibrating an image sensor, comprising:
    获取单元，用于通过多个接口分别获取多个标识信息；其中，所述多个接口与所述多个标识信息之间存在一一对应关系；所述标识信息中包括图像传感器的参数；an obtaining unit, configured to obtain a plurality of identification information through a plurality of interfaces; wherein, there is a one-to-one correspondence between the plurality of interfaces and the plurality of identification information; the identification information includes the parameters of the image sensor;
    确定单元，用于根据所述多个标识信息和预设信息，确定出多个图像传感器与所述多个接口之间的对应关系；其中，所述预设信息中包括所述多个图像传感器与所述多个标识信息之间的对应关系。a determining unit, configured to determine the correspondence between the multiple image sensors and the multiple interfaces according to the multiple identification information and the preset information; wherein the preset information includes the multiple image sensors and the corresponding relationship between the plurality of identification information.
  18. 根据权利要求17所述的装置，其特征在于，所述每个图像传感器的参数包括品牌、采集图像的频率、采集图像的清晰度中的至少一项。The apparatus according to claim 17, wherein the parameters of each image sensor include at least one of a brand, a frequency of capturing images, and a definition of capturing images.
  19. 一种图像传感器的标定装置，其特征在于，所述装置包括用户交互界面，所述装置包括：A device for calibrating an image sensor, characterized in that the device includes a user interface, and the device includes:
    获取单元，用于通过多个接口获取多个图像传感器采集的多个图像；其中，所述多个接口、所述多个图像传感器以及所述多个图像之间分别存在一一对应关系；an acquisition unit, configured to acquire multiple images collected by multiple image sensors through multiple interfaces; wherein, there is a one-to-one correspondence between the multiple interfaces, the multiple image sensors, and the multiple images;
    交互单元，用于将所述多个图像在所述用户交互界面中显示出来；an interaction unit, configured to display the plurality of images in the user interaction interface;
    所述交互单元还用于，响应于用户的操作；The interaction unit is further configured to respond to the user's operation;
    确定单元，用于确定并保存所述多个图像传感器与所述多个接口之间的对应关系；其中，所述用户的操作用于将所述多个图像与所述多个图像传感器进行对应。a determining unit, configured to determine and save the correspondence between the plurality of image sensors and the plurality of interfaces; wherein, the user's operation is used to correspond the plurality of images to the plurality of image sensors .
  20. 一种图像传感器的标定装置，其特征在于，所述装置包括：A device for calibrating an image sensor, characterized in that the device comprises:
    获取单元，用于所述计算设备通过多个接口获取多个图像传感器采集的多个图像，并将所述多个图像发送至所述显示系统；其中，所述多个接口、所述多个图像传感器以及所述多个图像之间分别存在一一对应关系；an acquiring unit, used for the computing device to acquire multiple images collected by multiple image sensors through multiple interfaces, and send the multiple images to the display system; wherein the multiple interfaces, the multiple There is a one-to-one correspondence between the image sensor and the plurality of images;
    通信单元，用于接收所述多个图像与所述多个图像传感器之间的对应关系；a communication unit, configured to receive the correspondence between the plurality of images and the plurality of image sensors;
    确定单元，用于确定出所述多个图像传感器与所述多个接口之间的对应关系。A determination unit, configured to determine the correspondence between the plurality of image sensors and the plurality of interfaces.
  21. 一种计算设备，其特征在于，包括存储器和处理器，所述存储器用于存储指令，所述处理器用于调用所述存储器中存储的指令，实现如权利要求1-10任一项所述的方法。A computing device, characterized in that it includes a memory and a processor, the memory is used to store instructions, and the processor is used to call the instructions stored in the memory, so as to implement the method according to any one of claims 1-10. method.
  22. 一种存储介质，其特征在于，包括程序指令，当所述程序指令在计算机上运行时，实现如权利要求1-10任一项所述的方法。A storage medium is characterized by comprising program instructions, when the program instructions are executed on a computer, the method according to any one of claims 1-10 is implemented.
  23. 一种计算机程序产品，其特征在于，包括计算机程序代码，当所述计算机程序代码被计算机执行时，实现如权利要求1-10任一项所述的方法。A computer program product, characterized by comprising computer program code, which, when executed by a computer, implements the method according to any one of claims 1-10.
PCT/CN2021/083466 2021-03-27 2021-03-27 Image sensor calibration method and apparatus, device, and storage medium WO2022204840A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202180001476.5A CN113196339A (en) 2021-03-27 2021-03-27 Calibration method, device and equipment of image sensor and storage medium
PCT/CN2021/083466 WO2022204840A1 (en) 2021-03-27 2021-03-27 Image sensor calibration method and apparatus, device, and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/083466 WO2022204840A1 (en) 2021-03-27 2021-03-27 Image sensor calibration method and apparatus, device, and storage medium

Publications (1)

Publication Number Publication Date
WO2022204840A1 true WO2022204840A1 (en) 2022-10-06

Family

ID=76976986

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/083466 WO2022204840A1 (en) 2021-03-27 2021-03-27 Image sensor calibration method and apparatus, device, and storage medium

Country Status (2)

Country Link
CN (1) CN113196339A (en)
WO (1) WO2022204840A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105353265A (en) * 2015-12-29 2016-02-24 河南农业大学 Probe type detection method and device for multiple models of multi-core cables
CN109211298A (en) * 2017-07-04 2019-01-15 百度在线网络技术（北京）有限公司 A kind of transducer calibration method and device
CN110211187A (en) * 2019-04-28 2019-09-06 上海小萌科技有限公司 A kind of multi-cam position calibration method
US20190335074A1 (en) * 2018-04-27 2019-10-31 Cubic Corporation Eliminating effects of environmental conditions of images captured by an omnidirectional camera
US10789730B2 (en) * 2016-03-18 2020-09-29 Teknologian Tutkimuskeskus Vtt Oy Method and apparatus for monitoring a position

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11436756B2 (en) * 2018-12-20 2022-09-06 Microsoft Technology Licensing, Llc Calibrating a machine vision camera

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105353265A (en) * 2015-12-29 2016-02-24 河南农业大学 Probe type detection method and device for multiple models of multi-core cables
US10789730B2 (en) * 2016-03-18 2020-09-29 Teknologian Tutkimuskeskus Vtt Oy Method and apparatus for monitoring a position
CN109211298A (en) * 2017-07-04 2019-01-15 百度在线网络技术（北京）有限公司 A kind of transducer calibration method and device
US20190335074A1 (en) * 2018-04-27 2019-10-31 Cubic Corporation Eliminating effects of environmental conditions of images captured by an omnidirectional camera
CN110211187A (en) * 2019-04-28 2019-09-06 上海小萌科技有限公司 A kind of multi-cam position calibration method

Also Published As

Publication number Publication date
CN113196339A (en) 2021-07-30

Similar Documents

Publication Publication Date Title
US10726579B1 (en) LiDAR-camera calibration
US11070756B2 (en) Method, device and system for image acquisition control
CN111383279B (en) External parameter calibration method and device and electronic equipment
US10582121B2 (en) System and method for fusing outputs of sensors having different resolutions
WO2021098448A1 (en) Sensor calibration method and device, storage medium, calibration system, and program product
US11715180B1 (en) Emirror adaptable stitching
US10482626B2 (en) Around view monitoring systems for vehicle and calibration methods for calibrating image capture devices of an around view monitoring system using the same
CN110753167B (en) Time synchronization method, device, terminal equipment and storage medium
WO2021098439A1 (en) Sensor calibration method and apparatus, and storage medium, calibration system and program product
US10750093B2 (en) Image processing device and image processing method
CN112308927B (en) Fusion device of panoramic camera and laser radar and calibration method thereof
WO2021203868A1 (en) Data processing method and device
JP6967098B2 (en) Methods and equipment for calibrating the camera
CN111429521A (en) External parameter calibration method, device, medium and electronic equipment for camera and laser radar
CN113160328A (en) External reference calibration method, system, robot and storage medium
US11953602B2 (en) Detecting three-dimensional structure models at runtime in vehicles
WO2018156321A1 (en) Method, system and apparatus for visual effects
CN110673607B (en) Feature point extraction method and device under dynamic scene and terminal equipment
CN115082565A (en) Camera calibration method, device, server and medium
WO2022204840A1 (en) Image sensor calibration method and apparatus, device, and storage medium
CN111210386A (en) Image shooting and splicing method and system
US20220277468A1 (en) Calibration Parameter Obtaining Method and Apparatus, Processor, and Electronic Device
CN109618085B (en) Electronic equipment and mobile platform
WO2020237553A1 (en) Image processing method and system, and movable platform
CN107992789B (en) Method and device for identifying traffic light and vehicle

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21933527

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21933527

Country of ref document: EP

Kind code of ref document: A1