WO2024062540A1 - 画像処理装置 - Google Patents

画像処理装置 Download PDF

Info

Publication number
WO2024062540A1
WO2024062540A1 PCT/JP2022/035027 JP2022035027W WO2024062540A1 WO 2024062540 A1 WO2024062540 A1 WO 2024062540A1 JP 2022035027 W JP2022035027 W JP 2022035027W WO 2024062540 A1 WO2024062540 A1 WO 2024062540A1
Authority
WO
WIPO (PCT)
Prior art keywords
parallax
reliability
unit
image
image processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/035027
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
茂 松尾
健 遠藤
雅幸 竹村
雅士 高田
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Astemo Ltd
Original Assignee
Hitachi Astemo Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hitachi Astemo Ltd filed Critical Hitachi Astemo Ltd
Priority to JP2024547981A priority Critical patent/JP7769811B2/ja
Priority to EP22959504.6A priority patent/EP4592638A1/en
Priority to PCT/JP2022/035027 priority patent/WO2024062540A1/ja
Priority to CN202280099875.4A priority patent/CN119836559A/zh
Publication of WO2024062540A1 publication Critical patent/WO2024062540A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C3/00—Measuring distances in line of sight; Optical rangefinders
    • G01C3/02—Details
    • G01C3/06—Use of electric means to obtain final indication

Definitions

  • the present invention relates to an image processing device.
  • a stereo camera can measure the distance to an object by using the parallax of an overlapping region of images taken by two cameras arranged at a predetermined interval. Therefore, the stereo camera can accurately grasp the risk of collision with an object.
  • Patent Document 1 discloses that an image capturing means for capturing an image of a target object and a stereo matching process are performed for each small region of a pair of images captured by the image capturing means to identify corresponding regions by determining mutual correlation.
  • stereo processing means for acquiring an evaluation function that is a matching processing result, distance information to the target object obtained based on the evaluation function, and reliability information indicating the reliability thereof; setting a re-search range around the area, and based on the evaluation function calculated for the small area within the re-search range, correcting the evaluation function calculated for the target small area to obtain a modified evaluation function;
  • a stereo image processing device characterized by comprising a parallax correction means for correcting distance information based on the obtained modified evaluation function.
  • the device disclosed in Patent Document 1 corrects parallax by correcting the evaluation function determined for each matching block set in stereo matching. Therefore, in the device disclosed in Patent Document 1, depending on the situation of the captured image, even the corrected parallax may contain a large error, and there is room for improvement.
  • the present invention has been made in view of the above, and an object of the present invention is to provide an image processing device that can accurately calculate parallax.
  • an image processing device of the present invention includes a parallax calculation unit that calculates the parallax of images captured by a plurality of cameras, and a parallax calculation unit that classifies objects that appear in the images and types of the classified objects.
  • a labeling unit that assigns a label to each pixel of the object according to the label
  • a reliability calculation unit that calculates the reliability of the parallax for each label assigned
  • a reliability calculation unit that calculates the parallax according to the calculated reliability.
  • a parallax correction unit that corrects the parallax.
  • FIG. 1 is a diagram showing the configuration of a stereo camera device including an image processing device according to the present embodiment.
  • 5 is a flowchart showing the operation of the image processing device.
  • 5 is a flowchart showing processing performed by a reliability calculation unit.
  • FIG. 3 is a diagram illustrating an example of the configuration of data written to one pixel of a reliability image.
  • 5 is a flowchart showing processing performed by an interpolation unit.
  • FIG. 11 is a diagram for explaining step S34 shown in FIG. 10 .
  • 5 is a flowchart showing processing performed by a recalculation unit.
  • FIG. 1 is a diagram showing the configuration of a stereo camera device 1 including an image processing device 20 of this embodiment.
  • FIG. 2 is a diagram showing the parallax image 130.
  • FIG. 3 is a diagram showing the label image 140.
  • FIG. 4 is a diagram illustrating the occlusion portion 241.
  • FIG. 5 is a diagram showing the reliability table 51.
  • FIG. 6 is a diagram showing the reliability image 150.
  • the stereo camera device 1 is a type of sensor that is mounted on a vehicle and monitors the area around the vehicle.
  • the stereo camera device 1 detects objects around the vehicle and measures the distance to the objects by utilizing the parallax of the overlapping areas of images 111, 112 captured by multiple cameras 11, 12 arranged at a predetermined interval in the horizontal direction.
  • the stereo camera device 1 shown in FIG. 1 performs image processing using a left camera 11 and a right camera 12, a left image 111 taken by the left camera 11, and a right image 112 taken by the right camera 12.
  • a device 20 is provided.
  • the image processing device 20 is configured by, for example, a computer including a processor such as a CPU and a storage device such as a ROM and a RAM.
  • the image processing device 20 realizes various functions of the image processing device 20 by the CPU executing programs stored in the ROM.
  • the image processing device 20 includes a parallax calculation section 30, a labeling section 40, a reliability calculation section 50, a parallax correction section 60, and a recognition processing section 70.
  • the parallax calculation unit 30 calculates the parallax of the left image 111 and the right image 112 captured by the left camera 11 and the right camera 12 for each pixel.
  • the parallax calculation unit 30 of this embodiment has a neural network model.
  • a left image 111 and a right image 112 are set in the input layer of the neural network model.
  • the parallax between the left image 111 and the right image 112 is set in the output layer of the neural network model.
  • weights, biases, and activation functions for each neuron are set.
  • the neural network model is constructed by machine learning the weights and biases of each neuron using error backpropagation, etc., and is installed in the parallax calculation unit 30 in advance.
  • the machine learning may be supervised learning.
  • the parallax calculation unit 30 may have a model constructed by machine learning other than the neural network model.
  • the model included in the parallax calculation unit 30 may be, for example, an optical flow model.
  • the parallax calculation unit 30 generates a parallax image 130 in which the calculated parallax is stored in each pixel.
  • the parallax image 130 is generated as an image in which the parallax corresponding to each pixel of the right image 112 is stored in each pixel of the right image 112, as shown in FIG. 2, for example. In the parallax image 130 shown in FIG. 2, the darker the color, the larger the parallax and the closer the distance.
  • the labeling unit 40 classifies the subject appearing in the right image 112 and assigns a label according to the type of the classified subject to each pixel of the subject.
  • the labeling unit 40 can classify objects and assign labels using a neural network model.
  • the labeling unit 40 of this embodiment classifies objects and assigns labels using panoptic segmentation.
  • a processing overview of panoptic segmentation is introduced in the paper by Kirillov et al., “Panoptic Feature Pyramid Networks,” 2019 IEEE/CVF Conference on CVPR.
  • the labeling unit 40 can refer to the paper.
  • the labeling unit 40 labels the objects in the right image 112 as road surfaces 210, puddles 220, gutters 230, vehicles 240, vehicles 250, and zebra zones 260, according to the learned label information. Classify by pixel. Then, the labeling unit 40 writes the label number of the label assigned to each subject for each pixel of the subject. In this way, the labeling unit 40 generates a label image 140 in which a label number is stored in each pixel.
  • the labeling unit 40 performs the following processing in order to identify the occlusion portion 241. For example, as shown in FIG. 4, it is assumed that the right image 112 and the left image 111 have different degrees of overlap between the vehicles 240 and 250, and the occlusion portion 241 is not captured in the left image 111.
  • the labeling unit 40 compares the parallaxes of the vehicle 240 and the vehicle 250, and identifies the area on the right side of the vehicle 240, which is the subject with a large parallax, as the occlusion portion 241.
  • the width of the occlusion portion 241 is the number of pixels that is the difference between the parallax of the vehicle 240 and the parallax of the vehicle 250.
  • the labeling unit 40 can also verify the consistency between the type of the classified subject and the label assigned to the subject based on the parallax calculated by the parallax calculation unit 30. For example, if the subject is a signboard with a picture of a person attached to it, the parallax of the pixels that make up the subject indicates a value that is approximately constant or changes linearly in the pixel group, and the subject is recognized as being flat. On the other hand, if the subject is an actual person, the parallax of the pixels that make up the subject indicates a value that changes nonlinearly in the pixel group, and the subject is recognized as being three-dimensional. For this reason, the labeling unit 40 can verify the consistency between the type of the classified subject and the label assigned to the subject by using the parallax image 130 in addition to the right image 112.
  • the reliability calculation unit 50 calculates the reliability of parallax for each label given by the labeling unit 40.
  • the reliability is an index that quantifies the magnitude of the error included in the parallax value. The smaller the error included in the parallax value, the higher the reliability. In this embodiment, the reliability is “low” if the error included in the parallax value is 20% or more, the reliability is “medium” if the error is 10% or more and less than 20%, and the reliability is “medium” if the error is less than 10%. is "high”.
  • the reliability calculation unit 50 calculates the reliability in order to clarify the subject whose parallax is assumed to be inaccurate.
  • the reliability calculation unit 50 calculates the reliability of the disparity using a reliability table 51 created in advance.
  • the reliability table 51 is a table showing the correspondence between labels and reliability, as shown in FIG. 5.
  • a reliability item of "0” indicates that the reliability is "invalid”
  • a reliability item of "1” indicates that the reliability is "low”
  • a reliability item of "2” indicates that the reliability is "medium”
  • a reliability item of "3” indicates that the reliability is "high”.
  • the reliability calculation unit 50 refers to the reliability table 51 and identifies the reliability corresponding to the label assigned by the labeling unit 40. The reliability calculation unit 50 then writes the identified reliability for each pixel of the subject to which the label is assigned. In this way, the reliability calculation unit 50 generates a reliability image 150 in which a label number and a reliability are stored for each pixel, as shown in FIG. 6.
  • the parallax correction unit 60 corrects parallax according to the reliability calculated by the reliability calculation unit 50.
  • the parallax correction unit 60 includes an interpolation unit 61 and a recalculation unit 62.
  • the interpolation unit 61 interpolates the parallax whose reliability does not meet the standard based on the parallax calculated for the images 111 and 112 taken in the past.
  • the interpolation unit 61 can correct the parallax image 130 by interpolating parallaxes whose reliability does not meet the standard.
  • the interpolation unit 61 of the present embodiment determines whether or not to interpolate parallaxes whose reliability is "low” or "invalid” based on the parallaxes calculated for images 111 and 112 taken in the past, When it is determined that interpolation is to be performed, a parallax with a "high” reliability is used to interpolate a parallax with a "low” or "invalid" reliability.
  • an area where the reliability calculated for the images 111 and 112 taken currently is “low” or “invalid” is calculated as "high” for the images 111 and 112 taken in the past.
  • the interpolation unit 61 of the present embodiment determines to interpolate the parallax of which the reliability of the area is "low” or “invalid”, and uses the parallax of which the reliability is "high” adjacent to the area, Interpolates parallaxes whose reliability in the area is "low” or "invalid".
  • the recalculation unit 62 recalculates the parallax whose reliability does not meet the standard using a method different from that of the parallax calculation unit 30.
  • the recalculation unit 62 can correct the parallax image 130 by replacing the parallax whose reliability does not meet the standard with the recalculated parallax.
  • the recalculation unit 62 of this embodiment recalculates the parallax whose reliability is "medium” or "low” by, for example, stereo matching.
  • the recalculation unit 62 of this embodiment may use the parallax interpolated by the interpolation unit 61 as a target for recalculation. Note that when the parallax calculation unit 30 calculates the parallax by stereo matching, the recalculation unit 62 may recalculate the parallax using a neural network model.
  • the recognition processing unit 70 detects objects around the vehicle based on the parallax corrected by the parallax correction unit 60, the reliability calculated by the reliability calculation unit 50, and the images 111 and 112. Specifically, the recognition processing unit 70 uses the corrected parallax image 130, reliability image 150, left image 111, and right image 112 to detect objects around the vehicle.
  • a road surface area with "high” reliability has accurate parallax, and even small objects on the road surface can be detected.
  • the recognition processing unit 70 changes the size of the object to be detected or the distance to the object according to the reliability. For example, in an area where the reliability is "low”, the recognition processing unit 70 detects, among objects on the road, objects whose height is greater than the error included in the parallax of the road.
  • the recognition processing unit 70 detects a subject whose height is larger than the error included in the parallax of the road surface in the road surface area where the reliability is "low", and detects the object whose height is larger than the error included in the road surface parallax in the road surface area where the reliability is "high". , subjects whose height is smaller than the road surface area with "high” reliability are also detected. In this way, the recognition processing unit 70 can change the size of the object to be detected or the distance to the object depending on the reliability. As for the method of object detection itself, the recognition processing unit 70 can employ a known method.
  • the parallax image 130 and the reliability image 150 are input to the recognition processing unit 70 in a manner that corresponds to each other pixel by pixel. That is, parallax, reliability, and label information are input to the recognition processing unit 70 in association with each other for each pixel.
  • the recognition processing unit 70 can, for example, re-label a pixel whose reliability does not meet the standard after the fact or recalculate the reliability after the fact. That is, the recognition processing unit 70 can reconsider the data (reliability or label) acquired at the stage before object detection, and can improve the accuracy of object detection.
  • FIG. 7 is a flowchart showing the operation of the image processing device 20.
  • step S11 the parallax calculation unit 30 of the image processing device 20 generates a parallax image 130 from the left image 111 and the right image 112.
  • step S12 the labeling unit 40 of the image processing device 20 classifies the subject using the right image 112 and the parallax image 130, and generates a label image 140 by assigning a label to each pixel of the subject.
  • step S13 the reliability calculation unit 50 of the image processing device 20 calculates the reliability of parallax for each label using the reliability table 51, and generates the reliability image 150.
  • step S14 the parallax correction unit 60 of the image processing device 20 corrects parallaxes whose reliability does not meet the standard.
  • step S15 the recognition processing unit 70 of the image processing device 20 performs object detection using the parallax image 130, the reliability image 150, the right image 112, and the left image 111. Thereafter, the image processing device 20 performs a process according to the object detection result, such as transmitting a signal to the vehicle control device to issue an alarm or activate an automatic emergency brake, and ends this process.
  • FIG. 8 is a flowchart showing the processing performed by the reliability calculation unit 50.
  • FIG. 9 is a diagram showing an example of the structure of data written to one pixel of the reliability image 150.
  • the X coordinate value indicates the horizontal coordinate value of the label image 140 and the reliability image 150.
  • step S23 the reliability calculation unit 50 reads the label number of the coordinates (X, Y) from the label image 140 one pixel at a time.
  • step S24 the reliability calculation unit 50 obtains the reliability of the parallax associated with the label number from the reliability table 51.
  • step S25 the reliability calculation unit 50 writes the label number and reliability at the coordinates (X, Y) of the reliability image 150.
  • the data written at this time is structured as shown in FIG. 9, for example.
  • one pixel of data to be written has 8 bits, bits 0 to 3 are composed of reliability levels, and bits 4 to 7 are composed of label numbers.
  • step S27 the reliability calculation unit 50 determines whether the X coordinate value has reached the maximum value.
  • the maximum value of the X coordinate value indicates the coordinate value of the horizontal end of the label image 140 and the reliability image 150.
  • the reliability calculation unit 50 moves to step S28. If the X coordinate value has not reached the maximum value, the reliability calculation unit 50 moves to step S23.
  • step S29 the reliability calculation unit 50 determines whether the Y coordinate value has reached the maximum value.
  • the maximum value of the Y coordinate value indicates the coordinate value of the vertical end of the label image 140 and the reliability image 150. When the Y coordinate value reaches the maximum value, the reliability calculation unit 50 ends this process. If the Y coordinate value has not reached the maximum value, the reliability calculation unit 50 moves to step S22.
  • FIG. 10 is a flowchart showing the processing performed by the interpolation unit 61.
  • FIG. 11 is a diagram illustrating step S34 shown in FIG. 10. In FIGS. 10 and 11, an example will be described in which a road surface between the preceding vehicle and the host vehicle is processed by the interpolation unit 61.
  • step S31 the interpolation unit 61 determines whether there is a preceding vehicle in the reliability image 150. If there is a preceding vehicle, the interpolation unit 61 moves to step S32. If there is no preceding vehicle, the interpolation unit 61 ends this process.
  • step S32 the interpolation unit 61 determines whether there is an area where the reliability is "low” or "invalid” on the road surface between the preceding vehicle and the host vehicle. If there is an area with "low” or “invalid” reliability, the interpolation unit 61 moves to step S33. If there is no area with "low” or “invalid” reliability, the interpolation unit 61 ends this process.
  • step S33 the interpolation unit 61 determines whether there is a past image (label image 140 or reliability image 150) that shows a preceding vehicle traveling in an area with a "low” or “invalid" reliability. If there is a past image that shows a preceding vehicle, the interpolation unit 61 proceeds to step S34. If there is no past image that shows a preceding vehicle, the interpolation unit 61 ends this process.
  • step S34 the interpolation unit 61 interpolates the parallax whose reliability is "low” or “invalid” in units of raster using the parallax of the road surface whose reliability is "high”. After step S34, the interpolation unit 61 ends this process.
  • the interpolation unit 61 performs interpolation using the parallax of the road surface 300 adjacent to the area 310 and having a "high" reliability. Specifically, the interpolation unit 61 converts the parallaxes B to E of the pixels 311 to 314 constituting the area 310 to the parallax A of the pixel 301 of the road surface 300 adjacent to the pixel 311 in the horizontal direction, and the parallax of the pixel 314. Interpolation is performed using the parallax F of the pixels 302 of the road surface 300 that are laterally adjacent to each other.
  • Interpolation unit 61 calculates equation (1).
  • Interpolated value (parallax F - parallax A) / (X coordinate value of pixel 302 - X coordinate value of pixel 301)...(1)
  • the interpolation unit 61 calculates equation (2).
  • Parallax B Parallax A + Interpolation value ⁇ (X coordinate value of pixel 311 - X coordinate value of pixel 301)...(2)
  • Parallaxes C to E are also calculated in the same manner as in equation (2).
  • the interpolation unit 61 can interpolate the parallax whose reliability is "low” or "invalid” using the parallax of the road surface whose reliability is "high".
  • FIG. 12 is a flowchart showing the processing performed by the recalculation unit 62.
  • Coordinates (X, Y) indicate the position of each pixel in the reliability image 150 and the parallax image 130.
  • the Y coordinate value indicates the vertical coordinate value of the reliability image 150 and the parallax image 130.
  • the X coordinate value indicates the horizontal coordinate value of the reliability image 150 and the parallax image 130.
  • step S43 the recalculation unit 62 reads the reliability of the coordinates (X, Y) pixel by pixel from the reliability image 150.
  • step S44 the recalculation unit 62 determines whether the reliability is “medium” or “low”. If the reliability is “medium” or “low”, the recalculation unit 62 moves to step S45. If the reliability is not “medium” or “low”, the recalculation unit 62 moves to step S46.
  • step S44 The case where the reliability is not "medium” or “low” in step S44 includes the case where the reliability is "high” and the case where it is "invalid". If the reliability is "high”, parallax correction is not necessary, and the recalculation unit 62 moves to step S46. If the reliability is "invalid”, there is a low possibility that a parallax with a "high” reliability will be calculated even if the recalculation is performed, so the recalculation unit 62 proceeds to step S46 to suspend the parallax correction. Transition.
  • step S45 the recalculation unit 62 recalculates the parallax of the coordinates (X, Y) using a method different from that of the parallax calculation unit 30.
  • the recalculation unit 62 recalculates the parallax using, for example, stereo matching.
  • step S47 the recalculation unit 62 determines whether the X coordinate value has reached the maximum value.
  • the maximum value of the X coordinate value indicates the coordinate value of the lateral end of the reliability image 150 and the parallax image 130.
  • the recalculation unit 62 moves to step S48. If the X coordinate value has not reached the maximum value, the recalculation unit 62 moves to step S43.
  • step S49 the recalculation unit 62 determines whether the Y coordinate value has reached the maximum value.
  • the maximum value of the Y coordinate value indicates the coordinate value of the end of the reliability image 150 and the parallax image 130 in the vertical direction.
  • the recalculation unit 62 ends this process. If the Y coordinate value has not reached the maximum value, the recalculation unit 62 moves to step S42.
  • the image processing device 20 of this embodiment includes the parallax calculation unit 30 that calculates the parallax of the images 111 and 112 taken by the plurality of cameras 11 and 12, and the parallax calculation unit 30 that calculates the parallax of the images 111 and 112 that are captured by the plurality of cameras 11 and 12. , a labeling section 40 that assigns a label according to the type of the classified object to each pixel of the object; a reliability calculation section 50 that calculates the reliability of parallax for each assigned label; A parallax correction unit 60 that corrects parallax accordingly.
  • the image processing device 20 of the present embodiment calculates the reliability of parallax that changes on a pixel-by-pixel basis depending on the situation of the captured images 111 and 112, and clarifies inaccurate parallax on a pixel-by-pixel basis. Can be done. Therefore, the image processing device 20 of this embodiment can correct parallax according to the situation of the captured images 111 and 112, and therefore can calculate accurate parallax. Therefore, according to this embodiment, it is possible to provide an image processing device that can accurately calculate parallax.
  • the parallax correction unit 60 includes a recalculation unit 62 that recalculates parallax whose reliability does not meet the standard using a method different from that of the parallax calculation unit 30.
  • the image processing device 20 of the present embodiment can calculate the parallax using a plurality of calculation methods, so even if an error occurs in the parallax due to the characteristics of one calculation method, the image processing device 20 of the present embodiment can calculate the parallax using a plurality of calculation methods. Parallax can be compensated for. Therefore, the image processing device 20 of this embodiment can calculate more accurate parallax.
  • the disparity calculation unit 30 calculates the disparity using a neural network model in which the images 111 and 112 are set in the input layer and the disparity is set in the output layer, and performs re-computation.
  • the calculation unit 62 recalculates the parallax by stereo matching.
  • the image processing device 20 of the present embodiment can compensate for the parallax by stereo matching even if an error occurs in the parallax due to the characteristics of the calculation method using the neural network model. Therefore, the image processing device 20 of this embodiment can calculate more accurate parallax.
  • the parallax correction unit 60 has an interpolation unit 61 that interpolates parallax whose reliability does not meet a standard based on the parallax calculated for the previously captured images 111 and 112.
  • the image processing device 20 of this embodiment interpolates the parallax from the parallax of the images 111, 112 taken in the past even if the situation of the images 111, 112 taken currently is not suitable for calculating the parallax. It is possible to interpolate the parallax by determining the validity of this. Therefore, the image processing device 20 of this embodiment can calculate more accurate parallax.
  • the labeling unit 40 verifies the consistency between the classified object type and the label given to the object based on the parallax.
  • the image processing device 20 of the present embodiment is able to add labels more accurately, and therefore can accurately calculate reliability. Therefore, the image processing device 20 of this embodiment can calculate more accurate parallax.
  • the reliability calculation unit 50 calculates reliability using a reliability table 51 that shows the correspondence between labels and reliability.
  • the image processing device 20 of this embodiment can reduce the amount of calculation for calculating the reliability, and therefore can easily calculate the reliability even with a simple configuration. Therefore, the image processing device 20 of this embodiment can easily calculate accurate parallax.
  • the image processing device 20 of this embodiment includes a recognition processing unit 70 that detects objects around the vehicle in which the plurality of cameras 11 and 12 are mounted based on the corrected parallax, reliability, and images 111 and 112. Furthermore, the recognition processing unit 70 changes the size of the object to be detected or the distance to the object depending on the reliability.
  • the image processing device 20 of the present embodiment can exclude from detection targets a subject having a parallax whose reliability does not meet the standard. Therefore, the image processing device 20 of this embodiment can calculate accurate parallax and can suppress false detection of objects.
  • information on parallax, reliability, and label is input to the recognition processing unit 70 in association with each other for each pixel.
  • the image processing device 20 of the present embodiment can reconsider the reliability and label acquired in the pre-object detection stage, thereby improving the accuracy of object detection. Therefore, the image processing device 20 of this embodiment can calculate accurate parallax and improve the accuracy of object detection.
  • the present invention is not limited to the above embodiments, and includes various modifications.
  • the above embodiments have been described in detail to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to having all the configurations described.
  • each of the above-mentioned configurations, functions, processing units, processing means, etc. may be partially or entirely realized by hardware, for example, by designing an integrated circuit. Further, each of the above-mentioned configurations, functions, etc. may be realized by software by a processor interpreting and executing a program for realizing each function. Information such as programs, tapes, and files that implement each function can be stored in a memory, a recording device such as a hard disk, an SSD (solid state drive), or a recording medium such as an IC card, SD card, or DVD.
  • a recording device such as a hard disk, an SSD (solid state drive), or a recording medium such as an IC card, SD card, or DVD.
  • control lines and information lines are shown that are considered necessary for explanation, and not all control lines and information lines are necessarily shown in the product. In reality, almost all components may be considered to be interconnected.

Landscapes

  • Physics & Mathematics (AREA)
  • Electromagnetism (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
PCT/JP2022/035027 2022-09-20 2022-09-20 画像処理装置 Ceased WO2024062540A1 (ja)

Priority Applications (4)

Application Number Priority Date Filing Date Title
JP2024547981A JP7769811B2 (ja) 2022-09-20 2022-09-20 画像処理装置
EP22959504.6A EP4592638A1 (en) 2022-09-20 2022-09-20 Image processing device
PCT/JP2022/035027 WO2024062540A1 (ja) 2022-09-20 2022-09-20 画像処理装置
CN202280099875.4A CN119836559A (zh) 2022-09-20 2022-09-20 图像处理装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/035027 WO2024062540A1 (ja) 2022-09-20 2022-09-20 画像処理装置

Publications (1)

Publication Number Publication Date
WO2024062540A1 true WO2024062540A1 (ja) 2024-03-28

Family

ID=90454073

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/035027 Ceased WO2024062540A1 (ja) 2022-09-20 2022-09-20 画像処理装置

Country Status (4)

Country Link
EP (1) EP4592638A1 (https=)
JP (1) JP7769811B2 (https=)
CN (1) CN119836559A (https=)
WO (1) WO2024062540A1 (https=)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016173248A (ja) * 2015-03-16 2016-09-29 株式会社リコー 視差値演算装置、物体認識装置、移動体機器制御システム及び視差演算用プログラム
US20180255283A1 (en) * 2017-03-03 2018-09-06 Sony Corporation Information processing apparatus and information processing method
WO2019138678A1 (ja) * 2018-01-15 2019-07-18 キヤノン株式会社 情報処理装置及びその制御方法及びプログラム、並びに、車両の運転支援システム

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016173248A (ja) * 2015-03-16 2016-09-29 株式会社リコー 視差値演算装置、物体認識装置、移動体機器制御システム及び視差演算用プログラム
US20180255283A1 (en) * 2017-03-03 2018-09-06 Sony Corporation Information processing apparatus and information processing method
WO2019138678A1 (ja) * 2018-01-15 2019-07-18 キヤノン株式会社 情報処理装置及びその制御方法及びプログラム、並びに、車両の運転支援システム

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
KIRILLOV ET AL.: "Panoptic Feature Pyramid Networks", IEEE/CVF CONFERENCE ON CVPR, 2019

Also Published As

Publication number Publication date
JPWO2024062540A1 (https=) 2024-03-28
JP7769811B2 (ja) 2025-11-13
CN119836559A (zh) 2025-04-15
EP4592638A1 (en) 2025-07-30

Similar Documents

Publication Publication Date Title
JP6266238B2 (ja) 接近物検出システム、及び車両
CN111937036B (zh) 用于处理传感器数据的方法、设备和具有指令的计算机可读存储介质
US6658150B2 (en) Image recognition system
EP3007099B1 (en) Image recognition system for a vehicle and corresponding method
JP4919036B2 (ja) 移動物体認識装置
WO2021072696A1 (zh) 目标检测与跟踪方法、系统、可移动平台、相机及介质
US8406472B2 (en) Method and system for processing image data
US11474234B2 (en) Device and method for estimating distance based on object detection
JP7091686B2 (ja) 立体物認識装置、撮像装置および車両
US10984264B2 (en) Detection and validation of objects from sequential images of a camera
EP3866110A1 (en) Target detection method, target detection apparatus and unmanned aerial vehicle
CN115187941A (zh) 目标检测定位方法、系统、设备及存储介质
US11069049B2 (en) Division line detection device and division line detection method
EP4009228A1 (en) Method for determining a semantic free space
JP6907513B2 (ja) 情報処理装置、撮像装置、機器制御システム、情報処理方法およびプログラム
JP7769811B2 (ja) 画像処理装置
JP2018088217A (ja) 情報処理装置、撮像装置、機器制御システム、情報処理方法およびプログラム
JP7466695B2 (ja) 画像処理装置
US11010625B2 (en) Vehicle exterior environment recognition apparatus and method of recognizing exterior environment outside vehicle
JP7134780B2 (ja) ステレオカメラ装置
JP7514139B2 (ja) 車外環境認識装置
EP4579581A1 (en) External world recognition device and external world recognition method
CN116580051B (zh) 对象追踪方法、装置、终端设备及存储介质
WO2024150543A1 (ja) 車載カメラシステム、及び、画像処理方法
KR20240103749A (ko) 차량 및 그 제어방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22959504

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202280099875.4

Country of ref document: CN

WWE Wipo information: entry into national phase

Ref document number: 2024547981

Country of ref document: JP

WWP Wipo information: published in national office

Ref document number: 202280099875.4

Country of ref document: CN

WWE Wipo information: entry into national phase

Ref document number: 2022959504

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2022959504

Country of ref document: EP

Effective date: 20250422

WWP Wipo information: published in national office

Ref document number: 2022959504

Country of ref document: EP