JP7117566B2 - Information processing system and information processing method - Google Patents

Description

The present disclosure relates to an information processing system and an information processing method.

Patent Document 1 discloses a system configured to recognize an object by sensor fusion using output data of a first sensor and output data of a second sensor.

JP 2017-102838 A

However, in the prior art, there are cases where the object recognition result using sensor fusion is worse than the object recognition result using individual sensors. The present disclosure provides an information processing system and an information processing method capable of suppressing deterioration of object recognition results due to fusion of recognition results.

An information processing system according to an aspect of the present disclosure includes a first acquisition unit that acquires a first recognition result that is a recognition result of an object based on sensing data from a first sensor; A second acquisition unit that acquires a second recognition result that is a recognition result of an object based on the sensing data of the above, and a first determination unit that performs a first determination that determines the degree of similarity between the first recognition result and the second recognition result a fusion control unit that controls fusion processing of the first recognition result and the second recognition result according to the result of the first determination; and the first recognition result according to the result of the first determination. and an output unit that outputs at least one of the second recognition result and a third recognition result obtained by fusion of the first recognition result and the second recognition result.

Further, an information processing system according to an aspect of the present disclosure includes a first acquisition unit that acquires a first recognition result that is a recognition result of an object based on sensing data from a first sensor; a second acquisition unit that acquires a second recognition result that is a recognition result of an object based on sensing data from a sensor; a fusion processing unit that performs fusion processing on the first recognition result and the second recognition result; and the first recognition determining a first similarity degree and a second similarity degree, which are degrees of similarity between a third recognition result obtained by fusion processing the result and the second recognition result, and the first recognition result and the second recognition result, respectively; and an output unit that outputs at least one of the first recognition result, the second recognition result, and the third recognition result according to the result of the second determination. Prepare.

It should be noted that these general or specific aspects may be realized by methods, integrated circuits, computer programs, or recording media such as computer-readable CD-ROMs. Any combination may be used.

The information processing system and information processing method according to the present disclosure can suppress deterioration of object recognition results due to fusion of recognition results.

FIG. 1 is an external view of an information processing system according to Embodiment 1. FIG. FIG. 2 is a block diagram showing an example of a hardware configuration of an information processing system including the information processing device according to Embodiment 1. As shown in FIG. 3 is a block diagram illustrating an example of the functional configuration of the information processing system according to Embodiment 1. FIG. FIG. 4 is a table showing whether or not the accuracy of the third recognition result is improved in a plurality of cases according to similarity and recognition accuracy. 5 is a flowchart showing an example of an information processing method in the information processing system according to Embodiment 1. FIG. FIG. 6 is a block diagram showing an example of a functional configuration of an information processing system according to Embodiment 2. As shown in FIG. FIG. 7 is a table showing whether or not the performance of the third recognition result is improved in a plurality of cases according to similarity and recognition accuracy. FIG. 8 is a flow chart showing an example of an information processing method in the information processing system according to the second embodiment. FIG. 9 is a block diagram showing an example of a functional configuration of an information processing system according to Embodiment 3. As shown in FIG. FIG. 10 is a table showing whether or not the performance of the third recognition result is improved in a plurality of cases according to similarity and recognition accuracy. 11 is a flowchart illustrating an example of an information processing method in an information processing system according to Embodiment 3. FIG.

(Knowledge on which the present invention is based)
The inventor of the present invention has found that the following problems arise with the above-described prior art.

As in the technique described in Patent Literature 1, there are cases where recognition accuracy is not improved even when object recognition is executed using the output data of the first sensor and the output data of the second sensor. In this case, there is a risk of outputting a recognition result with low accuracy.

When fusion is used for recognition results based on sensor data obtained from each of a plurality of types of sensors, if the recognition result corresponding to the first sensor of the plurality of types of sensors is far from correct, that is, the first sensor If the accuracy of the recognition result corresponding to is low, it may adversely affect the overall recognition result. For example, when the sensing data of the first sensor contains a lot of noise, when the first sensor is out of order, or when the first sensor is out of order, the recognition result corresponding to the first sensor may contain a large error. becomes.

Moreover, it is difficult to determine whether the sensing data of the first sensor is accurate sensing data. For example, if the sensing data is an image, if it is an image captured at night, it is possible to determine whether the image includes rough noise or an image of an object made of a rough material. difficult to distinguish from the value.

In order to solve such problems, an information processing system according to an aspect of the present disclosure includes a first acquisition unit that acquires a first recognition result that is a recognition result of an object based on sensing data from a first sensor; a second acquisition unit that acquires a second recognition result that is a recognition result of an object based on sensing data from a second sensor that is different from the first sensor; and a degree of similarity between the first recognition result and the second recognition result is determined. a first determination unit that performs a first determination; a fusion control unit that controls fusion processing of the first recognition result and the second recognition result according to the result of the first determination; and a result of the first determination an output unit that outputs at least one of the first recognition result, the second recognition result, and a third recognition result obtained by fusion of the first recognition result and the second recognition result, in response to Prepare.

According to this, the fusion processing of the first recognition result and the second recognition result is controlled according to the similarity determination result of the first recognition result and the second recognition result, and the first recognition result and the second recognition result are controlled. At least one of the result and the third recognition result obtained by the fusion processing is output. Therefore, it is possible to suppress the deterioration of the recognition result of the object due to the fusion of the recognition result.

Further, the first recognition result and the second recognition result may be recognition results of positions of objects, and the degree of similarity may be a degree of similarity regarding the positions of the objects.

According to this, at least one of the first recognition result, the second recognition result, and the third recognition result is output according to the determination result of the degree of similarity of the position of the recognized object. Deterioration of recognition results can be suppressed.

Further, the first recognition result and the second recognition result may be recognition results of attributes of objects, and the degree of similarity may be the degree of similarity of attributes of the objects.

According to this, at least one of the first recognition result, the second recognition result, and the third recognition result is output according to the determination result of the similarity of the attribute of the recognized object. Deterioration of recognition results can be suppressed.

Further, the first recognition result and the second recognition result may be recognition results of existence of objects, and the similarity may be similarity of the number of the objects.

According to this, at least one of the first recognition result, the second recognition result, and the third recognition result is output according to the similarity determination result of the number of recognized objects. Deterioration of recognition results can be suppressed.

Further, a third obtaining unit for obtaining first correlation information correlated with the first evaluation value of the first recognition result and second correlation information correlated with the second evaluation value of the second recognition result and the fusion control unit may control fusion processing according to a result of the first determination and the acquired first correlation information and second correlation information.

According to this, by obtaining the first correlation information and the second correlation information, it is possible to perform the fusion processing considering the first evaluation value of the first recognition result and the second evaluation value of the second recognition result. . That is, by controlling the fusion processing according to the similarity determination result of the first recognition result and the second recognition result and the first correlation information and the second correlation information, a good object recognition result can be obtained. can be done.

Further, the first correlation information and the second correlation information are respectively the environment of the space in which the first sensor and the second sensor are arranged at the time of sensing, the first sensor and the second sensor at the time of sensing, respectively. and specifications of the first sensor and the second sensor.

According to this, at least one of the environment of the space where the first sensor and the second sensor are arranged at the time of sensing, the state of the first sensor and the second sensor at the time of sensing, and the specifications of the first sensor and the second sensor By obtaining one, it is possible to easily perform the fusion processing in consideration of the first evaluation value and the second evaluation value.

Further, a second determination is performed to determine a first similarity and a second similarity, which are degrees of similarity between the third recognition result and the first recognition result and the second recognition result, respectively. The output unit may output at least one of the first recognition result, the second recognition result, and the third recognition result according to the result of the second determination.

According to this, since at least one of the first recognition result, the second recognition result and the third recognition result is output according to the first similarity and the second similarity, a good object recognition result can be obtained. can be done.

Further, the fusion control section may control the fusion processing depending on whether or not to perform the fusion processing on the first recognition result and the second recognition result.

According to this, whether or not to perform the fusion process is controlled according to the result of the first determination. Therefore, for example, if the fusion process is performed according to the result of the first determination, the recognition result does not improve. By not performing the fusion process when it is predicted, it is possible to reliably prevent deterioration of the recognition result of the object due to the fusion of the recognition result.

The fusion control unit determines whether to perform fusion processing on the first recognition result and the second recognition result depending on whether the first recognition result or the second recognition result is input to a fusion processing unit that performs fusion processing. You may control whether or not

According to this, the input of the first recognition result and the second recognition result to the fusion processing section is controlled according to the result of the first determination. Therefore, even if the operation of the fusion processing section cannot be directly controlled, it is possible to control whether or not to perform the fusion processing.

Further, the fusion control section may control the fusion processing by adjusting a parameter for the fusion processing of the first recognition result and the second recognition result.

According to this, the parameters of the fusion processing are adjusted according to the result of the first determination. For this reason, for example, when it is predicted that the recognition result will not improve even if the fusion process is executed according to the result of the first determination, the weight in the fusion process is reduced, so that the fusion process is performed while the recognition result is improved. It is possible to suppress the deterioration of the recognition result of the object due to the fusion.

Further, an information processing system according to an aspect of the present disclosure includes a first acquisition unit that acquires a first recognition result that is a recognition result of an object based on sensing data from a first sensor; a second acquisition unit that acquires a second recognition result that is a recognition result of an object based on sensing data from a sensor; a fusion processing unit that performs fusion processing on the first recognition result and the second recognition result; and the first recognition determining a first similarity degree and a second similarity degree, which are degrees of similarity between a third recognition result obtained by fusion processing the result and the second recognition result, and the first recognition result and the second recognition result, respectively; and an output unit that outputs at least one of the first recognition result, the second recognition result, and the third recognition result according to the result of the second determination. Prepare.

According to this, since at least one of the first recognition result, the second recognition result and the third recognition result is output according to the first similarity and the second similarity, a better object recognition result can be obtained. be able to.

Further, a recognition unit that executes object recognition processing based on sensing data from the first sensor and sensing data from the second sensor, and outputs the first recognition result and the second recognition result. may be provided.

According to this, the first recognition result and the second recognition result can be obtained by acquiring the sensing data from the first sensor and the sensing data from the second sensor.

In addition, these general or specific aspects may be realized by a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM. Any combination may be used.

Hereinafter, an information processing system and an information processing method according to one aspect of the present invention will be specifically described with reference to the drawings.

It should be noted that each of the embodiments described below is a specific example of the present invention. Numerical values, shapes, materials, components, arrangement positions and connection forms of components, steps, order of steps, and the like shown in the following embodiments are examples and are not intended to limit the present invention. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in independent claims representing the highest concept will be described as arbitrary constituent elements.

(Embodiment 1)
Embodiment 1 will be described below with reference to FIGS. 1 to 5. FIG.

[1-1. Constitution]
FIG. 1 is an external view of an information processing system according to Embodiment 1. FIG.

Information processing system 10 according to Embodiment 1 is a system installed in vehicle 200 . The information processing system 10 includes an information processing device 100, a first sensor 201, and a second sensor 202, as shown in FIG. That is, vehicle 200 includes information processing device 100 , first sensor 201 and second sensor 202 .

The first sensor 201 is, for example, an imaging device that images the front of the vehicle 200 . The first sensor 201 is not limited to the area in front of the vehicle 200, and may image the surroundings of the vehicle 200, and may image the left direction, the right direction, the rear direction, and the like of the vehicle 200. FIG. An image as sensing data obtained by the first sensor 201 is used, for example, to recognize objects around the vehicle 200 . The recognized result is used, for example, for automatic driving or driving assistance of the vehicle 200 .

The second sensor 202 is a distance sensor that detects the distance to objects around the vehicle 200 . Distance information indicating the distance detected by the second sensor 202 is used, for example, to estimate the position of the vehicle 200 in automatic driving or automatic driving assistance.

The information processing apparatus 100 uses, for example, sensing data obtained by the first sensor 201 and sensing data obtained by the second sensor 202 to perform recognition processing on each sensing data. Then, the information processing apparatus 100 generates a first recognition result that is a recognition result based on the sensing data obtained by the first sensor 201 and a second recognition result that is a recognition result based on the sensing data obtained by the second sensor 202. and output a more accurate recognition result.

Next, a specific example of the hardware configuration of the information processing system 10 including the information processing device 100 will be described with reference to FIG.

FIG. 2 is a block diagram showing an example of a hardware configuration of an information processing system including the information processing device according to Embodiment 1. As shown in FIG.

As shown in FIG. 2, the information processing system 10 includes an information processing device 100, a first sensor 201, and a second sensor 202 as a hardware configuration. The information processing system 10 may further include a third sensor 203 and an operation control device 300 .

The information processing apparatus 100 includes a processor 101, a main memory 102, a storage 103, and a communication IF (Interface) 104 as a hardware configuration. Further, the information processing apparatus 100 may be an ECU (Electronic Control Unit), for example.

The processor 101 is a processor that executes a control program stored in the storage 103 or the like.

The main memory 102 is a volatile storage area used as a work area when the processor 101 executes the control program.

The storage 103 is a nonvolatile storage area that holds control programs, contents, and the like.

Communication IF 104 is a communication interface that communicates with first to third sensors 201 to 203 and operation control device 300 via a communication network such as CAN (Controller Area Network). Note that the communication interface 104 is not limited to a communication interface for wired communication, and may be a communication interface for wireless communication. Further, the communication IF 104 may be any communication interface as long as it can establish a communication connection with the first to third sensors 201 to 203, the operation control device 300, and the like. Also, the communication IF 104 may be a communication interface capable of communication connection with a general-purpose network such as the Internet or a dedicated network.

The first sensor 201 is an imaging device having an optical system such as a lens and an image sensor, that is, a camera. The first sensor 201 is communicably connected to the information processing apparatus 100 . The first sensor 201 captures images at a plurality of different timings, eg, 60 fps, and outputs the captured images as sensing data.

The second sensor 202 is a distance sensor that detects the distance to objects around the vehicle 200 . Specifically, the second sensor 202 measures the distance to an object within a detection range of an angle range of 360 degrees in all directions in the horizontal direction of the vehicle 200 and a predetermined angle (for example, 30 degrees) in the vertical direction. To detect. The distances detected by the second sensor 202 are used, for example, to generate a three-dimensional shape of terrain including objects around the vehicle 200 . The second sensor 202 is, for example, a laser sensor such as LIDAR (Light Detection and Ranging). The second sensor 202 outputs sensing data obtained by sensing at a plurality of different timings.

The third sensor 203 is, for example, an illuminance sensor arranged in the space where the first sensor 201 and the second sensor 202 are arranged and detecting the illuminance of the space. The third sensor 203 may be arranged inside the vehicle 200 or may be arranged outside the vehicle 200, for example. The third sensor 203 outputs sensing data obtained by sensing at a plurality of different timings.

The driving control device 300 is an information processing device that controls driving of the vehicle 200 . The operation control device 300 has, for example, a processor, main memory, storage, communication IF and the like. The operation control device 300 may be implemented with a common configuration with the information processing device 100 . That is, operation control device 300 may be realized by processor 101, main memory 102, storage 103, and communication IF104. Further, the operation control device 300 may be implemented by an ECU, for example, and when the information processing device 100 is implemented by an ECU, it may be implemented by an ECU that implements the information processing device 100, or the information processing device 100 may be implemented by an ECU. It may be implemented by an ECU different from the ECU that implements the processing device 100 .

In addition to the first to third sensors 201 to 203, the information processing system 10 includes various sensors such as an acceleration sensor that detects acceleration in each of the three axial directions of the vehicle 200, a gyro sensor that detects the rotational angular velocity of the vehicle 200, and the like. may have The information processing system 10 also uses GNSS (Global Navigation System) for detecting the current position of the vehicle 200 .
Satellite System) receiver.

The driving control device 300 is an information processing device that controls driving of the vehicle 200 . Specifically, the operation control device 300 controls the steering that steers the wheels, the engine that rotates the wheels, the power source such as the motor, the brake that brakes the wheels, and the like, thereby automatically driving or driving the vehicle 200. provide support. The operation control device 300 has, for example, a processor, main memory, storage, communication IF and the like. The operation control device 300 may be implemented with a common configuration with the information processing device 100 . That is, operation control device 300 may be realized by processor 101, main memory 102, storage 103, and communication IF104. Further, the operation control device 300 may be implemented by an ECU, for example, and when the information processing device 100 is implemented by an ECU, it may be implemented by an ECU that implements the information processing device 100, or the information processing device 100 may be implemented by an ECU. It may be implemented by an ECU different from the ECU that implements the processing device 100 .

Next, the functional configuration of the information processing system 10 will be described using FIG.

3 is a block diagram illustrating an example of the functional configuration of the information processing system according to Embodiment 1. FIG.

The information processing system 10 includes a first acquisition section 120 , a second acquisition section 130 , a first determination section 140 , a fusion control section 160 , a second determination section 170 and an output section 180 . The information processing system 10 may further include a first sensor 201, a second sensor 202, a third sensor 203, or a recognition unit 110. It may be provided, or the third acquisition unit 150 may be provided. The description of the configuration of the first to third sensors 201 to 203 is omitted here because it is the same as described above.

The recognition section 110 has a first recognition section 111 and a second recognition section 112 . The first recognition unit 111 executes object recognition processing (hereinafter also simply referred to as recognition processing) based on sensing data from the first sensor 201, and outputs a first recognition result as a result of the recognition processing. . Here, object recognition includes various kinds of object-related recognition, such as recognition of the existence of an object, recognition of the distance to the object, recognition of the attribute of the object, and recognition of the size of the object. The first recognition unit 111 acquires a plurality of sensing data obtained by the first sensor 201 sensing at a plurality of different timings, and performs recognition processing based on the acquired plurality of sensing data. The first recognition unit 111 may perform recognition processing on each of a plurality of sensing data, or may perform recognition processing on each of a plurality of combinations of two or more sensing data among the plurality of sensing data. .

The second recognition unit 112 executes recognition processing based on sensing data from the second sensor 202, and outputs a second recognition result as a result of the recognition processing. The second recognition unit 112 may acquire a plurality of sensing data obtained by sensing at a plurality of different timings of the second sensor 202, and perform recognition processing on each of the acquired plurality of sensing data. Recognition processing may be performed for each of a plurality of combinations of two or more pieces of sensing data among the plurality of sensing data.

The recognition processing executed by the recognition unit 110 is, for example, recognition processing using a machine learning model. Specifically, the first recognition result and the second recognition result are object recognition results. Further, the first recognition result and the second recognition result may be, for example, the result of recognizing the position of the object, the result of recognizing the attribute of the object, or the result of recognizing the number of objects. may be Recognition unit 110 may be implemented by information processing apparatus 100 or may be implemented by an information processing apparatus different from information processing apparatus 100 that is communicatively connected to information processing apparatus 100 .

The first acquisition unit 120 acquires the first recognition result from the first recognition unit 111 . The first acquisition unit 120 is implemented by, for example, the processor 101, the main memory 102, the storage 103, the communication IF 104, and the like.

A second acquisition unit 130 acquires the second recognition result from the second recognition unit 112 . The second acquisition unit 130 is implemented by, for example, the processor 101, the main memory 102, the storage 103, the communication IF 104, and the like.

The first determination unit 140 performs a first determination to determine the degree of similarity between the first recognition result and the second recognition result. The first recognition result and the second recognition result to be subjected to the first determination by the first determination unit 140 are, for example, the same space for the first sensor 201 and the second sensor 202, or a space including a common space. It is a recognition result based on sensing data obtained by sensing . That is, the sensing data on which the first recognition result to be subjected to the first determination is based and the sensing data on which the second recognition result is based are, for example, detected by the first sensor 201 and the second sensor 202 at the same timing. This is sensed data. Note that the same timing may be different timings with a small time difference that can be regarded as the same timing. The first determination unit 140 is implemented by, for example, the processor 101, the main memory 102, the storage 103, and the like.

Note that when the vehicle 200 is not moving, the spaces sensed by the first sensor 201 and the second sensor 202 do not change. The first recognition result and the second recognition result based on the two sensing data obtained by the above may not be used. That is, in the first determination in this case, the first recognition result and the second recognition result based on the two sensing data obtained by the first sensor 201 and the second sensor 202 at mutually different timings are used for the first determination. may

A first example of the degree of similarity is the degree of similarity regarding the position of the recognized object. Distance similarity is the amount of change in the position of the recognized object per unit time. The amount of change in the position of the recognized object per unit time is the moving speed of the position of the recognized object. For example, the first determination unit 140 determines the difference between the position of the object recognized in the first recognition result and the position of the object recognized in the second recognition result, the average value or the median value, and each recognized position. If it is less than the threshold value, it may be determined that the similarity degree is high, and if it is equal to or greater than a predetermined threshold value, it may be determined that the similarity degree is low. Similarly, the first determination unit 140 determines the difference between the average value or median value of the distance to the object and each recognized distance, or the average value or median value of the amount of change per unit time and the recognized each The degree of similarity may be determined depending on whether or not the difference from the amount of change in is less than a predetermined threshold.

A second example of the degree of similarity is the degree of similarity of the attribute of the recognized object, more specifically, the type, color, size, shape, etc. of the recognized object. For example, if the type of the object recognized in the first recognition result and the type of the object recognized in the second recognition result belong to the same classification in a predetermined classification, the first determination unit 140 determines the similarity The similarity may be determined to be high, and if they belong to different classifications, the similarity may be determined to be low. Similarly, the first determination unit 140 may determine the degree of similarity of the color or shape of an object according to whether it belongs to the same classification in a predetermined classification. . In addition, the first determination unit 140 determines the difference between the average value or median value of the numerical value of the index that quantifies the color of the object and the numerical value of each recognized color index, or the average value or median value of the size. and each recognized size, the degree of similarity may be determined according to whether or not the difference is less than a predetermined threshold.

A third example of similarity is the number of recognized objects. That is, the similarity of the number of recognized objects is the number of objects recognized in the space targeted for sensing by the first sensor 201 and the number of objects recognized in the space targeted for sensing by the second sensor 202. is the similarity of The first determination unit 140 determines, for example, the number of objects recognized in the first recognition result and the difference between the average or median number of objects recognized in the second recognition result and the number of each recognized object. If is less than a predetermined threshold, it may be determined that the similarity is high, and if it is equal to or greater than the predetermined threshold, it may be determined that the similarity is low.

Note that the predetermined threshold value may be set to a different value according to each degree of similarity.

The third acquisition unit 150 acquires first correlation information correlated with the first evaluation value of the first recognition result and second correlation information correlated with the second evaluation value of the second recognition result. The third acquisition unit 150 may acquire the first correlation information and the second correlation information from the third sensor 203, or may acquire the first correlation information and the second correlation information from an external device (not shown).

Here, the first evaluation value and the second evaluation value are, for example, accuracy rate, precision rate, recall rate, F value, error, incorrect answer rate, and the like. Hereinafter, evaluation values including the first evaluation value and the second evaluation value are also referred to as recognition accuracy. Each of the first correlation information and the second correlation information includes the environment of the space where the first sensor 201 and the second sensor 202 are arranged at the time of sensing, the state of the first sensor 201 and the second sensor 202 at the time of sensing, and At least one of the specifications of the first sensor 201 and the second sensor 202 .

The environment of the space relates to the illuminance of the space, for example. The illuminance of the space may be the illuminance detected by the third sensor 203, the illuminance predicted based on weather information obtained from an external device, or the illuminance predicted based on the current time. It may be illuminance. When the illuminance of the space is lower than the first illuminance, the image captured by the camera, which is an example of the first sensor 201, has a large number of pixels with pixel values below the noise level of the image sensor of the camera. Accurate object recognition becomes difficult. On the other hand, when the illuminance of the space is higher than the second illuminance, which is higher than the first illuminance, the image captured by the camera has many pixel values exceeding the maximum value that can be captured by the image sensor of the camera. It becomes difficult to accurately recognize an object from the image. In this way, the illuminance of the space is correlated with the evaluation value of the recognition processing using the image, which is the sensing data obtained by the camera. Therefore, the illuminance of the space may be used to determine whether the accuracy of the first recognition result is high or low.

The illuminance of the space may also affect the distance sensor, which is an example of the second sensor 202 . For example, when sunlight or reflected sunlight is incident on the distance sensor, that is, when an illuminance higher than the third illuminance is detected in the vicinity of the distance sensor, the sensing data obtained by the distance sensor can accurately detect the object. difficult to recognize well. Therefore, the illuminance of the space may be used to determine whether the accuracy of the second recognition result is high or low.

The states of the first sensor 201 and the second sensor 202 include the degree of contamination of the transmitting section or the receiving section of each sensor, the state of failure of each sensor, and the like. In the case of a distance sensor, the transmitting section is a section that emits or emits light such as a laser, radio waves, sound waves, or the like. In the case of a camera, the receiving section is the light receiving section of the camera, and in the case of a distance sensor, it is a section that receives reflected waves such as irradiated or transmitted light, radio waves, and sound waves. When the states of the first sensor 201 and the second sensor 202 indicate that the transmitter or receiver is dirty, the accuracy of the recognition result obtained from the sensing data by the sensor corresponding to the dirty transmitter or receiver can be assumed to be low.

Also, although not shown, the third acquisition unit 150 may acquire a value based on sensing data from the first sensor 201 as the first correlation information from the first sensor 201 .

In this case, for example, the first correlation information is the variance of all pixel values of the image captured by the camera. If the variance of the pixel values of the image is larger than the first variance, it can be assumed that the image contains a lot of noise, making it difficult to accurately recognize an object from the image. On the other hand, when the variance of the pixel values of the image is smaller than the second variance, which is smaller than the first variance, the edge cannot be detected, the pixel value is too dark and the pixel value is small, or the pixel value is uniform because the image is fog or rain. Therefore, it becomes difficult to accurately recognize an object from the image. In this way, the variance of pixel values of an image is correlated with the evaluation value of recognition processing using an image that is sensing data obtained by a camera. Therefore, the variance of the pixel values of the image may be used to determine whether the accuracy of the first recognition result is high or low.

The specifications of the first sensor 201 and the second sensor 202 are specifications that indicate the performance of each of the first sensor 201 and the second sensor 202 .

The third acquisition unit 150 is implemented by, for example, the processor 101, the main memory 102, the storage 103, the communication IF 104, and the like.

Fusion control section 160 controls fusion processing according to the result of the first determination and the acquired first correlation information and second correlation information. Specifically, the fusion control unit 160 (1) performs fusion processing on the first recognition result and the second recognition result, (2) does not perform fusion processing on the first recognition result and the second recognition result, and (3) The fusion processing is controlled by performing any one of adjusting the parameters of the fusion processing of the first recognition result and the second recognition result. Note that the parameters of the fusion processing are weights in the fusion processing of the first recognition result and the second recognition result.

If the accuracy of at least one of the first recognition result and the second recognition result is estimated to be high according to the first correlation information and the second correlation information, fusion control section 160 combines the first recognition result and the second recognition result. are fusion-processed. For example, the fusion control unit 160 inputs the first recognition result and the second recognition result to the fusion processing module and obtains the third recognition result as an output. Further, if the accuracy of the first recognition result and the second recognition result is estimated to be low, fusion control section 160 combines the first recognition result and the second recognition result according to the first correlation information and the second correlation information. Do not fuse. For example, the fusion control unit 160 does not input the first recognition result and the second recognition result to the fusion processing module. If the accuracy of one of the first recognition result and the second recognition result is estimated to be low according to the first correlation information and the second correlation information, fusion control section 160 estimates that the accuracy is low. A fusion process may be performed in which the weight of a recognition result that is estimated to be accurate is made smaller than the weight of a recognition result that is estimated to be highly accurate.

A fusion control unit 160 outputs a third recognition result obtained by performing fusion processing of the first recognition result and the second recognition result. The fusion control unit 160 is implemented by, for example, the processor 101, the main memory 102, the storage 103, and the like. Note that the fusion control unit 160 may perform the fusion processing itself, or may cause a fusion processing module that performs the fusion processing to perform the fusion processing. The fusion processing module may be provided in the fusion control section 160 or may be provided outside the fusion control section 160 .

Now, let us consider with reference to FIG. 4 how the third recognition result obtained by the fusion control unit 160 improves the recognition accuracy more than the first recognition result or the second recognition result.

FIG. 4 is a table showing whether or not the accuracy of the third recognition result is improved in a plurality of cases according to similarity and recognition accuracy. FIG. 4 shows whether the degree of similarity between the first recognition result and the second recognition result is large or small, and whether both the degrees of similarity between the first recognition result and the second recognition result and the third recognition result are large. or whether the recognition accuracy of the first recognition result is high or low, and whether the recognition accuracy of the second recognition result is high or low.

The combination of recognition accuracy in FIG. 4 indicates the combination of the accuracy of the first recognition result and the accuracy of the second recognition result. "High/High" indicates that the accuracy of the first recognition result and the accuracy of the second recognition result are both high, and "High/Low" indicates that the accuracy of the first recognition result and the accuracy of the second recognition result are both high. A case where one of the accuracies is high and the other is low is indicated, and "low/low" indicates a case where the accuracy of the first recognition result and the accuracy of the second recognition result cannot be determined. Also, "high similarity" and "low similarity" in FIG. show. "Large similarity with third recognition result" and "Large similarity with third recognition result" in "Large similarity" A case where two degrees of similarity between each and the third recognition result are both large, and a case where at least one of the two degrees of similarity is small are shown. Similarly, "high similarity with third recognition result" and "low similarity with third recognition result" in "low similarity" correspond to the first recognition result and the second recognition result in "low similarity", respectively. A case where two degrees of similarity between each of the recognition results and the third recognition result are both large and a case where the two degrees of similarity are small are shown. The two degrees of similarity referred to here correspond to a first degree of similarity and a second degree of similarity, which will be described later.

As shown in FIG. 4, when the combination of recognition accuracies is "high" and "high", the accuracy of the third recognition result improves regardless of the degree of similarity. For example, when the combination of recognition accuracies is "high" and "high", in the case of "similarity is large", both the accuracy of the first recognition result and the second recognition result are high, and the similarity of the recognition result is Since it is large, the accuracy of similar recognition results is high. Therefore, in this case, it contributes to improving the accuracy of mutual recognition results. Further, when the combination of recognition accuracies is "high/high", in the case of "low similarity", the accuracy of both the first recognition result and the second recognition result is high, and the similarity of the recognition result is Since they are small, they are not similar, that is, the accuracy of the recognition results of different recognition targets is high. Therefore, in this case, it contributes to an improvement in accuracy in which the mutual recognition results are complemented each other.

Conversely, when the combination of recognition accuracies is "low/low", the accuracy of the third recognition result increases or decreases regardless of the degree of similarity. This is because when the combination of recognition accuracies is "low/low", both the accuracy of the first recognition result and the accuracy of the second recognition result are low, so the combination does not necessarily contribute to accuracy improvement or performance supplementation. is.

When the combination of recognition accuracies is "high/low", the accuracy of the third recognition result is improved if the "similarity is high", but if the "similarity is low", the third recognition result is the same. The accuracy of the third recognition result is improved or lowered according to the degree of similarity between the two. Specifically, when the combination of recognition accuracies is "high" and "low", even if the degree of similarity between the first recognition result and the second recognition result is small, the third recognition result, which is the result of fusion processing, is If the degree of similarity between the images is large, it is considered that the fusion processing was performed effectively because the deviation from the result of the fusion processing is small. Therefore, in this case, the recognition results complement each other, contributing to improvement in performance. On the other hand, if the degree of similarity with the third recognition result, which is the result of the fusion processing, is small, it is considered that the fusion processing has not been executed effectively because the deviation from the result of the fusion processing is large. Therefore, in this case, fusion processing causes a decrease in accuracy.

In this way, the combination of high and low recognition accuracies of the first recognition result and the second recognition result, the magnitude of the similarity between the first recognition result and the second recognition result, and the first recognition result and the second recognition result, respectively. Depending on the degree of similarity with the third recognition result, it can be classified into cases where the accuracy or accuracy of the third recognition result, which is the result of fusion processing, is improved, and cases where accuracy or performance is reduced. Therefore, by considering the case shown in FIG. 4, fusion processing can be performed more effectively, and the output of recognition results with low accuracy or low performance can be reduced.

The second determination unit 170 performs a second determination to determine a first similarity and a second similarity, which are degrees of similarity between the third recognition result and the first recognition result and the second recognition result, respectively. That is, in the second determination, the second determination unit 170 determines the first similarity, which is the similarity between the third recognition result and the first recognition result, and also determines the degree of similarity between the third recognition result and the second recognition result. A second degree of similarity is determined. The second determination unit 170 is implemented by, for example, the processor 101, the main memory 102, the storage 103, and the like.

The output unit 180 outputs at least one of the first recognition result, the second recognition result, and the third recognition result according to the result of the second determination. Further, when the accuracy of the first recognition result and the second recognition result is estimated to be high according to the first correlation information and the second correlation information, output section 180 outputs the first recognition result, the second recognition result, and the At least the third recognition result among the third recognition results may be output.

Further, output section 180 outputs the first recognition result and the second recognition result when one of the first recognition result and the second recognition result is estimated to be highly accurate according to the first correlation information and the second correlation information. If the similarity of the results is high, at least the third recognition result among the first recognition result, the second recognition result, and the third recognition result may be output. Further, output section 180 outputs the first recognition result and the second recognition result when one of the first recognition result and the second recognition result is estimated to be highly accurate according to the first correlation information and the second correlation information. If the degree of similarity of the result is small and the degree of similarity between the third recognition result and the first recognition result and the second recognition result are both large, then the first recognition result, the second recognition result, and the third recognition result Of the recognition results, at least the third recognition result may be output. Further, output section 180 outputs the first recognition result and the second recognition result when one of the first recognition result and the second recognition result is estimated to be highly accurate according to the first correlation information and the second correlation information. If the similarity of the result is small and at least one of the two similarities between the third recognition result and each of the first recognition result and the second recognition result is small, the third recognition result may not be output. Output section 180 outputs the first recognition result and the second recognition result when it is estimated that the accuracy of one of the first recognition result and the second recognition result is high according to the first correlation information and the second correlation information. When outputting at least the third recognition result out of the result and the third recognition result, outputting the recognition result estimated to be the most accurate of the first recognition result and the second recognition result together with the third recognition result. good too.

Further, output section 180 outputs the first recognition result and the second recognition result when it is estimated that the accuracy of one of the first recognition result and the second recognition result is low according to the first correlation information and the second correlation information. At least one of the results may be output.

The output unit 180 is implemented by, for example, the processor 101, the main memory 102, the storage 103, the communication IF 104, and the like.

[1-2. motion]
Next, operation of the information processing system 10 according to the first embodiment will be described.

5 is a flowchart showing an example of an information processing method in the information processing system according to Embodiment 1. FIG.

First, in the information processing system 10, the first recognition unit 111 of the recognition unit 110 executes recognition processing based on sensing data from the first sensor 201, and outputs a first recognition result as a result of the recognition processing. (S11).

Then, the first acquisition unit 120 acquires the first recognition result from the first recognition unit 111 (S12).

On the other hand, the second recognition unit 112 of the recognition unit 110 executes recognition processing based on sensing data from the second sensor 202, and outputs a second recognition result as a result of the recognition processing (S13).

Then, the second acquisition unit 130 acquires the second recognition result from the second recognition unit 112 (S14).

One process of steps S11 and S12 and steps S13 and S14 is executed regardless of whether the other process is being executed.

Next, the first determination unit 140 performs a first determination to determine the degree of similarity between the first recognition result and the second recognition result (S15).

The third acquisition unit 150 acquires the first correlation information and the second correlation information (S16). Note that step S16 may be performed before step S17, and is not limited to being performed after step S15.

The fusion control unit 160 controls fusion processing according to the result of the first determination and the first correlation information and the second correlation information (S17).

The second determination unit 170 performs a second determination to determine a first similarity and a second similarity, which are degrees of similarity between the third recognition result and the first recognition result and the second recognition result, respectively (S18).

The output unit 180 outputs at least one of the first recognition result, the second recognition result, and the third recognition result according to the result of the second determination (S19).

[1-3. effects, etc.]
According to the information processing system 10 according to the present embodiment, the fusion processing of the first recognition result and the second recognition result is controlled according to the similarity determination result of the first recognition result and the second recognition result, and , at least one of a first recognition result, a second recognition result, and a third recognition result obtained by fusion processing. Therefore, it is possible to reduce the output of recognition results with low accuracy.

Further, in the information processing system 10, the first recognition result and the second recognition result are object recognition results, and the degree of similarity is the degree of similarity regarding the position of the object. That is, the information processing system 10 outputs at least one of the first recognition result, the second recognition result, and the third recognition result according to the determination result of the similarity of the position of the recognized object. Therefore, it is possible to suppress the deterioration of the recognition result of the object due to the fusion of the recognition result.

Further, in the information processing system 10, the first recognition result and the second recognition result are object recognition results, and the similarity is the similarity of object attributes. That is, the information processing system 10 outputs at least one of the first recognition result, the second recognition result, and the third recognition result according to the determination result of the attribute similarity of the recognized object. Therefore, it is possible to suppress the deterioration of the recognition result of the object due to the recognition result.

Further, in the information processing system 10, the first recognition result and the second recognition result are object recognition results, and the similarity is the similarity of the number of objects. That is, the information processing system 10 outputs at least one of the first recognition result, the second recognition result, and the third recognition result according to the similarity determination result of the number of recognized objects. Therefore, deterioration of the object recognition result can be suppressed.

Further, the information processing system 10 acquires the first correlation information and the second correlation information, thereby performing fusion processing considering the first evaluation value of the first recognition result and the second evaluation value of the second recognition result. . In other words, the information processing system 10 controls the fusion process according to the similarity determination result of the first recognition result and the second recognition result, and the first correlation information and the second correlation information. can be obtained.

Information processing system 10 also controls the environment of the space where first sensor 201 and second sensor 202 are arranged during sensing, the states of first sensor 201 and second sensor 202 during sensing, and the first sensor 201 and second sensor 202 . Since at least one of the specifications of the second sensor 202 is acquired, the fusion processing can be easily performed in consideration of the first evaluation value and the second evaluation value.

Further, the information processing system 10 performs a second determination to determine a first similarity and a second similarity, which are degrees of similarity between the third recognition result and the first recognition result and the second recognition result, respectively. At least one of a first recognition result, a second recognition result, and a third recognition result is output according to the result of the second determination. Therefore, a good object recognition result can be obtained.

In addition, the information processing system 10 performs any one of performing fusion processing, not performing fusion processing, and adjusting parameters of fusion processing as control of fusion processing. For this reason, for example, when it is predicted that the recognition result will not improve even if the fusion process is performed according to the result of the first determination, the recognition result is It is possible to suppress the deterioration of the recognition result of the object due to the fusion.

The information processing system 10 also executes recognition processing based on sensing data from the first sensor 201 and sensing data from the second sensor 202, and outputs a first recognition result and a second recognition result. Therefore, the information processing system 10 can obtain the first recognition result and the second recognition result by acquiring the sensing data from the first sensor 201 and the sensing data from the second sensor 202 .

[1-4. Modification]
[1-4-1. Modification 1]
In the first embodiment described above, by acquiring the first correlation information and the second correlation information, the accuracy of the first recognition result based on the sensing data by the first sensor 201 and the second recognition result based on the sensing data by the second sensor 202 are obtained. Although the accuracy of the recognition result is estimated, the present invention is not limited to this. For example, the recognition unit 110 acquires a plurality of sensing data sensed at a plurality of different timings from each of the first sensor 201 and the second sensor 202, and performs recognition processing on each of the acquired plurality of sensing data. Therefore, a plurality of first recognition results are obtained, and a plurality of second recognition results are obtained. Therefore, if the difference obtained by comparing the current first recognition result and the first recognition result based on the sensing data at the previous timing is larger than a predetermined threshold, the recognition unit 110 detects the current first recognition result. It may be estimated that the recognition accuracy of one recognition result is low. Similarly, if the difference obtained by comparing the current second recognition result and the second recognition result based on the sensing data at the previous timing is larger than a predetermined threshold, the recognition unit 110 detects the current second recognition result. 2, it may be estimated that the recognition accuracy of the recognition result is low. In this case, the fusion control unit 160 obtains the accuracy of the first recognition result and the accuracy of the second recognition result from the recognition unit 110, and obtains the accuracy of the first recognition result and the accuracy of the second recognition result. Fusion processing may be controlled according to accuracy.

Further, for example, the recognition unit 110 may combine SLAM (Simultaneous Localization and Mapping) generated based on sensing data obtained from the second sensor 202 and ADAS (Advanced When the generated SLAM is significantly different in shape from the ADAS map, it may be estimated that the accuracy of the second recognition result is low.

[1-4-2. Modification 2]
In the information processing system 10 according to the first embodiment, the second determination unit 170 is executed after the fusion process, but the invention is not limited to this. For example, if the accuracy of one of the first recognition result and the second recognition result is estimated to be high according to the first correlation information and the second correlation information, the second determination unit 170 determines that the first recognition result and the second determination may be performed only when the similarity of the second recognition result is small.

(Embodiment 2)
Next, Embodiment 2 will be described.

FIG. 6 is a block diagram showing an example of a functional configuration of an information processing system according to Embodiment 2. As shown in FIG.

Information processing system 10A differs from information processing system 10 according to the first embodiment in that it does not include second determination section 170 and in the function of output section 180a. Since other configurations of the information processing system 10A are the same as those of the information processing system 10 according to the first embodiment, description thereof is omitted.

Output section 180a outputs at least one of a first recognition result, a second recognition result, and a third recognition result according to the first correlation information and the second correlation information. For example, when the accuracy of the first recognition result and the second recognition result is estimated to be high according to the first correlation information and the second correlation information, output section 180a outputs the first recognition result, the second recognition result, and the At least the third recognition result among the third recognition results may be output.

Further, output section 180a outputs the first recognition result and the second recognition result when one of the first recognition result and the second recognition result is estimated to be highly accurate according to the first correlation information and the second correlation information. If the similarity of the results is high, at least the third recognition result among the first recognition result, the second recognition result, and the third recognition result may be output. Further, output section 180a outputs the first recognition result and the second recognition result when one of the first recognition result and the second recognition result is estimated to be highly accurate according to the first correlation information and the second correlation information. If the similarity of the result is small, it is not necessary to output the third recognition result. Output section 180a outputs the first recognition result and the second recognition result when it is estimated that the accuracy of one of the first recognition result and the second recognition result is high according to the first correlation information and the second correlation information. When outputting at least the third recognition result out of the result and the third recognition result, outputting the recognition result estimated to be the most accurate of the first recognition result and the second recognition result together with the third recognition result. good too.

Further, output section 180a outputs the first recognition result and the second recognition result when it is estimated that the accuracy of one of the first recognition result and the second recognition result is low according to the first correlation information and the second correlation information. At least one of the results may be output.

FIG. 7 is a table showing whether or not the performance of the third recognition result is improved in a plurality of cases according to similarity and recognition accuracy. FIG. 7 shows whether the similarity between the first recognition result and the second recognition result is large or small, whether the recognition accuracy of the first recognition result is high or low, whether the recognition accuracy of the second recognition result is high or low, It is classified according to the situation. In the second embodiment, since the second determination is not performed, FIG. 7 shows whether the degree of similarity between the first recognition result and the third recognition result in FIG. It becomes a table that does not consider whether the degree is large or small.

Therefore, in the case of the table shown in FIG. 7, when the combination of recognition accuracies is "high/low", performance of the third recognition result is improved if "similarity is high", but if "similarity is low", If so, the performance of the third recognition result is degraded. The case where the combination of recognition accuracies is "high/high" and the case where the combination is "low/low" are the same as in the case of FIG. 4, so description thereof will be omitted.

FIG. 8 is a flow chart showing an example of an information processing method in the information processing system according to the second embodiment.

The information processing method according to the second embodiment differs from the information processing method according to the first embodiment in that step S18 is not performed and step S19a for outputting is different, so step S19a will be described.

After step S17 ends, the output unit 180a outputs at least one of the first recognition result, the second recognition result, and the third recognition result according to the first correlation information and the second correlation information (S19a).

Even if the second determination is not performed as in the second embodiment, the information processing system 10A outputs the recognition result according to the result of the first determination and the first correlation information and the second correlation information. to select. Specifically, when the accuracy of one of the first recognition result and the second recognition result is estimated to be high according to the first correlation information and the second correlation information, the information processing system 10A And when the degree of similarity of the second recognition result is small, the third recognition result is not output. Therefore, it is possible to suppress the deterioration of the recognition result of the object due to the fusion of the recognition result.

(Embodiment 3)
Next, Embodiment 3 will be described.

FIG. 9 is a block diagram showing an example of a functional configuration of an information processing system according to Embodiment 3. As shown in FIG.

The information processing system 10B differs from the information processing system 10 of the first embodiment in that the first determination unit 140 is not provided and the functions of the fusion control unit 160b and the output unit 180b. Since other configurations of the information processing system 10B are the same as those of the information processing system 10 according to the first embodiment, description thereof is omitted.

The fusion control unit 160b acquires the first recognition result and the second recognition result from the recognition unit 110, and performs fusion processing on the acquired first recognition result and the second recognition result. In other words, the fusion control unit 160b according to the second embodiment uses the first recognition result and the second recognition result without considering the result of the first determination, as compared with the fusion control unit 160 according to the first embodiment. Fusion process.

If the accuracy of the first recognition result and the second recognition result is estimated to be high according to the first correlation information and the second correlation information, output section 180b outputs the first recognition result, the second recognition result, and the third recognition result. Of the recognition results, at least the third recognition result may be output.

Further, output section 180b outputs the third recognition result and the first recognition result when it is estimated that one of the first recognition result and the second recognition result is highly accurate according to the first correlation information and the second correlation information. At least the third recognition result among the first recognition result, the second recognition result, and the third recognition result may be output when the degree of similarity with each of the result and the second recognition result is high. On the other hand, when one of the first recognition result and the second recognition result is estimated to be highly accurate according to the first correlation information and the second correlation information, output section 180b outputs the third recognition result and the first recognition result. If the degree of similarity between the recognition result and the second recognition result is small, the third recognition result may not be output. Output section 180b outputs the first recognition result and the second recognition result when it is estimated that the accuracy of one of the first recognition result and the second recognition result is high according to the first correlation information and the second correlation information. When outputting at least the third recognition result out of the result and the third recognition result, outputting the recognition result estimated to be the most accurate of the first recognition result and the second recognition result together with the third recognition result. good too.

Further, output section 180b outputs the first recognition result and the second recognition result when it is estimated that the accuracy of one of the first recognition result and the second recognition result is low according to the first correlation information and the second correlation information. At least one of the results may be output.

FIG. 10 is a table showing whether or not the performance of the third recognition result is improved in a plurality of cases according to similarity and recognition accuracy. FIG. 10 shows whether both the degrees of similarity between the first recognition result and the second recognition result and the third recognition result are large, whether the recognition accuracy of the first recognition result is high or low, 2. Cases are classified according to whether the recognition accuracy of the recognition result is high or low. Since the first determination is not performed in the third embodiment, FIG. 10 is a table that does not consider whether the similarity between the first recognition result and the second recognition result in FIG. 4 is large or small.

Therefore, in the case of the table shown in FIG. 10, when the combination of recognition accuracy is "high/low", the performance of the third recognition result is improved if "similarity to third recognition result is large". , "similarity to third recognition result is small", the performance of the third recognition result is degraded. The case where the combination of recognition accuracies is "high/high" and the case where the combination is "low/low" are the same as in the case of FIG. 4, so description thereof will be omitted.

11 is a flowchart illustrating an example of an information processing method in an information processing system according to Embodiment 3. FIG.

The information processing method according to the third embodiment differs from the information processing method according to the first embodiment in that step S15 is not performed and the fusion processing in step S17b is different, so step S17b will be described.

When step S16 ends, the fusion control unit 160b performs fusion processing on the first recognition result and the second recognition result (S17b).

Then, step S18 is performed, after which the output unit 180b outputs at least one of the first recognition result, the second recognition result, and the third recognition result according to the result of the second determination (S19b).

Even if the first determination is not performed as in the third embodiment, the information processing system 10B outputs the recognition result according to the result of the second determination and the first correlation information and the second correlation information. to select. Specifically, when the accuracy of one of the first recognition result and the second recognition result is estimated to be high according to the first correlation information and the second correlation information, the information processing system 10B and the first recognition result and the second recognition result, respectively, the third recognition result is not output. Therefore, it is possible to suppress the deterioration of the recognition result of the object due to the fusion of the recognition result.

(Other embodiments)
The information processing systems 10, 10A, and 10B described in Embodiments 1 to 3 above are information processing systems mounted on the vehicle 200. However, they are not limited to being mounted on the vehicle 200. It can be applied to any information processing system configured to perform fusion processing of two or more types of recognition results based on the sensing data.

Further, although the information processing systems 10, 10A, and 10B described in the first to third embodiments output recognition results of recognition processing from sensing data, the present invention is not limited to this. For example, a third recognition result is output, that is, a case in which the third recognition result determined to improve performance can be obtained, and the corresponding sensor data, and a determination that performance decreases A case in which the third recognition result can be obtained and the sensor data at that time are associated with each other may be used as the learning data.

Further, for example, when the third recognition result is output, that is, when the third recognition result determined to improve the performance can be obtained, the third recognition result may be used to add an annotation. . Annotation means, for example, if it is a recognition result in an image, it is to associate a graphic such as a frame indicating the size or position of the object in the image with each type of object or with a specific type of object. be.

In each of the above-described embodiments, each component may be configured by dedicated hardware, or realized by executing a software program suitable for each component. Each component may be realized by reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory by a program execution unit such as a CPU or processor. Here, the software that implements the information processing method and the like of each of the above embodiments is the following program.

That is, the program causes the computer to acquire a first recognition result, which is an object recognition result based on sensing data from a first sensor, and recognizes an object based on sensing data from a second sensor different from the first sensor. Acquiring a second recognition result that is a result, performing a first determination to determine the degree of similarity between the first recognition result and the second recognition result, and performing the first recognition result and the second recognition result according to the result of the first determination controlling fusion processing of the second recognition result, and fusion of the first recognition result, the second recognition result, and the first recognition result and the second recognition result according to the result of the first determination; an information processing method for outputting at least one of the third recognition result and the third recognition result.

Further, the program causes the computer to obtain a first recognition result, which is an object recognition result based on sensing data from a first sensor, and recognizes an object based on sensing data from a second sensor different from the first sensor. obtaining a second recognition result that is a result, fusing the first recognition result and the second recognition result; performing a second determination for determining a first degree of similarity and a second degree of similarity, which are degrees of similarity between the recognition result and the second recognition result, respectively; An information processing method for outputting at least one of the second recognition result and the third recognition result may be executed.

The information processing system and information processing method according to one or more aspects of the present invention have been described above based on the embodiments, but the present invention is not limited to these embodiments. As long as it does not depart from the gist of the present invention, one or more modifications of the present embodiment that can be considered by those skilled in the art, or a form constructed by combining the components of different embodiments may be included within the scope of the embodiments.

INDUSTRIAL APPLICABILITY The present disclosure is useful as an information processing system, an information processing method, and the like capable of suppressing deterioration of object recognition results due to fusion of recognition results.

10, 10A, 10B Information processing system 100 Information processing device 101 Processor 102 Main memory 103 Storage 104 Communication IF
110 recognition unit 111 first recognition unit 112 second recognition unit 120 first acquisition unit 130 second acquisition unit 140 first determination unit 150 third acquisition units 160, 160b fusion control unit 170 second determination units 180, 180a, 180b output Part 200 Vehicle 201 First sensor 202 Second sensor 203 Third sensor 300 Operation control device

Claims (13)

a first acquisition unit that acquires a first recognition result that is an object recognition result based on sensing data from a first sensor;
a second acquisition unit that acquires a second recognition result that is an object recognition result based on sensing data from a second sensor different from the first sensor;
a first determination unit that performs a first determination that determines the degree of similarity between the first recognition result and the second recognition result;
a fusion control unit that controls fusion processing of the first recognition result and the second recognition result according to the result of the first determination;
A first degree of similarity and a second degree of similarity, which are degrees of similarity between a third recognition result obtained by fusion processing the first recognition result and the second recognition result, and the first recognition result and the second recognition result, respectively. A second determination unit that performs a second determination that determines
Said No.2According to the determination result, the first recognition result, the second recognition result, the previousdiaryan output unit that outputs at least one of 3 recognition results;
An information processing system comprising the first recognition result and the second recognition result are recognition results of the position of an object;
The information processing system according to claim 1, wherein the degree of similarity is a degree of similarity regarding the position of the object. The first recognition result and the second recognition result are object attribute recognition results,
The information processing system according to claim 1 or 2, wherein the degree of similarity is the degree of similarity of attributes of the objects. the first recognition result and the second recognition result are recognition results of existence of an object;
The information processing system according to any one of claims 1 to 3, wherein the degree of similarity is a degree of similarity in number of the objects. moreover,
a third obtaining unit that obtains first correlation information correlated with the first evaluation value of the first recognition result and second correlation information correlated with the second evaluation value of the second recognition result;
5. The fusion control unit according to any one of claims 1 to 4, wherein the fusion control unit controls fusion processing according to the result of the first determination and the acquired first correlation information and second correlation information. Information processing system as described. Each of the first correlation information and the second correlation information is the environment of the space in which the first sensor and the second sensor are arranged at the time of sensing, and the state of the first sensor and the second sensor at the time of sensing. , and specifications of the first sensor and the second sensor. The information processing system according to any one of claims 1 to 6 , wherein the fusion control unit controls the fusion processing depending on whether or not to perform the fusion processing on the first recognition result and the second recognition result. The fusion control unit determines whether to perform fusion processing on the first recognition result and the second recognition result depending on whether the first recognition result or the second recognition result is input to a fusion processing unit that performs fusion processing. The information processing system according to claim 7 , which controls the . The information processing system according to any one of claims 1 to 6 , wherein the fusion control unit controls the fusion processing by adjusting a parameter for fusion processing of the first recognition result and the second recognition result. . a first acquisition unit that acquires a first recognition result that is an object recognition result based on sensing data from a first sensor;
a second acquisition unit that acquires a second recognition result that is an object recognition result based on sensing data from a second sensor different from the first sensor;
a fusion processing unit that performs fusion processing on the first recognition result and the second recognition result;
A first degree of similarity and a second degree of similarity, which are degrees of similarity between a third recognition result obtained by fusion processing the first recognition result and the second recognition result, and the first recognition result and the second recognition result, respectively. A second determination unit that performs a second determination that determines
an output unit that outputs at least one of the first recognition result, the second recognition result, and the third recognition result according to the result of the second determination;
An information processing system comprising moreover,
A recognition unit that executes object recognition processing based on sensing data from the first sensor and sensing data from the second sensor, and outputs the first recognition result and the second recognition result. 8. The information processing system according to any one of 1 to 7 . Acquiring a first recognition result, which is an object recognition result based on sensing data from the first sensor,
Acquiring a second recognition result, which is an object recognition result based on sensing data from a second sensor different from the first sensor,
perform a first determination to determine the degree of similarity between the first recognition result and the second recognition result;
controlling fusion processing of the first recognition result and the second recognition result according to the result of the first determination;
A first degree of similarity and a second degree of similarity, which are degrees of similarity between a third recognition result obtained by fusion processing the first recognition result and the second recognition result, and the first recognition result and the second recognition result, respectively. A second judgment is made to judge the
Said No.2According to the determination result, the first recognition result, the second recognition result, the previousdiaryOutput at least one of 3 recognition results
Information processing methods. Acquiring a first recognition result, which is an object recognition result based on sensing data from the first sensor,
Acquiring a second recognition result, which is an object recognition result based on sensing data from a second sensor different from the first sensor,
fusing the first recognition result and the second recognition result;
A first similarity and a second similarity, which are degrees of similarity between a third recognition result obtained by fusion of the first recognition result and the second recognition result, and the first recognition result and the second recognition result, respectively, are calculated. Make a second judgment to judge,
An information processing method, wherein at least one of the first recognition result, the second recognition result, and the third recognition result is output according to the result of the second determination.

Images