WO2022074700A1 - 情報処理装置、情報処理システム、情報処理方法 - Google Patents

情報処理装置、情報処理システム、情報処理方法 Download PDF

Info

Publication number
WO2022074700A1
WO2022074700A1 PCT/JP2020/037672 JP2020037672W WO2022074700A1 WO 2022074700 A1 WO2022074700 A1 WO 2022074700A1 JP 2020037672 W JP2020037672 W JP 2020037672W WO 2022074700 A1 WO2022074700 A1 WO 2022074700A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
margin
target
area
information processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/037672
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
勇人 逸身
悠介 篠原
浩一 二瓶
孝法 岩井
フロリアン バイエ
チャルヴィ ヴィタル
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to JP2022554976A priority Critical patent/JP7533601B2/ja
Priority to PCT/JP2020/037672 priority patent/WO2022074700A1/ja
Priority to US18/028,486 priority patent/US12283110B2/en
Publication of WO2022074700A1 publication Critical patent/WO2022074700A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30168—Image quality inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Definitions

  • This disclosure relates to information processing devices, information processing systems, and information processing methods.
  • a method of providing necessary information is expected by improving the image quality of only the camera images and areas that are important for driving.
  • Patent Document 1 discloses an in-vehicle image processing device that performs image processing on an image signal output from an image pickup device that photographs the rear side of a vehicle. Further, in the same document, in order to ensure visibility to the object of the driver, the width of the margin M1 is wide with respect to the rectangular region R1 set for the object close to the vehicle, and the object is long away. It is also disclosed that the width of the margin M2 is set narrower with respect to the set rectangular region R2.
  • Patent Document 1 an appropriate margin is not set in consideration of subsequent image processing. Therefore, after performing image processing on the target area including the margin as shown in Patent Document 1 so as to have higher image quality than other areas, the remote monitoring device performs image processing on the image processed. When recognition is performed, the recognition accuracy of the target may decrease.
  • the present invention has been made to solve such a problem, and an object of the present invention is to provide an information processing apparatus or the like that can maintain recognition accuracy even when subsequent image processing is performed.
  • the information processing apparatus includes an image acquisition unit that acquires an image taken by an image pickup unit mounted on a vehicle, and an image acquisition unit.
  • a target detection unit that detects a target area including a target in the acquired image, and a target detection unit.
  • a classification identification unit that specifies the type of the detected target and the classification including the size of the detected target area,
  • An area determination unit that determines an image processing area as an area in which a margin corresponding to the specified classification is added to the target area. To prepare for.
  • the information processing system is An image acquisition unit that acquires images taken by the image pickup unit mounted on the vehicle, and an image acquisition unit.
  • a target detection unit that detects a target area including a target in the acquired image, and a target detection unit.
  • a classification identification unit that specifies the type of the detected target and the classification including the size of the detected target area,
  • An area determination unit that determines an image processing area as an area in which a margin corresponding to the specified classification is added to the target area. To prepare for.
  • the information processing method is The image taken by the image pickup unit mounted on the vehicle is acquired and The target area including the target in the acquired image is detected, and the target area is detected. Identify the type of the detected target and the classification including the size of the detected target area. An area in which a margin corresponding to the specified classification is added to the target area is determined as an image processing area.
  • FIG. It is a block diagram which shows the structure of the information processing apparatus which concerns on Embodiment 1.
  • FIG. It is a flowchart which shows the information processing method which concerns on Embodiment 1.
  • It is a schematic diagram explaining the outline of a remote monitoring operation system.
  • It is a figure which shows an example of the image taken by the in-vehicle camera of a vehicle.
  • It is a flowchart which shows the optimum margin determination method.
  • It is a block diagram which shows the structure of the image processing apparatus which concerns on Embodiment 2.
  • FIG. 1 is a block diagram showing a configuration of an information processing apparatus according to the first embodiment.
  • the information processing device 100 is, for example, a computer mounted on a vehicle.
  • the information processing apparatus 100 includes an image acquisition unit 101 that acquires an image captured by an image pickup unit mounted on a vehicle, a target detection unit 111 that detects a target area including an object in the acquired image, and the detection unit.
  • Image processing is performed on the classification specifying unit 112 that specifies the type of the target and the classification including the size of the detected target area, and the area in which the margin corresponding to the specified classification is added to the target area.
  • An area determination unit 110 for determining an area is provided.
  • FIG. 2 is a flowchart showing the image processing method according to the first embodiment.
  • the information processing method includes the following steps. That is, an image taken by an image pickup unit mounted on the vehicle is acquired (step S101), a target area including an object in the acquired image is detected (step S102), and the type of the detected object and the type of the detected object are determined. A classification including the size of the detected target region is specified (step S103), and a region to which a margin corresponding to the specified classification is added to the target region is determined as an image processing region (step S104). ..
  • FIG. 3 is a schematic diagram illustrating an outline of the remote monitoring operation system.
  • the remote monitoring driving system remotely controls the vehicle 5 that does not require a driver from the remote monitoring center.
  • images taken by a plurality of in-vehicle cameras 10A to 10D mounted on the vehicle 5 are remotely monitored and controlled by a remote monitoring control device 400 (hereinafter, simply referred to as a method) via a wireless communication network and the Internet. It may be described as a remote monitoring device).
  • the image processing device 200 mounted on the vehicle is used to perform predetermined image processing on the image from the in-vehicle camera and transmit the image processed image to the remote monitoring control device 800 via the network.
  • the remote monitoring control device 800 displays the received image on a display unit such as a monitor, and the remote driver 3 remotely controls the vehicle 5 while viewing the received image on the monitor.
  • the remote monitoring and control device 400 may display information for the remote driver 3 to remotely control the vehicle 5 in addition to the received video.
  • the remote monitoring control device 800 may display the received image and the analysis result to the remote driver 3.
  • the remote driving control device mounted on the vehicle vehicle 5 performs bidirectional communication with the remote monitoring control device 400 by using a communication method (for example, LTE, 5G) using a mobile phone network.
  • the image recognition unit 410 of the remote monitoring control device 400 can analyze the received video or image and detect and recognize the target by using the image recognition engine.
  • the remote-controlled driving system may be switched to remote control or automatic control when a vehicle under remote monitoring is running and a danger of the vehicle is detected. That is, the vehicle driven by a person may be temporarily switched to such control, or the vehicle may have a driver.
  • the in-vehicle camera 10A photographs the front of the vehicle
  • the in-vehicle camera 10B photographs the rear of the vehicle
  • the in-vehicle camera 10C photographs the right side of the vehicle
  • the in-vehicle camera 10D photographs the left side of the vehicle.
  • the number of in-vehicle cameras is not limited to this, and may be 5 or more. Further, the performance of each camera is basically the same, but may be slightly different.
  • a normal driver such as a taxi is required to have a second-class license, which requires that he / she can recognize an object (also called an object) within a range that a person with a visual acuity of 0.8 or more can see.
  • the image provided to the remote driver can also recognize an object within the range that a person with a visual acuity of 0.8 or more can see (for example, in the case of a road sign on a general road, the driver signs at a distance of 10.66 m). Can be recognized).
  • the remote driver needs to visually recognize not only the object but also the peripheral information of the object, and such peripheral information can be transmitted to the remote driver as a relatively high-quality image.
  • FIG. 4 is a diagram showing an example of an image taken by an in-vehicle camera of a vehicle.
  • FIG. 4 shows an example of an image taken in front of the vehicle 5 by the vehicle-mounted camera 10A shown in FIG.
  • the detected object is surrounded by a bounding box (this area is also called ROI (Region of Interest)).
  • ROI this area
  • those that can affect the driving of the vehicle that is, other vehicles, pedestrians, bicycles, traffic lights, traffic signs, and the like are detected.
  • the image analysis accuracy and recognition accuracy at the remote monitoring center can be maintained while suppressing the bandwidth used. Can be done.
  • a tight detection area may be set for the target.
  • the margin of the detection area surrounding the target here, the vehicle
  • different image processing for example, encoding, various compression processes, etc.
  • the image recognition accuracy at the remote monitoring center may decrease.
  • information on the area around the subject is also important in the video analysis of the subject. For example, in order to recognize a vehicle, information on surrounding areas such as roads is also considered to be important.
  • an area in which an optimum margin is set (transmission in FIG. 4). ROI) needs to be determined.
  • the optimum margin is derived according to the type of object (for example, vehicle, person, bicycle) and the class of the size of the object occupied in the image (for example, large, medium, small).
  • FIG. 5 is a flowchart showing a method for determining the optimum margin.
  • the teacher image data is clustered for each type of target and a class of the size of the target occupied in the image (step S201).
  • the size of the object may be indicated by the ratio of the area of the object (detection area) to the screen, or it is indicated by the area because the angle of view of the in-vehicle camera that captured the teacher image data is fixed. May be good.
  • This process is also called clustering of teacher image data.
  • the size of the target is classified into 3 classes (large, medium and small) by K-means or the like.
  • the target class is the one closest to the large, medium, and small reference points (average size in each class).
  • the teacher image data is duplicated, and in each duplicated image data, the ROI for which various margins are set is set in the area where the image quality should be improved, and the other areas are set in the area where the image quality should be lowered (step S202). ..
  • image data (a plurality of ROI image data) is subjected to different image processing for each set area (step S203).
  • the different image processes are high image quality processing and low image quality processing.
  • the image quality reduction process may include, for example, a contrast reduction process, a resolution reduction process, a gradation number reduction process, a color number reduction process, or a dynamic range reduction process.
  • the high image quality processing may also include a contrast reduction processing, a low resolution processing, a gradation number reduction processing, a color number reduction processing, or a dynamic range reduction processing, but various types such as higher image quality than the low image quality processing.
  • Image processing After that, video analysis is performed on the image-processed ROI image data to evaluate the recognition accuracy of the target. In this recognition accuracy evaluation, an image recognition engine that is the same as or similar to the image recognition engine used in the server computer located in the remote monitoring center can be used. A margin whose recognition accuracy is equal to or greater than a threshold value is determined as an optimum margin (step S204). The optimum margin is stored in the storage unit (storage unit 250 in FIG. 8) of each image processing device together with the above-mentioned classification (type of target and size of the target in the image) (step S205).
  • FIG. 6 is a diagram illustrating an accuracy result of image recognition of certain teacher image data.
  • the teacher image data IMG one target (for example, the type is "car” and the size of the target is "small") is detected (that is, the correct answer is labeled) before the image processing.
  • This teacher image data IMG is duplicated, and the duplicated teacher image data IMG1, IMG2, and IMG3 have different margins (for example, 5% for IMG1, 10% for IMG2, and 15% for IMG3).
  • Image data is image-processed (encoded) so as to improve the image quality of the area including the margin in the image and lower the image quality of the other areas.
  • Image recognition is performed for each encoded image. In this example, IMG2 with a margin of 10% was able to recognize images properly, but IMG1 with a margin of 5% and IMG3 with a margin of 15% were able to recognize images properly. It shall be.
  • FIG. 7 shows the recognition accuracy of the target for the ROI margin for each class of the target type and the target size occupying the image.
  • the horizontal axis shows the ROI margin (%), and the vertical axis shows the recognition accuracy of the object.
  • the margin at which the recognition accuracy peaks is set as the optimum margin.
  • the classification that is, the type of the object and the size of the object occupied in the image are "car / small”
  • the optimum margin is 20%.
  • the optimum margin is 10%
  • in the case of "car / large it is 5%.
  • the optimum margin is 10%.
  • the optimum margin tends to become narrower as the size on the screen increases.
  • the size of the object occupied in the image may be evaluated by subdividing it into not only the three classes of "large, medium, and small” but also more classes.
  • the optimum margin is assumed to have the maximum accuracy (peak value), but is not limited to this, and may be set as the optimum margin as long as the accuracy is equal to or higher than the threshold value.
  • the threshold value of the recognition accuracy can be arbitrarily set in consideration of the suppression of the band used in the image transmission system and the required recognition accuracy.
  • the classification is "target type” and "target size class in the image", but other classifications may be added.
  • driving environment For example, “driving environment”, “target orientation”, “time zone”, and “weather” may be added.
  • Examples of the driving environment include highways and residential areas.
  • Examples of the time zone include daytime, evening, and nighttime.
  • Examples of the weather include sunny, cloudy, rainy, and snowy.
  • FIG. 8 is a block diagram showing the configuration of the image processing apparatus according to the second embodiment.
  • the image processing device 200 is an information processing device configured by a computer.
  • the image processing device 200 includes an image acquisition unit 201, an ROI determination unit 210, an optimum margin storage unit 250, an encoder 220, and a communication unit 230.
  • the ROI determination unit 210 includes a target detection unit 211, a classification identification unit 212, and an optimum margin acquisition unit 213.
  • the image processing device 200 may have a margin setting unit 260.
  • the image processing device 200 may be provided with a margin setting unit realized by another computer.
  • the image acquisition unit 201 acquires an image (frame) taken by an image pickup unit such as an in-vehicle camera.
  • the ROI determination unit 210 detects an object of the acquired image and determines an ROI that is an appropriate region from the viewpoint of image recognition.
  • the target detection unit 211 detects the target in the image from the image acquisition unit 201.
  • the target to be detected can be arbitrarily set in advance.
  • an object for example, a person, a vehicle, a motorcycle, a bicycle, a truck, a bus, etc.
  • the target detection unit 211 can also identify the type of target (for example, a person, a vehicle, a bicycle, a motorcycle, etc.) by using a known image recognition technique.
  • the image acquisition unit 201 can continuously acquire the image captured by the image pickup unit as an image frame at a predetermined frame rate.
  • the classification specifying unit 212 specifies the classification including the type of the detected target and the size of the detected target area.
  • the size of the target area is calculated from the area of the bounding box, and a class of size corresponding to the calculated area (for example, "large”, “medium”, and "small”) is specified.
  • the optimum margin acquisition unit 213 acquires the optimum margin corresponding to the specified classification from the optimum margin storage unit 250.
  • the optimum margin storage unit 250 stores the optimum margin for each category evaluated in advance. As a result, low delay can be realized and the optimum margin can be obtained.
  • the optimum margin storage unit 250 may be inside the image processing device 200, or may be located in an external storage device connected to the image processing device 200 via a network.
  • the ROI determination unit 210 can determine an appropriate ROI by adding an optimum margin corresponding to each classification to the target area.
  • the ROI determined in this way is appropriately set so that the recognition accuracy of the target can be maintained at a certain level or higher even when the subsequent image processing (encoding) is performed.
  • the encoder 220 performs image processing so as to improve the image quality of the ROI and lower the image quality of other areas in the image.
  • the compression processing is performed at a lower compression rate than in the low image quality region.
  • the image quality reduction process may include a contrast reduction process, a resolution reduction process, a gradation number reduction process, a color number reduction process, or a dynamic range reduction process.
  • the high image quality processing may also include a contrast reduction processing, a low resolution processing, a gradation number reduction processing, a color number reduction processing, or a dynamic range reduction processing, but various types such as higher image quality than the low image quality processing.
  • Image processing may include a contrast reduction process, a resolution reduction process, a gradation number reduction processing, a color number reduction processing, or a dynamic range reduction processing, but various types such as higher image quality than the low image quality processing.
  • the communication unit 230 is a communication interface with a network.
  • the communication unit 230 is used to communicate with another network node device (for example, an information processing device on the remote monitoring center side) constituting the image processing system.
  • the communication unit 230 may be used for wireless communication.
  • the communication unit 230 may be used to perform wireless LAN communication specified in the IEEE 802.11 series, or mobile communication specified in 3GPP (3rd Generation Partnership Project), 4G, 5G, and the like.
  • the communication unit 230 can also be communicably connected to the smartphone via Bluetooth (registered trademark) or the like.
  • the communication unit 230 can be connected to the camera via a network.
  • the communication unit 230 wirelessly transmits the image processed image data to the remote monitoring center.
  • the communication unit 230 wirelessly transmits the encoded image data to the remote monitoring control device via a mobile network such as LTE or 5G.
  • the margin setting unit 260 specifies the accuracy with which the remote monitoring device for monitoring the vehicle recognizes the object according to the type of the detected object and the size of the detected target area, and according to the specified accuracy. , Set the margin corresponding to the type of the detected target and the size of the target area mentioned above. The margin setting unit 260 sets the optimum margin shown in FIG.
  • the margin setting unit 260 collects teacher image data whose target and type have been determined by the target detection unit 211 for the image from the image acquisition unit 201. Next, the margin setting unit 260 clusters the teacher image data according to the type of the target and the classification including the size in the image to be detected. The margin setting unit 260 sets a region in which various margins are added to the region including the target in the teacher image data as the ROI. For example, as shown in FIG. 6, a plurality of margin areas that gradually increase can be set for the detection area. The margin setting unit 260 duplicates the original image data for each of the various ROIs, and performs image processing on each duplicated image data so that the ROI has a high image quality and the other areas have a low image quality.
  • the margin setting unit 260 recognizes the image data after image processing by using the image recognition engine.
  • the margin setting unit 260 evaluates the recognition accuracy for each margin based on the image recognition results for a large number of image data.
  • the margin setting unit 260 sets a margin whose recognition accuracy is equal to or higher than the threshold value as an optimum margin.
  • the margin setting unit 260 stores the set optimum margin together with the classification in the margin storage unit 250.
  • the margin setting unit may be provided inside the image processing device 200, or may be provided as an information processing device realized by another computer.
  • the margin setting device 300 may be realized by a computer different from the image processing device 200.
  • the image recognition engine used by the image recognition unit 410 of the remote monitoring device is installed in the margin setting device 300.
  • the optimum margin set for each classification by the margin setting device 300 is stored in the optimum margin storage units 250a and 250b of the image processing devices 200a and 200b mounted on the vehicles of the same vehicle type.
  • the vehicle of the same vehicle type as used herein means a vehicle of the same shape, a vehicle of the same size, a vehicle of a similar size, or a vehicle of a similar size, in which the angle of view from the imaging unit is substantially the same.
  • FIG. 10 is a flowchart showing the image processing method according to the second embodiment.
  • the operation of the ROI determination unit 210 is shown.
  • Acquire an image step S301).
  • the target in the image is detected (step S302).
  • the classification is specified from the type of the detection target and the size of the target in the image (step S303).
  • the optimum margin corresponding to the specified classification is acquired from the storage unit 250 (step S304).
  • the area to be detected is expanded by the optimum margin, and the ROI size is determined (step S305). Notify the encoder 220 of the ROI (step S306).
  • the encoder 220 encodes the image data so as to improve the image quality of the ROI and lower the image quality of other areas. Further, the communication unit 230 wirelessly transmits the encoded image data to the remote monitoring device 400.
  • the image processing apparatus can acquire the optimum margin for each of the detected target classifications and determine an appropriate ROI.
  • determining the optimum margin for each classification in advance it is possible to reduce the delay and maintain the recognition accuracy of the target in automatic driving or the like.
  • FIG. 11 is a block diagram showing a configuration of an information processing system according to another embodiment.
  • the information processing system includes a vehicle 5 equipped with the information processing device 100 shown in FIG. 1 and a remote monitoring control device 400 that is communicably connected to the information processing device 100 via a network.
  • the information processing apparatus 100 includes an image acquisition unit 101 that acquires an image captured by an image pickup unit 10 mounted on the vehicle 5, an object detection unit 111 that detects an object region including an object in the acquired image, and an object detection unit 111.
  • An image of the classification specifying unit 112 that specifies the type of the detected target and the classification including the size of the detected target area, and the area in which the margin corresponding to the specified classification is added to the target area.
  • An area determination unit 110 for determining a processing area is provided.
  • the information processing apparatus 100 further includes an image processing unit (encoder) 220 that performs image processing based on a determined area, and a communication unit that transmits image processed image data to a remote monitoring device 400. 230 can be included.
  • image processing unit encoder
  • 230 can be included.
  • the information processing device 100 including the image acquisition unit 101, the target detection unit 111, the classification identification unit 112, and the area determination unit 110 is described, but it may be mounted as a system mounted on different devices. good.
  • a device including the image acquisition unit 101 and the target detection unit 111, and a device including the classification identification unit 112 and the area determination unit 110 may be configured to communicate with each other via a network.
  • the information processing system can add a margin corresponding to the classification of the detected target and determine an appropriate area for image processing.
  • the information processing system performs image processing on an appropriate area that may affect the driving of the vehicle while suppressing the band used, and transmits the image data after image processing to the remote monitoring and control device to remotely control the vehicle. Monitoring and remote control can be realized.
  • the remote driver 3 remotely operates the unmanned driving vehicle 5, but the present invention is not limited to this.
  • a integrated control device that controls the unmanned driving vehicle 5 may be provided.
  • the integrated control device may generate information for autonomous driving of the unmanned driving vehicle 5 based on the information acquired from the unmanned driving vehicle 5, and the unmanned driving vehicle 5 may operate according to the information.
  • FIG. 12 is a block diagram showing a hardware configuration example of an information processing device 100, an image processing device 200, a margin setting device 300, and a remote monitoring control device 400 (hereinafter referred to as an information processing device 100 or the like).
  • the information processing apparatus 100 and the like include a network interface 1201, a processor 1202, and a memory 1203.
  • the network interface 1201 is used to communicate with other network node devices constituting the communication system.
  • Network interface 1201 may be used to perform wireless communication.
  • the network interface 1201 may be used to perform wireless LAN communication specified in the IEEE 802.11 series or mobile communication specified in 3GPP (3rd Generation Partnership Project).
  • the network interface 1201 may include, for example, a network interface card (NIC) compliant with the IEEE802.3 series.
  • NIC network interface card
  • the processor 1202 reads software (computer program) from the memory 1203 and executes it to perform processing of the information processing apparatus 100 or the like described by using the flowchart or the sequence in the above-described embodiment.
  • the processor 1202 may be, for example, a microprocessor, an MPU (MicroProcessingUnit), or a CPU (CentralProcessingUnit).
  • Processor 1202 may include a plurality of processors.
  • the memory 1203 is composed of a combination of a volatile memory (RAM (RandomAccessMemory)) and a non-volatile memory (ROM (ReadOnlyMemory)).
  • Memory 1203 may include storage located away from processor 1202.
  • processor 1202 may access memory 1203 via an I / O interface (not shown).
  • the memory 1203 does not necessarily have to be a part of the device, and may be an external storage device or a cloud storage connected to the computer device 500 via a network.
  • the memory 1203 is used to store the software module group.
  • the processor 1202 can perform the processing of the information processing apparatus 100 and the like described in the above-described embodiment.
  • each of the processors included in the information processing apparatus 100 and the like executes one or a plurality of programs including a set of instructions for causing a computer to perform the algorithm described with reference to the drawings.
  • each process described with reference to the above flowchart does not necessarily have to be processed in chronological order in the order described as the flowchart, and is a process executed in parallel or individually (for example, a parallel process or an object). Processing by) is also included. Further, the program may be processed by one CPU or may be distributed processed by a plurality of CPUs.
  • Non-temporary computer-readable media include various types of tangible storage mediums.
  • Examples of non-temporary computer-readable media include magnetic recording media, magneto-optical recording media (eg, magneto-optical disks), CD-ROMs (Read OnlyMemory), CD-Rs, CD-R / Ws, and semiconductor memories.
  • the magnetic recording medium may be, for example, a flexible disk, a magnetic tape, or a hard disk drive.
  • the semiconductor memory may be, for example, a mask ROM, a PROM (Programmable ROM), an EPROM (Erasable PROM), a flash ROM, or a RAM (Random Access Memory).
  • the program may also be supplied to the computer by various types of transient computer readable medium.
  • Examples of temporary computer readable media include electrical, optical, and electromagnetic waves.
  • the temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.
  • (Appendix 1) An image acquisition unit that acquires images taken by the image pickup unit mounted on the vehicle, and an image acquisition unit.
  • a target detection unit that detects a target area including a target in the acquired image, and a target detection unit.
  • a classification identification unit that specifies the type of the detected target and the classification including the size of the detected target area,
  • An area determination unit that determines an image processing area as an area in which a margin corresponding to the specified classification is added to the target area.
  • An information processing device equipped with An information processing device equipped with.
  • the margin is set so that the recognition accuracy of the image becomes equal to or higher than the threshold value even if image processing is performed on the image to which the region to which the margin corresponding to the specified classification is added is set.
  • the information processing apparatus according to Appendix 1. (Appendix 3) The information processing apparatus according to Appendix 1 or 2, wherein the area determination unit determines an area in which the margin is added to the target area as an image processing area having higher image quality than other areas.
  • the accuracy with which the remote monitoring device for monitoring the vehicle recognizes the target is specified according to the type of the detected target and the size of the detected target area.
  • Information processing device described in. (Appendix 5)
  • the margin setting unit is The teacher image data for which the type of the target to be included is known is classified according to the type of the known target and the size of the known target. An area with a margin added to the target area including the target included in the teacher image data is set.
  • the teacher image data is image-processed so that the region to which the margin is added has a higher image quality than the other regions, the accuracy with which the remote monitoring device recognizes the target is specified.
  • the information processing apparatus wherein the margin at which the recognition accuracy is equal to or higher than the threshold value is set as the margin corresponding to the classification.
  • Appendix 6 Areas with different margins are set for each of the plurality of teacher image data. When each of the plurality of teacher image data to which the different margins are added is image-processed, the accuracy with which the remote monitoring device recognizes the target is specified.
  • the information processing apparatus according to Appendix 5, wherein a margin at which the recognition accuracy is equal to or higher than a threshold value is set as a margin corresponding to the classification.
  • Appendix 7 The information processing apparatus according to any one of Supplementary note 1 to 5, which includes a storage unit that stores a margin corresponding to the classification.
  • An image acquisition unit that acquires images taken by the image pickup unit mounted on the vehicle, and an image acquisition unit.
  • a target detection unit that detects a target area including a target in the acquired image, and a target detection unit.
  • a classification identification unit that specifies the type of the detected target and the classification including the size of the detected target area,
  • An area determination unit that determines an image processing area as an area in which a margin corresponding to the specified classification is added to the target area.
  • An information processing system equipped with. (Appendix 9) The margin is set so that the recognition accuracy of the image becomes equal to or higher than the threshold value even if image processing is performed on the image to which the region to which the margin corresponding to the specified classification is added is set. , The information processing system according to Appendix 8.
  • Appendix 10 The information processing system according to Appendix 8 or 9, wherein the area determination unit determines an area in which the margin is added to the target area as an image processing area having higher image quality than other areas.
  • Appendix 11 The accuracy with which the remote monitoring device for monitoring the vehicle recognizes the target is specified according to the type of the detected target and the size of the detected target area. Any one of Appendix 8 to 10, further comprising a margin setting device that sets a margin corresponding to the type of the detected target and the size of the detected target area according to the specified accuracy.
  • Information processing system described in. (Appendix 12)
  • the margin setting device is The teacher image data for which the type of the target to be included is known is classified according to the type of the known target and the size of the known target.
  • An area with a margin added to the target area including the target included in the teacher image data is set.
  • the teacher image data is image-processed so that the area to which the margin is added has a higher image quality than the other areas, the accuracy with which the remote monitoring device recognizes the target is specified.
  • the margin setting device is Areas with different margins are set for each of the plurality of teacher image data. When each of the plurality of teacher image data to which the different margins are added is image-processed, the accuracy with which the remote monitoring device recognizes the target is specified.
  • the information processing system wherein a margin at which the recognition accuracy is equal to or higher than a threshold value is set as a margin corresponding to the classification.
  • Appendix 14 The information processing system according to any one of Supplementary note 8 to 12, which includes a storage unit that stores a margin corresponding to the classification.
  • Appendix 15 The image taken by the image pickup unit mounted on the vehicle is acquired and The target area including the target in the acquired image is detected, and the target area is detected. Identify the type of the detected target and the classification including the size of the detected target area. An information processing method for determining an area in which a margin corresponding to the specified classification is added to the target area as an image processing area.
  • the margin is set so that the recognition accuracy of the image becomes equal to or higher than the threshold value even if image processing is performed on the image to which the region to which the margin corresponding to the specified classification is added is set.
  • the information processing method according to Appendix 15. (Appendix 17) The information processing method according to Supplementary note 15 or 16, wherein the region to which the margin is added to the target region is determined as an image processing region having higher image quality than other regions.
  • the accuracy with which the remote monitoring device for monitoring the vehicle recognizes the target is specified according to the type of the detected target and the size of the detected target area.
  • the margin is set so that the recognition accuracy of the image becomes equal to or higher than the threshold value even if image processing is performed on the image to which the region to which the margin corresponding to the specified classification is added is set.
  • the program described in Appendix 21. (Appendix 23) The program according to Supplementary Note 21 or 22, which causes a computer to determine an area in which the margin is added to the target area as an image processing area having higher image quality than other areas.
  • the accuracy with which the remote monitoring device for monitoring the vehicle recognizes the target is specified according to the type of the detected target and the size of the detected target area.
  • the program described. (Appendix 25)
  • the teacher image data in which the type of the object included in the image is known is classified according to the type of the known object and the size of the known object.
  • An area with a margin added to the target area including the target included in the teacher image data is set.
  • the program according to Appendix 24 which causes a computer to specify the accuracy at which the remote monitoring device recognizes the object and set a margin at which the recognition accuracy becomes equal to or higher than a threshold value as a margin corresponding to the classification.
  • Appendix 26 Areas with different margins are set for each of the plurality of teacher image data.
  • Image processing system 3 Remote driver 5 Vehicle 10 In-vehicle camera 100 Information processing device 101 Image acquisition unit 110 Area determination unit 111 Target detection unit 112 Classification identification unit 200 Image processing device 201 Image acquisition unit 210 ROI determination unit 211 Target detection unit 212 Classification identification unit 213 Margin determination unit 220 Image processing unit (encoder) 230 Communication unit 250 Margin storage unit 260 Margin setting unit 300 Margin setting device 400 Remote control device 410 Image recognition unit

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Geometry (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)
PCT/JP2020/037672 2020-10-05 2020-10-05 情報処理装置、情報処理システム、情報処理方法 Ceased WO2022074700A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2022554976A JP7533601B2 (ja) 2020-10-05 2020-10-05 情報処理装置、情報処理システム、情報処理方法
PCT/JP2020/037672 WO2022074700A1 (ja) 2020-10-05 2020-10-05 情報処理装置、情報処理システム、情報処理方法
US18/028,486 US12283110B2 (en) 2020-10-05 2020-10-05 Information processing device, information processing system, and information processing method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/037672 WO2022074700A1 (ja) 2020-10-05 2020-10-05 情報処理装置、情報処理システム、情報処理方法

Publications (1)

Publication Number Publication Date
WO2022074700A1 true WO2022074700A1 (ja) 2022-04-14

Family

ID=81126735

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/037672 Ceased WO2022074700A1 (ja) 2020-10-05 2020-10-05 情報処理装置、情報処理システム、情報処理方法

Country Status (3)

Country Link
US (1) US12283110B2 (https=)
JP (1) JP7533601B2 (https=)
WO (1) WO2022074700A1 (https=)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7708572B2 (ja) * 2021-03-30 2025-07-15 本田技研工業株式会社 移動体の制御装置、制御方法及びプログラム
JP7775151B2 (ja) * 2022-06-21 2025-11-25 株式会社東芝 情報処理装置、情報処理方法、および情報処理プログラム

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016181072A (ja) * 2015-03-24 2016-10-13 クラリオン株式会社 物体認識装置
JP2017062638A (ja) * 2015-09-25 2017-03-30 日立オートモティブシステムズ株式会社 画像認識処理装置、及びプログラム
JP2018172051A (ja) * 2017-03-31 2018-11-08 パナソニックＩｐマネジメント株式会社 駐車支援装置
JP2020064603A (ja) * 2018-10-18 2020-04-23 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ 情報処理装置、プログラム及び情報処理方法

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019125894A (ja) 2018-01-16 2019-07-25 クラリオン株式会社 車載画像処理装置
US11017266B2 (en) * 2018-05-09 2021-05-25 Figure Eight Technologies, Inc. Aggregated image annotation
JP7213662B2 (ja) * 2018-11-09 2023-01-27 キヤノン株式会社 画像処理装置、画像処理方法
IT201900011403A1 (it) * 2019-07-10 2021-01-10 Ambarella Int Lp Detecting illegal use of phone to prevent the driver from getting a fine

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016181072A (ja) * 2015-03-24 2016-10-13 クラリオン株式会社 物体認識装置
JP2017062638A (ja) * 2015-09-25 2017-03-30 日立オートモティブシステムズ株式会社 画像認識処理装置、及びプログラム
JP2018172051A (ja) * 2017-03-31 2018-11-08 パナソニックＩｐマネジメント株式会社 駐車支援装置
JP2020064603A (ja) * 2018-10-18 2020-04-23 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ 情報処理装置、プログラム及び情報処理方法

Also Published As

Publication number Publication date
US20230360405A1 (en) 2023-11-09
US12283110B2 (en) 2025-04-22
JPWO2022074700A1 (https=) 2022-04-14
JP7533601B2 (ja) 2024-08-14

Similar Documents

Publication Publication Date Title
US12304475B1 (en) Automatic collision detection, warning, avoidance and prevention in parked cars
US11488398B2 (en) Detecting illegal use of phone to prevent the driver from getting a fine
US11205068B2 (en) Surveillance camera system looking at passing cars
JP7505500B2 (ja) 送信方法、送信システム及びシステム制御装置
US11970156B1 (en) Parking assistance using a stereo camera and an added light source
EP3700198B1 (en) Imaging device, image processing apparatus, and image processing method
US11109152B2 (en) Optimize the audio capture during conference call in cars
US11721100B2 (en) Automatic air recirculation systems for vehicles
US11161456B1 (en) Using the image from a rear view camera in a three-camera electronic mirror system to provide early detection of on-coming cyclists in a bike lane
US11586843B1 (en) Generating training data for speed bump detection
US10678255B2 (en) Systems, methods and apparatuses are provided for enhanced surface condition detection based on image scene and ambient light analysis
US11659154B1 (en) Virtual horizontal stereo camera
KR20170115544A (ko) 환경적 장면 조건 검출
US11645779B1 (en) Using vehicle cameras for automatically determining approach angles onto driveways
US20200143177A1 (en) Systems and methods of detecting moving obstacles
CN110580808B (zh) 一种信息处理方法、装置、电子设备及智能交通系统
US20150139500A1 (en) Method and System for Optimizing Image Processing in Driver Assistance Systems
US11482012B2 (en) Method for driving assistance and mobile device using the method
US11531197B1 (en) Cleaning system to remove debris from a lens
JP7533601B2 (ja) 情報処理装置、情報処理システム、情報処理方法
US12096533B1 (en) Monitoring and responding to poor visibility conditions by analyzing images using deep neural network
US11840253B1 (en) Vision based, in-vehicle, remote command assist
JP7529034B2 (ja) 情報処理装置、情報処理システム、および情報処理方法
WO2022070426A1 (ja) 情報処理装置、画像送信システム、及び情報処理方法
US12437504B1 (en) Video correctness checking

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20956649

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022554976

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20956649

Country of ref document: EP

Kind code of ref document: A1

WWG Wipo information: grant in national office

Ref document number: 18028486

Country of ref document: US