JP2019211822A - Travel area determination apparatus, travel area determination method and method for generating road surface image machine learning model - Google Patents

Abstract

To provide a travel area determination apparatus and a travel area determination method capable of accurately determining a travel improper area or a travel proper area in a road where a vehicle travels.SOLUTION: A travel area determination apparatus according to the embodiment includes an acquisition unit and a determination unit. The acquisition unit acquires an image of a periphery of a vehicle captured by an imaging apparatus. The determination unit determines a specified area in the image acquired by the acquisition unit by a machine learning model that has machine-learned from a plurality of known images that an object is a road surface.SELECTED DRAWING: Figure 1

Description

  The embodiment of the disclosure relates to a travel area determination device, a travel area determination method, and a road surface image machine learning model generation method.

  An obstacle that detects an obstacle that may collide with a vehicle by comparing an image obtained by imaging the periphery of the vehicle with pattern matching data that has been created assuming the shape of the obstacle in advance. There exists a detection apparatus (for example, refer patent document 1). This obstacle detection result is used as information for an obstacle (unsuitable travel area), a travel suitable area that does not interfere with travel, and information for automatic driving control.

JP 2009-140023 A

  However, since the device that detects the obstacle using the pattern matching data created by assuming the shape of the obstacle in advance cannot detect the unexpected obstacle, there is an obstacle on the road surface. In some cases, it is not possible to accurately determine a region that is likely to be used.

  An embodiment of the present invention has been made in view of the above, and an apparatus and method for accurately determining a travel inappropriate region or a travel preferable region on a road on which the vehicle travels, for example, an obstacle on a road surface on which the vehicle travels It is an object of the present invention to provide a travel region determination device and a travel region determination method that can accurately determine a region where there is a possibility that the vehicle is present.

  A travel region determination device according to an aspect of an embodiment includes an acquisition unit and a determination unit. The acquisition unit acquires an image obtained by imaging the periphery of the vehicle from the imaging device. The determination unit determines a specific region in the image acquired by the acquisition unit based on a machine learning model obtained by machine learning of a plurality of images whose subjects are known to be road surfaces.

  The travel region determination device and the travel region determination method according to one aspect of the embodiment can accurately determine a travel inappropriate region or a travel preferable region on a road on which a vehicle travels.

FIG. 1 is an explanatory diagram illustrating an outline of a travel region determination method according to the embodiment. FIG. 2 is a block diagram illustrating an example of a configuration of the travel region determination device according to the embodiment. FIG. 3 is an explanatory diagram illustrating an example of a road surface image collection according to the embodiment. FIG. 4 is an explanatory diagram illustrating an example of the operation of the travel region determination device according to the embodiment. FIG. 5 is a flowchart illustrating an example of processing executed by the control unit of the travel region determination device according to the embodiment. FIG. 6 is a block diagram illustrating an example of a configuration of a travel region determination device according to a modification of the embodiment.

  Hereinafter, embodiments of a travel region determination device, a travel region determination method, and a road surface image machine learning model generation method will be described in detail with reference to the accompanying drawings. In addition, this invention is not limited by embodiment shown below. FIG. 1 is an explanatory diagram illustrating an outline of a travel region determination method according to the embodiment.

  The travel region determination device according to the embodiment determines a specific region in an image by analyzing an image around the vehicle imaged by an imaging device mounted on the vehicle. The travel area determination device determines whether or not the specific area is a travel suitable area suitable for traveling of the vehicle.

  The travel region determination device stores a road surface image machine learning model obtained by machine learning a plurality of images whose objects are known to be road surfaces in advance. The road surface image machine learning model is a program that outputs a determination result for determining a region having a feature similar to a road surface feature previously machine-learned in the image as a road surface when an image is input.

  For example, as illustrated in FIG. 1A, the travel region determination device, when an image 100 having a box-like falling object 101 on the course of a vehicle is input from the imaging device, the image is used as a road surface image machine learning model. To enter.

  As shown in FIG. 1B, the road surface image machine learning model includes an area A1 having characteristics similar to the road surface characteristics machine-learned in advance in the driving lane (including the same direction lane) of the vehicle in the image 100. It is determined as a road surface (an example of a travel suitable region suitable for vehicle travel).

  At this time, although the road surface image machine learning model cannot determine what the object exists in the area where the falling object 101 exists, it is different from the road surface characteristics machine-learned in advance, It can be determined that the road surface is not at least.

  Therefore, as illustrated in FIG. 1C, the traveling region determination device may have an obstacle in a region A2 other than the region A1 determined as a road surface by the road surface image machine learning model in the traveling lane of the vehicle. Is determined to be a certain region (an example of a travel inappropriate region that is inappropriate for vehicle travel).

  1A, 1B, and 1C is a road lane of the vehicle in the image 100 determined by the road area (the own vehicle traveling road (excluding the oncoming lane)) image machine learning model. It is an area outside the road excluding (including the same lane), and this area is a non-travelable area.

  In addition, this region is a region that is not subject to determination by the road surface image machine learning model. In addition, the traveling region determination apparatus also sets a region above the horizon in the image 100 as a region that is not determined by the road surface image machine learning model.

  In addition, the road surface image machine learning model can be changed to the process related to the road region described above by learning and generating image data indicating a road region where the vehicle can travel (image data that does not consider the opposite lane region as a road surface). .

  In this case, a road surface machine machine learning model may be generated by learning processing with image data in which the road with no separation zone (center) that cannot be overcome is regarded as a road surface. In this case, the vehicle using the result is used. When performing control or the like, it is necessary to separately perform processing such as determining obstacles including the speed of oncoming vehicles.

  As a result, when there is an obstacle that should not be on the road surface, the travel area determination device may have an obstacle rather than a device that detects the obstacle using pattern matching data. It is possible to accurately determine a certain area A2.

  Specifically, if the obstacle detection device using the pattern matching data is an obstacle that is assumed to exist on the road surface, such as a vehicle or a pedestrian, An area where an object is present can be determined.

  However, there may be an object on the road surface that sometimes cannot be on the road. For example, on the road, there may be a case where an object that should not normally be on the road, such as a household tool or a building material, has fallen for some reason.

  Such fallen objects have a variety of shapes, unlike vehicles and pedestrians. For this reason, it is difficult for an apparatus that detects an obstacle using pattern matching data to detect such a falling object as an obstacle.

  In addition, it is possible to detect falling objects as obstacles by preparing pattern matching data for all articles such as household goods and building materials, but this method is a huge amount of pattern matching. It is not realistic because it is necessary to prepare data.

  On the other hand, in the traveling region determination method according to the embodiment, the obstacle A2 is not specified in the traveling lane of the vehicle, but the region A2 other than the region A1 determined as the road surface by the road surface image machine learning model is obstructed. It is determined that there is a possibility that an object exists.

  Therefore, according to the traveling region determination method according to the embodiment, even if an object that should not normally be on the road is falling on the road regardless of the type or shape of the obstacle, the object It is possible to accurately determine a region where there is a possibility of the presence of.

  In the above description, the area where the road surface has not been determined is determined as an obstacle. However, the road surface determination information is used (no obstacle determination is performed), and a travel path (traveling preferable area) is displayed and guided automatically. It can also be used for control.

  Next, an example of the configuration of the travel region determination device 1 according to the embodiment will be described with reference to FIG. FIG. 2 is a block diagram illustrating an example of a configuration of the travel region determination device 1 according to the embodiment. As shown in FIG. 2, the travel region determination device 1 is connected to an imaging device 41, a warning device 42, an image display device 43, and a vehicle control device 44.

  The road surface image collection 110 is an image collection including a plurality of images in which only various road surfaces in which no obstacle is captured are captured. Each image included in the road image collection 110 is provided with information indicating that the subject is a road surface. The road surface image collection 110 is used by the traveling area determination apparatus 1 as a teaching material for supervised machine learning of road surface images. An example of the road surface image collection 110 will be described later with reference to FIG.

  The imaging device 41 is, for example, an in-vehicle camera that is installed in the front part of a vehicle on which the travel region determination device 1 is mounted and images the front of the vehicle. Note that the imaging device 41 may be an in-vehicle camera that is provided on a side portion or a rear portion of the vehicle and images the surroundings of the vehicle. The imaging device 41 outputs the captured image to the travel area determination device 1.

  The warning device 42 is a device that warns the user of the vehicle when the travel region determination device 1 determines a region where an obstacle may exist, for example, a speaker that emits a warning sound. Or an indicator lamp for notifying the presence of an obstacle. The image display device 43 is, for example, a liquid crystal display that displays an image taken by the imaging device 41.

  The vehicle control device 44 is, for example, an ECU (Electronic Control Unit) that performs overall control of the entire vehicle. For example, when the travel region determination device 1 determines a region where an obstacle may exist, the vehicle control device 44 performs an obstacle control on the vehicle by performing vehicle speed control, steering control, and braking control. To avoid.

  The travel area determination device 1 includes a control unit 2 and a storage unit 3. The storage unit 3 is an information storage device such as a data flash, for example, and stores a road surface image machine learning model 31. The control unit 2 includes a microcomputer having a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), and various circuits.

  The control unit 2 includes a learning unit 21, an acquisition unit 22, a division unit 23, a road surface determination unit 24, a synthesis unit 25, which function when the CPU stores a program stored in the ROM using the RAM as a work area. An obstacle determination unit 26, a display control unit 27, and a setting unit 28 are provided.

  The control unit 2 may be configured by hardware such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). Each of the plurality of processing units provided in the control unit 2 realizes or executes an operation of information processing described below. Note that the internal configuration of the control unit 2 is not limited to the configuration illustrated in FIG. 2, and may be another configuration as long as information processing described later is performed.

  The learning unit 21 acquires a plurality (for example, several thousand) of road surface images from the road surface image collection 110, and generates and stores a road surface image machine learning model 31 using the road surface images as teaching materials for supervised machine learning. Store in part 3.

  Here, an example of the road surface image collection 110 according to the embodiment will be described with reference to FIG. FIG. 3 is an explanatory diagram illustrating an example of the road surface image collection 110 according to the embodiment. As shown in FIG. 3, the road surface image collection 110 includes, for example, a road surface image 111 in which a solid center line 111a is provided on the road surface, a road image 112 in which a broken center line 112a is provided on the road surface, and a center line. A road surface image 113 provided with a safety zone 113a is included.

  In addition, the road surface image collection 110 includes, for example, a road surface image 114 in which characters 114a of the upper limit of the traveling speed are described on the road surface, a road surface image 115 in which a mark 115a indicating that there is a pedestrian crossing is described, and a road surface image 115 A road surface image 116 provided with a pedestrian crossing 116a is included.

  The road surface image collection 110 includes, for example, a road surface image 117 having a manhole 117a on the road surface, a road surface image 118 having a road joint 118a on the road surface, and a road surface image 119 having a groove cover 119a on the road surface. . As described above, the road surface image collection 110 includes road surface images 111 to 119 including various road components.

  The learning unit 21 performs supervised machine learning on features such as the color of the road surface, the color, shape, and arrangement of the road surface for each of the road surface images 111 to 119 in which these subjects are known to be road surfaces. A machine learning model 31 is generated.

  Returning to FIG. 2, the description of the processing units other than the learning unit 21 provided in the control unit 2 will be continued. Here, description will be made with reference to FIG. FIG. 4 is an explanatory diagram illustrating an example of the operation of the travel region determination device 1 according to the embodiment.

  The acquisition unit 22 acquires an image in which the periphery of the vehicle is captured from the imaging device 41. Here, as illustrated in FIG. 4A, the acquisition unit 22 acquires an image 100 in which a box-like falling object 101 is present on the course of the vehicle.

  The acquisition unit 22 outputs the image 100 acquired from the imaging device 41 to the dividing unit 23. As shown in FIG. 4A, the dividing unit 23 divides the image 100 input from the obtaining unit 22 into a plurality of divided regions, and outputs the images of the divided regions to the road surface determining unit 24. The road surface determination unit 24 reads the road surface image machine learning model 31 from the storage unit 3 and inputs the image of each divided region input from the division unit 23 to the road surface image machine learning model 31.

  Conceptually, the road surface image machine learning model 31 includes a processing layer having a three-layer structure of an input layer 31a, an intermediate layer 31b, and an output layer 31c, as shown in FIG. 4B. FIG. 4A shows a case where the intermediate layer 31b is a single layer, but a plurality of intermediate layers 31b may be provided.

  The input layer 31a, the intermediate layer 31b, and the output layer 31c include a plurality of nodes 31z that perform road surface determination of the image of each divided region in parallel. Each node 31z of the input layer 31a is connected to all the nodes 31z of the intermediate layer 31b. Each node 31z of the intermediate layer 31b is connected to all the nodes 31z of the output layer 31c. As described above, the road surface image machine learning model 31 has a neural network structure in which the nodes 31z are connected to each other.

  Each node 31z of the input layer 31a determines whether or not the image of each divided region sequentially input from the dividing unit 23 is a road surface, and the determination result that is weighted higher as the possibility of being a road surface is higher is determined in the intermediate layer 31b. Output to the node 31z.

  Similarly, each node 31z of the intermediate layer 31b determines whether or not the image of each divided region is a road surface, and outputs a determination result weighted higher as the possibility of being a road surface increases, to the node 31z of the output layer 31c.

  Each node 31z of the output layer 31c determines whether or not the image of each divided region is a road surface, and outputs a determination result 31d. FIG. 4B shows a determination result 31d of the divided region in the second row from the bottom in the image 100.

  In the example shown in FIG. 4B, the road surface image machine learning model 31 determines that the divided areas in the second row from the bottom in the image 100 are road images other than the images of the second and third divided areas from the left. Yes.

  The road surface determination unit 24 assigns a determination result 31d by the road surface image machine learning model 31 to the image of each divided region and outputs the result to the synthesis unit 25. As shown in FIG. 4C, the combining unit 25 combines the images of the divided areas input from the road surface determination unit 24, returns the image to the image 100 before the division, and outputs it to the obstacle determination unit 26.

  The obstacle determination unit 26 determines whether there is an obstacle for each divided region of the image 100 based on the determination result indicating whether or not the road surface is given to each divided region of the combined image 100 input from the combining unit 25. It is determined whether there is an area that may exist.

  The obstacle determination unit 26 determines whether there is an area where there is no possibility that an obstacle exists in each divided area of the image 100 based on the determination result indicating whether the road surface is given to each divided area of the image 100. Judgment is also made. As described above, the obstacle determination unit 26 can accurately determine a region where an obstacle may exist by determining each divided region of the image 100.

  In addition, the obstacle determination unit 26, when there is a region where there is a possibility that an obstacle exists, the position and size of the region where there is a possibility that an obstacle exists in the traveling lane may hinder the traveling of the vehicle The warning device 42 warns the driver of the vehicle. Then, the obstacle determination unit 26 outputs to the display control unit 27 the image 100 input from the synthesis unit 25 and information indicating the position of the area where the obstacle may exist in the image 100.

  The display control unit 27 causes the image display device 43 to display the image 100 input from the obstacle determination unit 26. Further, as shown in FIG. 4C, the display control unit 27 is located in a region other than the area A2 where the obstacle determination unit 26 determines that an obstacle may exist in the traveling lane of the vehicle in the image 100. The area A1 is highlighted by the image display device 43 as a travelable area.

  Thereby, the display control unit 27 can clearly notify the user of the travelable area of the vehicle. Thereafter, the display control unit 27 outputs information indicating the position of an area where an obstacle may exist in the image 100 to the setting unit 28.

  The setting unit 28 derives a travel route L1 (a route connecting the travelable regions) that avoids the position of a region where an obstacle may exist in the image 100, and performs vehicle control on information indicating the derived travel route L1. Output to the device 44 to set the travel route. Accordingly, the vehicle control device 44 can avoid the falling object 101 by controlling the vehicle so as to travel on the set travel route L1.

  Next, an example of a process executed by the control unit 2 of the travel area determination device 1 according to the embodiment will be described with reference to FIG. FIG. 5 is a flowchart illustrating an example of processing executed by the control unit 2 of the travel region determination device 1 according to the embodiment. The control unit 2 repeatedly executes the process of the flowchart shown in FIG. 5 during the operation of the travel region determination device 1 (during vehicle driving, engine operation, etc.).

  As shown in FIG. 5, the control unit 2 first acquires an image in which the surroundings of the vehicle are imaged from the imaging device 41 (step S101). Thereafter, the control unit 2 divides the acquired image into a plurality of divided regions (step S102).

  Subsequently, the control unit 2 determines whether or not the image in the divided area is a road surface (step S103), and gives a determination result to each divided area (step S104). Thereafter, the control unit 2 determines whether there is an undetermined divided region in one frame image (step S105).

  If the control unit 2 determines that there is an undetermined divided area (Yes in step S105), the control unit 2 repeats the processes of steps S103 to S105 until there is no undetermined divided area.

  Further, when it is determined that there is no undetermined divided area (No in step S105), the control unit 2 combines the images of the divided areas (step S106) and returns the image to the image before the division. Thereafter, the control unit 2 determines, for each divided region, whether there is an obstacle region in which an obstacle may exist based on the determination result given to each divided region (step S107).

  And when it determines with the control part 2 having no obstruction area | region (step S107, No), a process is complete | finished. Moreover, when it determines with the control part 2 having an obstruction area | region (step S107, Yes), it is determined whether a warning is required (step S108).

  The control unit 2 determines that a warning is not necessary when the position of the obstacle is a road shoulder or the size (area) of the obstacle area is equal to or less than a threshold value and does not hinder the traveling of the vehicle. To do. If the control unit 2 determines that the warning is not necessary (No at Step S108), the process ends. If the control unit 2 determines that a warning is necessary (Yes in step S108), the control unit 2 causes the warning device 42 to issue a warning (step S109).

  Subsequently, the control unit 2 highlights a travelable area other than the obstacle area in the travel lane of the vehicle in the image around the vehicle displayed by the image display device 43 (step S110). At this time, the control unit 2 can also highlight the obstacle area. At this time, the control unit 2 can also superimpose and display a travel route that avoids the obstacle area in the display image.

  Then, the control part 2 outputs the information which shows the driving route which avoids an obstacle area | region to the vehicle control apparatus 44, sets a driving route (step S111), and complete | finishes a process. Each time the control unit 2 acquires an image from the imaging device 41, the control unit 2 executes the processes in steps S <b> 101 to S <b> 110.

  In addition, the process which the control part 2 performs is not limited to the process shown in FIG. For example, after the processing in step S101 described above, the control unit 2 determines the region outside the road (the region A shown in FIG. 1 or the region above the horizon in the image 100) and limits the processing range of the road surface determination processing. Or can be used to display an area outside the road.

  In the above-described embodiment, the travel region determination device 1 uses the road surface image machine learning model 31 to determine whether the image is a road surface. However, whether the image is a road surface is determined by pattern matching processing. You may judge. Next, with reference to FIG. 6, a travel region determination device 1 a that determines whether an image is a road surface by pattern matching processing will be described.

  FIG. 6 is a block diagram illustrating an example of a configuration of a travel region determination device 1a according to a modification of the embodiment. Here, among the components shown in FIG. 6, the same components as those shown in FIG. 2 are given the same reference numerals as those shown in FIG.

  As illustrated in FIG. 6, the travel region determination device 1 a is different from the travel region determination device 1 illustrated in FIG. 2 in the configuration of the control unit 2 a and the information stored in the storage unit 3 a. The storage unit 3a stores a road formation image 32.

  The road formation image 32 includes, for example, the road shown in FIG. 3, a solid center line 111a, a dashed center line 112a, a safety zone 113a, characters 114a, a mark 115a, a pedestrian crossing 116a, a manhole 117a, a road joint 118a, and Images of the groove lid 119a and the like are included.

  The control unit 2a includes an acquisition unit 22, a road surface determination unit 24a, an obstacle determination unit 26, a display control unit 27, and a setting unit 28. Among these multiple processing units included in the control unit 2a, processing units other than the road surface determination unit 24a perform the same processing as the acquisition unit 22, the obstacle determination unit 26, the display control unit 27, and the setting unit 28 illustrated in FIG. Do.

  The road surface determination unit 24a performs a pattern matching process using images of a plurality of types of road formations included in the road formation image 32 on the image of the surroundings of the vehicle input from the acquisition unit 22. To determine the area of the road surface in the image.

  Then, the road surface determination unit 24 a outputs information indicating the position of the area determined as the road surface in the traveling lane of the vehicle to the obstacle determination unit 26. Thereby, the obstacle determination unit 26 can determine an area other than the area determined as the road surface by the road surface determination unit 24a in the driving lane of the vehicle as an obstacle area where an obstacle may exist. .

  As described above, the traveling region determination device 1a can determine whether or not the image is a road surface by the pattern matching process. Therefore, the obstacle can be detected by a simpler process without using the road surface image machine learning model 31. It can be determined that there is an obstacle area that may exist.

  In addition, although embodiment mentioned above demonstrated the case where the travel area determination apparatus 1 was provided with the learning part 21, the travel area determination apparatus which concerns on embodiment does not necessarily need to be provided with the learning part 21. FIG. The travel region determination device stores the road surface image machine learning model 31 generated by machine learning using a plurality of images whose object is known to be a road surface by another machine learning device, and thus the obstacle described above. An object determination process can be performed.

In this case, for example, the center-side information processing device connected to the travel region determination device 1 mounted on the vehicle so as to be able to perform wireless communication acquires a plurality of images whose subject is a road surface.
Then, the information processing device generates a road surface image machine learning model (a machine learning model of a traveling region in which the vehicle can travel safely) by machine learning of the plurality of acquired images, and distributes it to the traveling region determination device 1. .

  Thereby, the travel area determination apparatus 1 can perform the above-described obstacle determination process by using the road surface image machine learning model distributed from the information processing apparatus on the center side. In addition to the travel area determination device 1, the information processing device on the center side can also distribute the road surface image machine learning model to any road travel support device that supports the road travel of the vehicle and use it for road travel support. .

  Further effects and modifications can be easily derived by those skilled in the art. Thus, the broader aspects of the present invention are not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications can be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents.

DESCRIPTION OF SYMBOLS 1,1a Traveling area determination apparatus 2,2a Control part 21 Learning part 22 Acquisition part 23 Dividing part 24,24a Road surface determination part 25 Synthesis | combination part 26 Obstacle determination part 27 Display control part 28 Setting part 3, 3a Storage part 31 Road surface image Machine learning model 32 Road formation image 41 Imaging device 42 Warning device 43 Image display device 44 Vehicle control device

Claims (10)

An acquisition unit that acquires an image of the surroundings of the vehicle from the imaging device;
A travel region determination apparatus comprising: a determination unit that determines a specific region in an image acquired by the acquisition unit using a machine learning model obtained by machine learning a plurality of images whose subjects are known to be road surfaces. A dividing unit that divides the image acquired by the acquiring unit into a plurality of divided regions;
The determination unit
The travel region determination device according to claim 1, wherein for each of the divided regions, it is determined whether or not the specific region is a travel suitable region suitable for travel of the vehicle. The determination unit
The travel area determination device according to claim 1, wherein the specific area determined to be a road surface is determined as a preferable travel area. The determination unit
The travel region determination device according to any one of claims 1 to 3, wherein the specific region that has not been determined to be a road surface is determined to be a travel inappropriate region that is inappropriate for travel of the vehicle. The determination unit
The travel region determination device according to any one of claims 1 to 4, wherein the specific region is determined for a region in a travel lane of the vehicle in the image acquired by the acquisition unit. The display control part which highlights the area | region determined as the driving | running | working suitable area | region by the said determination part in the driving | running | working lane of the said vehicle in the image acquired by the said acquisition part by a display apparatus is further provided. Or the travel area determination apparatus of Claim 3. The travel region determination according to any one of claims 2, 3, and 6, further comprising: a setting unit that sets the travel preferable region in the image acquired by the acquisition unit as a travel route of the vehicle. apparatus. An acquisition unit that acquires an image of the surroundings of the vehicle from the imaging device;
A storage unit for storing images of road formations;
An area including the road formation stored in the storage unit in the image acquired by the acquisition unit is determined as a road surface, and there is an obstacle that hinders traveling in an area other than the area determined as the road surface. A travel region determination device comprising: a determination unit that determines a region. An acquisition step of acquiring an image obtained by imaging the periphery of the vehicle from the imaging device;
And a determination step of determining a specific region in the image acquired by the acquisition step using a machine learning model obtained by machine learning of a plurality of images whose subjects are known to be road surfaces. A method for generating a road surface image machine learning model used by a road driving support device that performs road driving support,
An acquisition step of acquiring a plurality of images whose subjects are known to be road surfaces;
And a generation step of generating a road surface image machine learning model by machine learning of the plurality of images acquired by the acquisition step.

Images