JP7167850B2 - Apparatus for generating trained model set for person determination, generation method, generation program, person region determination apparatus, determination method, and determination program - Google Patents

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a trained model set generation apparatus, generation method, generation program, person area determination apparatus, determination method, and determination program for a person determination.

There is known a technique of analyzing captured images obtained from an in-vehicle camera and detecting pedestrians in the traveling direction. Vehicles equipped with in-vehicle cameras are expected to travel in a variety of environments, making it difficult to improve pedestrian detection accuracy. Therefore, techniques for analyzing captured images by using detection results from sonar and radar, selecting an identification dictionary that best matches the detection results, and the like are being studied (see, for example, Patent Document 1).

JP 2016-31564 A

I want to determine the presence or absence of a person from an input image without using other sensor outputs such as sonar. A determination device that uses a trained model to determine whether or not a person appears in an input captured image is conceivable as a device that can meet such a demand. However, when such a determination device is installed in a vehicle, there is the problem that the detection accuracy does not reach the desired level due to the difference in the driving environment that changes greatly as described above.

The present invention has been made to solve such problems, and is a trained model for person determination that can determine with a high probability whether or not a person appears in an input image even if the imaging environment changes. It is an object of the present invention to provide a generation device and a person region determination device or the like equipped with a trained model thereof.

According to the first aspect of the present invention, the apparatus for generating a trained model set for person determination corrects first person image data in which a person is captured, and first non-person images in which a person is not captured. a first generating unit that generates a first trained model that outputs a score indicating the likelihood that a person is shown in input image data by giving data as an incorrect answer and performing supervised learning; Second human image data captured in a second target area different from the input image data and a second generation unit that generates a second trained model that outputs a score indicating the probability that a person is captured in the image.

A method for generating a trained model set for person determination according to a second aspect of the present invention is to correct first person image data in which a person is captured and first non-person image data in which a person is not captured, captured in a first target area. a first generation step of generating a first trained model that outputs a score indicating the likelihood that a person is shown in input image data by giving data as an incorrect answer and performing supervised learning; Second human image data captured in a second target area different from the input image data and a second generation step of generating a second trained model that outputs a score indicating the probability that a person is captured in the image.

According to the third aspect of the present invention, a program for generating a learned model set for person determination corrects first person image data in which a person is captured, and first non-person image data in which a person is not captured, captured in a first target area. a first generation step of generating a first trained model that outputs a score indicating the likelihood that a person is shown in input image data by giving data as an incorrect answer and performing supervised learning; Second human image data captured in a second target area different from the input image data and a second generation step of generating a second trained model outputting a score indicating the likelihood that a person is captured in the image.

The human area determination device according to the fourth aspect of the present invention provides first human image data in which a person is captured as a correct answer and first non-human image data in which a person is not captured as an incorrect answer. A first trained model, which is generated by supervised learning and outputs the likelihood that a person appears in the input image data as a score, and a second target area different from the first target area. , the probability that a person is shown in the input image data generated by giving the second person image data in which a person is shown as a correct answer and the second non-person image data in which a person is not shown as an incorrect answer, and performing supervised learning. as a score, an acquisition unit for acquiring captured image data output by a camera mounted on a moving object, A determining unit for determining which predetermined target area the body is located in, and reading the first learned model from the storage unit when the determining unit determines that the moving object is located in the first target area. a reading unit that reads out the second trained model from the storage unit when the determining unit determines that the mobile object is located in the second target area; a determination unit that calculates a score using the learned model and determines that a specific image region whose score exceeds a predetermined threshold is a person region.

In the human area determination method according to the fifth aspect of the present invention, first human image data in which a person is captured is given as a correct answer, and first non-human image data in which a person is not shown is given as an incorrect answer. A first trained model, which is generated by supervised learning and outputs the likelihood that a person appears in the input image data as a score, and a second target area different from the first target area. , the probability that a person is shown in the input image data generated by giving the second person image data in which a person is shown as a correct answer and the second non-person image data in which a person is not shown as an incorrect answer, and performing supervised learning. as a score, and a second learned model for outputting as a score. An acquisition step of acquiring captured image data output by a camera, a determination step of determining in which predetermined target area the moving object is located, and a determination step determining that the moving object is located in the first target area. a reading step of reading out the first learned model from the storage unit if the moving object is located in the second target area, and reading out the second learned model from the storage unit if the determination step determines that the moving object is located in the second target area; calculating a score using the learned model read out in the reading step for the image of the captured image data acquired in the acquiring step, and determining a specific image region in which the score exceeds a predetermined threshold value as a person region; and a determining step to

A human area determination program according to a sixth aspect of the present invention gives first human image data in which a person is captured as a correct answer and first non-human image data in which a person is not captured as an incorrect answer. A first trained model, which is generated by supervised learning and outputs the likelihood that a person appears in the input image data as a score, and a second target area different from the first target area. , the probability that a person is shown in the input image data generated by giving the second person image data in which a person is shown as a correct answer and the second non-person image data in which a person is not shown as an incorrect answer, and performing supervised learning. as a score, and a second learned model for outputting as a score. An acquisition step of acquiring captured image data output by a camera, a determination step of determining in which predetermined target area the moving object is located, and a determination step determining that the moving object is located in the first target area. a reading step of reading out the first learned model from the storage unit if the moving object is located in the second target area, and reading out the second learned model from the storage unit if the determination step determines that the moving object is located in the second target area; calculating a score using the learned model read out in the reading step for the image of the captured image data acquired in the acquiring step, and determining a specific image region in which the score exceeds a predetermined threshold value as a person region; and a determination step to be performed by the computer.

INDUSTRIAL APPLICABILITY According to the present invention, a trained model generating apparatus for person determination that can determine with a high probability whether or not a person is captured in an input image even if the imaging environment changes, and a person area determination apparatus equipped with the learned model etc. can be provided.

FIG. 4 is a diagram showing how human images and non-human images for learning are created from images captured in an urban area. FIG. 10 is a diagram showing how human images and non-human images for learning are created from images captured in a non-urban area; FIG. 4 is a block diagram showing the configuration of a generation device that generates a trained model set; FIG. 10 is a flow diagram showing a processing procedure for generating a trained model set; FIG. 3 is a schematic diagram showing a state in which the traveling direction is observed from inside the vehicle equipped with the determination device; FIG. 10 is an explanatory diagram illustrating a method of cutting out a judgment image from a captured image; FIG. 10 is an explanatory diagram illustrating a method of determining whether or not a determination image is a person area; FIG. 3 is a block diagram showing the configuration of a determination device that determines a human region from an input image; FIG. 10 is a flow diagram showing a processing procedure for determining a person area;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, the present invention will be described through embodiments of the invention, but the invention according to the scope of claims is not limited to the following embodiments. Moreover, not all the configurations described in the embodiments are essential as means for solving the problems.

2. Description of the Related Art Recently, there has been known a person determination device that uses a trained model to determine whether a person is shown in an input captured image. However, when a captured image captured from a running vehicle is used as an input image, the person determination device often indicates an erroneous determination result. After investigating the cause, the inventors of the present application have found that the main cause is that a person and a background object appear together in an image, and that the background object is diverse. In particular, in order to deal with the variety of background objects, we found that even if machine learning was performed using many images in which people were captured against all kinds of background objects, it was not possible to achieve a certain level of detection accuracy. rice field.

The inventors of the present application further investigated the cause and found that the region in which the input captured image was captured had a large effect on the determination result. In other words, even though objects that appear as background objects have characteristics in each region, even if images taken in all regions are used as training data without considering such characteristics, learning with high detection accuracy can be achieved. I noticed that I can't get a ready model. Therefore, we constructed a method of classifying regions based on the characteristics of background objects and generating trained models for each region by using images captured in each region as teacher images. Moreover, not only images with people in them are treated as correct images, but also images with background objects that are characteristic of each region but without people in them are taught as incorrect images. We succeeded in improving the judgment accuracy. A specific description will be given below.

The learned model set for person determination according to the present embodiment described below is incorporated into a person area determination device that determines an area in which a person is captured from an image captured by a camera unit mounted on a vehicle. I assume. Specifically, the human area determination device according to the present embodiment is installed as part of a driving support system device or a drive recorder.

A human region determining device installed in a vehicle capable of traveling in various environments determines a human region mainly from a captured image having an outdoor environment as a background. At this time, it is between the environment of the urban area and the environment of the non-urban area that a remarkable difference appears as the background of the outdoor environment. Therefore, in the present embodiment, an urban area is defined as the first target area, and a non-urban area other than the urban area is defined as the second target area. Specifically, based on factors such as population density and building density, areas in which vehicles can travel are divided into urban areas and non-urban areas. In preparation for generating the training image data set, sufficient images taken in both urban and non-urban areas are prepared. Then, teacher data for learning is created from the prepared images.

FIG. 1 is a diagram showing how a human image and a non-human image are created as teaching data for learning from images captured in an urban area. As shown in the figure, first, a large number of images captured in an urban area are prepared. Then, a person area and a non-person area are cut out from the image at a preset aspect ratio. Here, as for the aspect ratio, it is preferable that the vertical length is set larger than the horizontal length, because people generally stand in an outdoor environment in many cases. Specifically, length:width=2:1, 4:3, 5:2, etc. In this embodiment, as indicated by the dotted line frame, an aspect ratio of 2:1 is employed.

An area in which a person appears (person area) is cut out from an image taken in an urban area with an aspect ratio of 2:1, and then enlarged or reduced to a specified size, and the image data of the person image is obtained. Generate. Correct tag information is added to the person image data. In addition, regarding the clipping of the area in which the person is captured, the size of the clipping frame is adjusted so that the whole body of the target person is included. At this time, it is preferable that an object, typically an artificial object, that constitutes an urban area is reflected around the person. It should be noted that a plurality of persons may be captured in one cropped area. In this case, it is preferable that the size of the clipping frame is adjusted so that the whole bodies of the plurality of persons can be accommodated. Also, a person riding a bicycle, a person sitting on a bench, or the like may be cut out as a person area.

Similarly, a region (non-human region) in which no person is captured from an image captured in an urban area is cut out at an aspect ratio of 2:1, and is enlarged or reduced to a specified size to obtain a non-human image. Generate image data of a person image. Incorrect tag information is added to the non-person image data. Here, the non-human area to be cut out is selected from among objects, typically man-made objects, that make up the urban area and whose contours are about 2:1. In the example of the figure, the target is a traffic light for pedestrians and a building. In general, a background object having a contour with such an aspect ratio is often erroneously recognized as a person in an urban area. actively learn as

FIG. 2 is a diagram showing how a human image and a non-human image are created as teaching data for learning from images captured in a non-urban area. As illustrated, first, a large number of images captured in a non-urban area are prepared. Then, the person area and the non-person area are cut out with the same aspect ratio as the aspect ratio adopted for the image of the urban area.

Image data of a person image by extracting an area in which a person appears (human area) from an image taken in a non-urban area with an aspect ratio of 2:1, and then enlarging or reducing it to a specified size. to generate Correct tag information is added to the person image data. As in the case of generating person image data in an urban area, when extracting an area in which a person is captured, the size of the clipping frame is adjusted so that the entire body of the target person is included. At this time, it is preferable that an object constituting a non-urban area, typically a natural object, is captured around the person. It should be noted that a plurality of persons may be captured in one cropped area.

Similarly, from an image captured in a non-urban area, a region in which no person is shown (non-human region) is cut out at an aspect ratio of 2:1, and enlarged or reduced to a specified size. Generate image data of a non-person image. Incorrect tag information is added to the non-person image data. Here, the non-human area to be cut out is selected from objects, typically natural objects, that make up the non-urban area and whose contours are about 2:1. In the example shown in the figure, the object is trees, but weeds, fields, and the like can also be objects. However, when the trained model set generated as in the present embodiment is incorporated into a human area determination device equipped as part of a drive recorder, it is observed while driving on a road in a non-urban area. An artificial object may also be cut out as a non-human area. For example, railroad crossing alarms and traffic signs may also be targeted. A background object having a contour with such an aspect ratio is often erroneously recognized as a person in a non-urban area. to learn.

Next, human image data and non-human image data in urban areas and human image data and non-human image data in non-urban areas prepared by such preparations are used as training data for a trained model for person determination. A generation device 100 that generates a set will be described. FIG. 3 is a block diagram showing the configuration of the generation device 100 that generates a trained model set. The generation device 100 is a machine learning device, and a general-purpose computer can also be used. Higher speed processing can be achieved by using GPGPUs (General-Purpose computing on Graphics Processing Units) or large-scale PC clusters.

The generation device 100 mainly includes an image acquisition unit 110 , an operation input unit 120 , a calculation unit 130 and an output unit 140 . The image acquiring unit 110 acquires the person image data and non-person image data in urban areas and the person image data and non-person image data in non-urban areas prepared as described above. If the person image data and the non-person image data are sent via a network, the image acquisition unit 110 is, for example, a LAN interface.

The operation input unit 120 is an interface that receives user input operations, and is a touch panel, a keyboard, or the like superimposed on a monitor. When the image data acquired by the image acquisition unit 110 is not tagged with “correct” or “incorrect” tag information, the user operates the operation input unit 120 while checking the image of the image data. "Correct" or "Incorrect" information can be added to the image data. Also, if there is an error in the tag information, it can be corrected.

The calculation unit 130 is a processor, and is responsible for controlling the entire generation device 100 and performing various calculation processes. A first generation unit 131 serving as a function execution unit of the calculation unit 130 corrects human image data created by cutting out an image captured in an urban area from among the image data acquired by the image acquisition unit 110. , supervised learning is performed with non-human image data as incorrect answers. By performing such machine learning, a first model, which is a trained model for urban areas, is generated that outputs a score indicating the probability that a person appears in the input image data. A second generation unit 132 serving as a function execution unit of the calculation unit 130 generates human image data extracted from an image captured in a non-urban area from among the image data acquired by the image acquisition unit 110. Supervised learning is performed with correct answers and non-human image data as incorrect answers.

By performing such machine learning, a second model, which is a trained model for non-urban areas, is generated, which outputs a score indicating the probability that a person appears in the input image data. In this embodiment, the score is output as a value between 0 and 1, and the closer to 1, the higher the probability that a person is captured. Therefore, the first generation unit 131 and the second generation unit 132 reduce the error between the output value of 1 when the person image data is input, and the output value of 0 when the non-person image data is input. Learning to adjust edge weights using, for example, error backpropagation is repeated so as to reduce the error between .

The output unit 140 outputs the first model generated by the first generation unit 131 and the second model generated by the second generation unit 132 to an external device as a trained model set. If the external device is connected via a network, the output unit 140 is, for example, a LAN interface. If the image acquisition unit 110 is also configured with the same interface, one interface may be shared by the image acquisition unit 110 and the output unit 140 . The external device is, for example, a storage device for storing the output first model and second model, or a system server for mounting the output first model and second model in a determination device to be described later. be.

Note that the generation device 100 may have a configuration for performing the preparation described with reference to FIGS. 1 and 2 . As long as the generation device 100 has a configuration for making preparations, it is possible to consistently perform processing up to generation of a trained model set. Specifically, when the image acquisition unit 110 acquires image data of images captured in an urban area and a non-urban area, the user visually recognizes the images displayed on a monitor (not shown) and The operation input unit 120 is operated to perform the above-described extraction work. The calculation unit 130 enlarges or reduces the cut-out area to a specified size, and generates person image data and non-person image data corresponding to each area. Also, the user operates the operation input unit 120 to add tag information of “correct” or “incorrect” to each image data. The person image data and the non-person image data prepared in this manner are temporarily stored in a storage unit (not shown) and sequentially supplied to the first generation unit 131 and the second generation unit 132 .

FIG. 4 is a flow diagram showing a processing procedure for generating a trained model set. Here, a processing procedure will be described from a state in which the image acquisition unit 110 acquires the person image data and the non-person image data with tag information of “correct” or “incorrect”.

In step S101, the first generating unit 131 selectively takes in human image data in an urban area and performs learning. Specifically, the edge weight of the first model is adjusted so that a value close to 1 is output when human image data in an urban area, which is training data, is input. In step S102, the first generation unit 131 selectively takes in non-human image data of an urban area and performs learning. Specifically, the edge weight of the first model is adjusted so that a value close to 0 is output when non-human image data of an urban area, which is teacher data, is input. Steps S101 and S102 may be performed in reverse order, or it may be recognized whether the captured image data of an urban area is human image data or non-human image data, and processing may be performed accordingly. After completing the learning of the first model, the first generation unit 131 outputs the first model from the output unit 140 in step S103.

In step S104, the second generation unit 132 selectively takes in the person image data in the non-urban area and performs learning. Specifically, the weight of the edge of the second model is adjusted so that a value close to 1 is output when human image data in a non-urban area, which is teacher data, is input. In step S104, the second generation unit 132 selectively captures non-human image data of non-urban areas and performs learning. Specifically, the edge weight of the second model is adjusted so that a value close to 0 is output when non-human image data of a non-urban area, which is teacher data, is input. Steps S104 and S105 may be performed in the reverse order, or it may be recognized whether the captured image data of the non-urban area is human image data or non-human image data, and processing may be performed accordingly. After completing the learning of the second model, the second generation unit 132 outputs the second model from the output unit 140 in step S106.

When the output of the first model and the second model is completed, the series of processing ends. The order of the processing of the first generation unit 131 in steps S101 to S103 and the processing of the second generation unit 132 in steps S104 to S106 may be changed, or the respective processing may be performed in parallel. good.

Next, the determination device 300 incorporating the trained model set generated by the generation device 100 will be described. Determination device 300 is an example of a person region determination device. In this embodiment, the determination device 300 that determines the human region is equipped as part of a driving support system device or a drive recorder. FIG. 5 is a schematic diagram showing a state in which the traveling direction is observed from the interior of the vehicle in which the determination device 300 is mounted. It should be noted that illustration of passengers is omitted in FIG.

The determination device 300 includes a camera unit 320, and the camera unit 320 captures an image of the front through the windshield. The determination device 300 is linked with the navigation system 500 , and the image captured by the camera unit 320 is displayed as the captured image 512 on the display panel 510 provided in the navigation system 500 .

The determination device 300 causes the camera unit 320 to take an image while the vehicle is in a drivable state, and transmits the taken image to the navigation system 500 . Further, the determination device 300 analyzes the captured image to determine whether or not a person appears in the image, and if a person appears in the image, transmits the position information of the area to the navigation system 500. do.

Upon receiving the position information of the person area, the navigation system 500 superimposes a person frame 513 drawn by CG on the person area of the captured image 512 and displays a warning display 514 . Warning display 514 is prominently displayed using text or animation, for example, "Attention!" as shown. The warning may be accompanied by sound. By recognizing such a warning, the driver can drive more safely.

Next, processing up to determining a region in which a person is captured from the captured image 512 captured by the camera unit 320 will be described. FIG. 6 is an explanatory diagram illustrating a method of cutting out the determination image CF from the captured image 512. FIG. Here, it is assumed that the vehicle is traveling in a non-urban area.

When acquiring the captured image 512, the determination device 300 generates a frame window FW having a preset aspect ratio. The aspect ratio set here is equal to the aspect ratio of the human image and the non-human image learned by the above-described first model and second model. In addition, a plurality of frame windows FW of different sizes are prepared in consideration of the size of a person who may appear in the captured image, and the following processing is executed for the frame windows FW of each size. be done.

The determination device 300 scans the generated frame window FW over the image area of the captured image 512 while shifting it by a predetermined width. Then, contour extraction is performed on the image area surrounded by the frame window FW at each position to be scanned, and it is determined whether or not there is an object suitable for the frame window FW. Specifically, for example, when there is a closed contour having an area of 50% or more of the area of the frame window FW, it is determined that a suitable object exists.

If the determination device 300 determines that there is an object suitable for the frame window FW, the determination device 300 cuts out the area surrounded by the frame window FW as the determination image CF. The cut-out determination image CF is shaped into a format that the first model and the second model accept as input image data, and is stored in order as determination image data CF ₁ , CF ₂ , CF ₃ . Each piece of determination image data is associated with clipping coordinate information indicating which region of the captured image 512 has been clipped.

FIG. 7 is an explanatory diagram illustrating a method for determining whether or not the determination image CF cut out from the captured image 512 is a person area. The determination device 300 inputs the determination image CF _m shown in the upper part to the second model, which is a trained model for non-urban areas. Assume that a score of 0.0127 is output as a result. The determination device 300 determines that the area of the determination image _CFm is not the person area because the score=0.0127 is equal to or less than the threshold value (for example, 0.75) for determining that the person is a person.

Similarly, the determination device 300 inputs the determination image _CFn shown in the lower stage to the second model. Assume that a score of 0.8752 is output as a result. Since the score of 0.8752 exceeds the threshold, the determination device 300 determines that the area of the determination image _CFn is the human area. When the determination device 300 transmits the captured image 512 to the navigation system 500 together with the cutout coordinate information of the determination image CF determined as the human area, the navigation system 500 superimposes the human frame 513 on the captured image 512 as described above. can be displayed. When a plurality of person areas exist in the captured image 512, a person frame 513 is superimposed on each area. Since the camera unit 320 generates captured images at, for example, 30 fps, the determination device 300 sequentially performs the above-described processing on the generated captured images.

5 to 7 are examples in which the vehicle travels in a non-urban area. Switch to some first model. Other processing is the same when traveling in an urban area. Using this method, trained models corresponding to both urban and non-urban areas can be prepared and selectively used according to the current environment. It is possible to determine with high probability whether or not a person appears in the image.

Next, the configuration of the determination device 300 that determines a human region from an input image will be described. FIG. 8 is a block diagram showing the configuration of the determination device 300. As shown in FIG. The determination device 300 mainly includes an arithmetic unit 310 , a camera unit 320 , an image processing unit 330 , a GPS unit 340 , an output unit 350 and a storage unit 360 . The calculation unit 310 is a processor, and is responsible for controlling the entire determination device 300 and performing various calculation processes. The camera unit 320 includes a lens and an imaging element, images the surrounding environment, generates an imaging signal, and outputs the imaging signal to the image processing section 330 . The image processing section 330 receives an imaging signal from the camera unit 320 and generates captured image data according to a specified format. The image processing unit 330 also executes the above-described clipping processing, shaping processing, and the like on the captured image data to generate judgment image data. Note that the camera unit 320 may be configured to be connected to the determination device 300 by wire or wirelessly as an external device. In this case, the determination device 300 may include an image acquisition section as an interface that receives image signals from the camera unit 320 .

The GPS unit 340 receives GPS signals, converts the current position into latitude and longitude information, and transfers the information to the calculation unit 310 . Note that the GPS unit 340 may be configured to be connected to the determination device 300 by wire or wirelessly as an external device. In this case, the determination device 300 may include an information acquisition section as an interface for receiving the latitude and longitude information of the current position from the GPS unit 340 . As described above, the output unit 350 outputs the generated captured image data captured by the camera unit 320 and generated by the image processing unit 330 and the extraction coordinate information of the determination image CF determined as the person region to an external device (this embodiment). In the form, it outputs to the navigation system 500).

The storage unit 217 is, for example, an SSD (Solid State Drive), and is used for control and calculation in addition to a control program for controlling the determination device 300 and a determination program for performing a determination calculation for a human area. It stores various parameter values, functions, lookup tables, etc. In particular, it stores a first model 361 and a second model 362 generated by the generation device 100 and map information 363 of a range where the determination device 300 is expected to be used. The map of the map information 363 describes the urban area and the non-urban area defined as described above.

An area determination unit 311 serving as a function execution unit of the calculation unit 310 receives latitude and longitude information from the GPS unit 340 and refers to the map information 363 to determine whether the local point is located in an urban area or not. It determines whether it is positioned, and hands over the determination result to the reading unit 312 . The reading unit 312 reads the first model 361 from the storage unit 360 when the determination result indicates that the model is located in an urban area, and reads the first model 361 from the storage unit 360 when the determination result indicates that the model is located in a non-urban area. Read the second model 362 from the .

The determination unit 313 sequentially inputs the determination image data generated from the captured image data by the image processing unit 330 to the trained model read by the reading unit 312 to calculate a score. If the calculated score exceeds a preset threshold value, the captured image area corresponding to the determination image data is determined as the person area.

FIG. 9 is a flow diagram showing a processing procedure for determining a person area. Here, the processing procedure from the time when the vehicle becomes drivable will be described. The determination device 300 acquires captured image data in step S201. Specifically, the image processing section 330 receives an imaging signal from the camera unit 320 and generates captured image data. In step S202, the region determination unit 311 receives the latitude and longitude information from the GPS unit 340, refers to the map information 363 in the storage unit 360, and determines whether the local point is located in an urban area or a non-urban area. determine whether

Proceeding to step S203, the reading unit 312 reads out the first model 361 from the storage unit 360 when the determination result of step S202 indicates that the model is located in the non-urban area. , the second model 362 is read from the storage unit 360 . In step S204, the calculation unit 310 cooperates with the image processing unit 330 to cut out the judgment image CF from the captured image data acquired in step S201. The image processing unit 330 generates determination image data from the cut-out determination image CF. Proceeding to step S205, the calculation unit 310 determines whether or not the scanning process has been completed for all of the prepared frame windows FW, that is, whether or not the clipping process has been completed. If not completed, the process returns to step S204, and if completed, the process proceeds to step S206.

In step S206, the determination unit 313 sequentially inputs each of the generated determination image data to the trained model read out in step S203 to calculate a score. If the calculated score exceeds a preset threshold value, the captured image area corresponding to the determination image data is determined as the person area. As the thresholds applied here, a first threshold used for the score output by the first model 361 and a second threshold used for the score output by the second model 362 may be set. That is, different thresholds may be applied depending on whether the first model 361 or the second model 362 is used. In this case, the first threshold is preferably greater than the second threshold. Since there are a wide variety of artifacts that can be background objects in urban areas, the first model tends to output a relatively high score even if the determination image does not include a person. In this way, if the first threshold is set to a value larger than the second threshold, it is possible to more accurately determine the person area in the urban area.

Proceeding to step S<b>207 , the calculation unit 310 outputs the captured image data acquired in step S<b>201 and the cropped coordinate information of the determination image determined as the person region in step S<b>206 from the output unit 350 to an external device. Calculation unit 310 checks in step S208 whether or not there is an end instruction. The end instruction is generated when the vehicle stops or the power button of the determination device 300 is turned off. If there is no end instruction, the process returns to step S201. If there is an end instruction, the series of processing ends.

In the present embodiment described above, it is assumed that the visible image is obtained by capturing visible light, but an infrared image obtained by capturing infrared light may be used. Infrared images have fewer pixels than visible images and are often rough images, and when shooting far-infrared images, the heat emitted by the object to be shot is captured as an image, so there are many heat sources. In urban areas, an object with an aspect ratio close to that of a person is likely to be erroneously recognized as a person. Therefore, the above method is particularly effective when determining a person area from an infrared image (including a far-infrared image).

Further, in the present embodiment described above, the aspect ratio of the determination image input to the first model 361 and the aspect ratio of the determination image input to the second model 362 are the same, but they may be different. If there is a background object that tends to cause erroneous recognition in each target area, the aspect ratio may be individually set according to the outline of the background object.

Further, in the present embodiment described above, the determination device 300 mounted on a vehicle is assumed, but the application of the determination device for determining the human region is not limited to this. If it is installed in a vehicle, it is effective to divide the target area into an urban area and a non-urban area assuming a motorway. It is preferable to separate For example, even within urban areas, there are residential areas and commercial areas, park areas and non-park areas, and even within non-urban areas, there are residential areas and suburban areas. . Also, the number of divisions is not limited to two, and may be divided into three or more. In this case, the learned model is generated according to the division of the target area.

In addition, in the present embodiment described above, it is assumed that the determination device 300 is installed as part of a driving support system device or a drive recorder. may serve as the determination device. In this case, the server receives captured image data captured by a camera unit installed in the vehicle, determines the person's area, and returns the coordinate information of the person's area to the vehicle's driving support system or the like.

100 generation device 110 image acquisition unit 120 operation input unit 130 calculation unit 131 first generation unit 132 second generation unit 140 output unit 300 determination device 310 calculation unit 311 area determination unit 312 reading unit , 313 determination unit, 320 camera unit, 330 image processing unit, 340 GPS unit, 350 output unit, 360 storage unit, 361 first model, 362 second model, 363 map information, 500 navigation system, 510 display panel, 512 imaging Image, 513 person frame, 514 warning display, 901 person

Claims (8)

The first person image data captured in the first target area, in which a person is captured, is defined as the correct first person image data, and the first non-person image data in which the person is not captured. a first generation unit that generates a first trained model that outputs a score indicating the likelihood that a person is shown in the input image data by giving an object that is incorrect as an incorrect answer and performing supervised learning;
Second person image data in which a person is captured in a second target area different from the first target area is erroneously recognized as a person in the second target area as second non-person image data in which a person is not captured. Generating a second trained model that outputs, as a score, the likelihood that a person appears in the input image data by giving an object that constitutes the second target area as an incorrect answer and performing supervised learning. 2 generating unit for generating a trained model set for person determination. The first person image data and the first non-person image data have a width greater than a preset width from the first image captured in the first target area defined as an urban area. is based on an image region cropped at a large aspect ratio with a length of
The first non-human image data is image data obtained by photographing an artificial object,
The second person image data and the second non-person image data are image areas cut out at the aspect ratio from the second image captured in the second target area defined as an area other than an urban area. was created on the basis of
The second non-human image data is image data obtained by photographing a natural object,
2. The generator of claim 1. The first person image data captured in the first target area, in which a person is captured, is defined as the correct first person image data, and the first non-person image data in which the person is not captured. a first generation step of generating a first trained model that outputs a score indicating the probability that a person appears in the input image data, by giving an object as an incorrect answer and performing supervised learning;
Second person image data in which a person is captured in a second target area different from the first target area is erroneously recognized as a person in the second target area as second non-person image data in which a person is not captured. Generating a second trained model that outputs, as a score, the likelihood that a person appears in the input image data by giving an object that constitutes the second target area as an incorrect answer and performing supervised learning. 2. A method for generating a trained model set for person determination, including a generation step. The first person image data captured in the first target area, in which a person is captured, is defined as the correct first person image data, and the first non-person image data in which the person is not captured. a first generation step of generating a first trained model that outputs a score indicating the probability that a person appears in the input image data, by giving an object as an incorrect answer and performing supervised learning;
Second person image data in which a person is captured in a second target area different from the first target area is erroneously recognized as a person in the second target area as second non-person image data in which a person is not captured. Generating a second trained model that outputs, as a score, the likelihood that a person appears in the input image data by giving an object that constitutes the second target area as an incorrect answer and performing supervised learning. 2. A program for generating a trained model set for person determination that causes a computer to execute the generation step. The first person image data captured in the first target area, in which a person is captured, is defined as the correct first person image data, and the first non-person image data in which the person is not captured. a first trained model that outputs a score indicating the probability that a person appears in the input image data, which is generated by supervised learning by giving an object as an incorrect answer, and the first target area; The second person image data captured in a different second target area, in which a person is captured, is defined as the correct second person image data, and the second target area, which is erroneously recognized as a person in the second target area, as second non-person image data in which the person is not captured. learning for person determination, including a second trained model that outputs a score indicating the probability that a person appears in the input image data, which is generated by supervised learning by giving constituent objects as incorrect answers; a storage unit that stores the completed model set;
an acquisition unit that acquires captured image data output by a camera mounted on a moving object;
a determination unit that determines in which predetermined target area the mobile object is located;
When the determining unit determines that the moving object is located in the first target area, the determining unit reads the first trained model from the storage unit, and determines that the moving object is located in the second target area. a reading unit that reads the second trained model from the storage unit when determined by
The score is calculated using the learned model read by the reading unit for the image of the captured image data acquired by the acquiring unit, and a specific image region in which the score exceeds a predetermined threshold is defined as a person region. A person region determination device comprising a determination unit for determining. The first target area is an area defined as an urban area, the first non-human image data is image data obtained by photographing an artificial object,
The second target area is an area defined as an area other than an urban area, the second non-human image data is image data obtained by photographing a natural object,
The threshold includes a first threshold used for the score output by the first trained model and a second threshold used for the score output by the second trained model,
6. The person area determination device according to claim 5, wherein the first threshold is greater than the second threshold. The first person image data captured in the first target area, in which a person is captured, is defined as the correct first person image data, and the first non-person image data in which the person is not captured. a first trained model that outputs a score indicating the probability that a person appears in the input image data, which is generated by supervised learning by giving an object as an incorrect answer, and the first target area; The second person image data captured in a different second target area, in which a person is captured, is defined as the correct second person image data, and the second target area, which is erroneously recognized as a person in the second target area, as second non-person image data in which the person is not captured. learning for person determination, including a second trained model that outputs a score indicating the probability that a person appears in the input image data, which is generated by supervised learning by giving constituent objects as incorrect answers; A human region determination method for determining a human region using a storage unit that stores a completed model set,
an acquisition step of acquiring captured image data output by a camera mounted on a moving object;
a determination step of determining in which predetermined target area the mobile object is located;
when it is determined in the determining step that the moving object is located in the first target area, the first learned model is read from the storage unit, and the determination is made that the moving object is located in the second target area. a reading step of reading the second trained model from the storage unit if determined in step;
calculating the score using the learned model read out in the reading step for the image of the captured image data acquired in the acquiring step, and determining a specific image region in which the score exceeds a predetermined threshold; A human region determination method including a determination step of determining a human region. The first person image data captured in the first target area, in which a person is captured, is defined as the correct first person image data, and the first non-person image data in which the person is not captured. a first trained model that outputs a score indicating the probability that a person appears in the input image data, which is generated by supervised learning by giving an object as an incorrect answer, and the first target area; The second person image data captured in a different second target area, in which a person is captured, is defined as the correct second person image data, and the second target area, which is erroneously recognized as a person in the second target area, as second non-person image data in which the person is not captured. learning for person determination, including a second trained model that outputs a score indicating the probability that a person appears in the input image data, which is generated by supervised learning by giving constituent objects as incorrect answers; A human region determination program for determining a human region using a storage unit that stores a completed model set,
an acquisition step of acquiring captured image data output by a camera mounted on a moving object;
a determination step of determining in which predetermined target area the mobile object is located;
when it is determined in the determining step that the moving object is located in the first target area, the first learned model is read from the storage unit, and the determination is made that the moving object is located in the second target area. a reading step of reading the second trained model from the storage unit if determined in step;
The score is calculated using the learned model read out in the reading step for the image of the captured image data acquired in the acquiring step, and a specific image region in which the score exceeds a predetermined threshold is selected. A human area determination program that causes a computer to execute a human area determination step.

Images