JP7272705B1 - Information processing device, information processing method and information processing program - Google Patents

Abstract

An information processing device, an information processing method, and an information processing program capable of estimating the number of people based on an image are provided. An information processing apparatus includes a reception unit that receives first image information in which a person is recorded, and an estimation unit that estimates the first image information based on the first trained model and the first image information. a second trained model generated by learning a heat map indicating presence/absence of a person and the number of persons in the heat map when the content estimated by the estimation unit is the first; a first acquiring unit for acquiring the number of persons recorded in the first image information based on the first image information; and an image in which the persons are recorded when the content estimated by the estimating unit is second. , a third trained model generated by learning the person, and the first image information, each person recorded in the first image information is recognized, and the number of persons is determined based on the recognition result. and a second acquisition unit that acquires the [Selection drawing] Fig. 1

Description

The present disclosure relates to an information processing device, an information processing method, and an information processing program.

Conventionally, there is a device for estimating congestion conditions. An image processing apparatus described in Patent Literature 1 generates a heat map according to the level of the degree of congestion based on an input image, and uses the heat map to estimate the degree of congestion. In addition, the image processing device includes an analysis unit for normal congestion (normal image analysis unit) and an analysis unit for congestion (congestion image analysis unit), depending on the estimated degree of congestion. and perform image analysis using the selected image analysis unit.

Retable 2018/061976

The image analysis device described in the above-mentioned Patent Document 1 is used when detecting a person staying and an abandoned object.
However, there are cases where it is desired to estimate the number of persons (persons) recorded in an image, and the technique described in Patent Document 1 cannot estimate the number of persons.

The present disclosure provides an information processing device, an information processing method, and an information processing program capable of estimating the number of people based on an image.

An information processing apparatus according to one aspect is generated by learning a reception unit that receives first image information in which a person is recorded, an image in which a plurality of persons are recorded, and the number of persons recorded in the image. an estimating unit for estimating the number of persons recorded in the first image information based on the first trained model received by the receiving unit and the first image information received by the receiving unit; In this case, based on a second trained model generated by learning a heat map indicating the presence or absence of a person, the number of people in the heat map, and the first image information received by the reception unit, When the number of people estimated by the first acquisition unit that acquires the number of people recorded in the first image information and the estimation unit is less than a threshold value, the image is generated by learning the image in which the people are recorded and the people. second acquisition for recognizing each person recorded in the first image information based on the third learned model received by the reception unit and the first image information received by the reception unit, and acquiring the number of persons based on the recognition result; and

According to one aspect, first image information in which a person is recorded is received, estimation is performed on the first image information based on the first image information and the first trained model, and when the estimation is the first , a second trained model generated by learning a heat map indicating the presence or absence of a person and the number of people in the heat map, and the first image information, based on the first image information. A third trained model generated by obtaining the number of people recorded and learning the images in which the people are recorded and the people in a second case where the above estimation is different from the first case , and the first image information, each person recorded in the first image information can be recognized, and the number of persons can be acquired based on the recognition result.

1 is a diagram for explaining an information processing device according to an embodiment; FIG. 1 is a block diagram for explaining an information processing device according to an embodiment; FIG. 4 is a first flowchart for explaining an information processing method according to one embodiment; 7 is a second flowchart for explaining the information processing method according to one embodiment;

An embodiment of the present invention will be described below.

[Overview of information processing device 1]
First, an outline of an information processing device 1 according to an embodiment will be described.
FIG. 1 is a diagram for explaining an information processing device 1 according to one embodiment.

The information processing device 1 may be configured as, for example, a people estimation device that estimates the number of people recorded in an image. Further, the information processing device 1 may be configured as an estimation device or the like for estimating the content recorded in the image, for example.

The information processing device 1 may be, for example, a computer such as a server, desktop, laptop, and tablet.
The information processing apparatus 1 estimates the number of persons recorded in the first image information 101 based on the first image information 101 and the first trained model 201 . The first trained model 201 may be, for example, a trained model generated by learning an image in which a plurality of persons are recorded and the number of persons recorded in the image.
In addition, based on the first image information 101 and the first trained model 201, the information processing apparatus 1 performs population estimation using a heat map or human recognition. may be selected. In this case, the first trained model 201 learns an image in which a plurality of people are recorded, and the selection of either estimation of the number of people using a heat map or estimation of the number of people using person recognition. It may be a trained model that is generated.

When the number of people estimated as described above is equal to or greater than the preset threshold and when the number of people estimation using the heat map is selected, the information processing apparatus 1 uses the first image information 101 and the second learned Based on the model 202, the number of persons recorded in the first image information 101 is acquired. The second trained model 202 may be, for example, a trained model generated by learning a heat map indicating the presence or absence of persons and the number of persons in the heat map.

On the other hand, when the number of people estimated as described above is less than the threshold and when the number of people estimation using person recognition is selected, the information processing apparatus 1 and each person recorded in the first image information 101 is recognized. The third trained model 203 may be, for example, a trained model generated by learning an image in which a person is recorded and the person. The information processing apparatus 1 acquires the number of persons recorded in the first image information 101 by tallying the number of recognized persons.

[Details of information processing device 1]
Next, details of the information processing apparatus 1 according to one embodiment will be described.
FIG. 2 is a block diagram for explaining the information processing device 1 according to one embodiment.

The information processing device 1 includes, for example, a communication unit 21, a storage unit 22, a display unit 23, a control unit 11, and the like. The communication unit 21, the storage unit 22, and the display unit 23 may be an embodiment of the output unit. The control unit 11 includes, for example, a reception unit 12, an estimation unit 13, a selection unit 14, a first acquisition unit 15, a second acquisition unit 16, an output control unit 17, and the like. Note that the selection unit 14 will be described in a modified example described later. The control unit 11 may be configured by, for example, an arithmetic processing unit of the information processing device 1 or the like. The control unit 11 (for example, an arithmetic processing unit, etc.) reads and executes various programs stored in the storage unit 22, etc. as appropriate, so that each unit (for example, the reception unit 12, the estimation unit 13, the selection unit 14 , first acquisition unit 15, second acquisition unit 16, output control unit 17, etc.).

The communication unit 21 can transmit and receive various information with, for example, a device (external device) outside the information processing device 1 .

The storage unit 22 may store various information and programs, for example. An example of the storage unit 22 may be a memory, a solid state drive, a hard disk drive, and the like.

The display unit 23 can display various characters, symbols, images, and the like, for example.

The reception unit 12 receives first image information in which a person is recorded. The first image information may be moving image information and still image information.
For example, the reception unit 12 may receive the first image information from an external device (external device) or the like. The external device in this case may be, for example, a server and a camera (not shown). The cameras may be arranged outdoors and indoors, for example, and may be surveillance cameras or the like, for example. The camera may, for example, generate image information by capturing an image in a predetermined direction (subject or the like). The server may, for example, acquire and accumulate image information generated by the camera.
That is, the receiving unit 12 may receive image information (first image information) accumulated in the server via the communication unit 21, for example.
The reception unit 12 may also receive, for example, image information (first image information) generated by a camera. In this case, the reception unit 12 may receive image information (first image information) transmitted from the camera via the communication unit 21, for example. Further, for example, when a memory (not shown) in which image information generated by a camera is recorded is inserted into an interface (not shown) of the information processing apparatus 1, the reception unit 12 reads the image information (first 1 image information) may be received.

The estimation unit 13 estimates the number of persons recorded in the first image information based on the first trained model and the first image information received by the reception unit 12 . The first trained model may be, for example, a trained model generated by learning an image in which a plurality of persons are recorded and the number of persons recorded in the image. That is, the first trained model learns, for example, images of a plurality of patterns (plurality of images) in which a plurality of persons are recorded, and the number of persons recorded in each image of each pattern (number of persons). It may be a trained model.

In this case, the first trained model includes, for example, an image in which a person is recorded, the number of people estimated by inputting the image into a second trained model described later, and the image as a third trained model described later. It may be a trained model generated by learning the number of people estimated using . At this time, the first trained model is, for example, the number of people estimated by inputting the image into the second trained model described later, and the number of people estimated by using the third trained model described later. It may be a trained model that is generated by further learning which of the two is the correct number of people. When using such a first trained model, the estimating unit 13, for example, uses second image information in which people are recorded, the number of people estimated based on the second trained model, and the second image information and estimating the number of people using a first learned model generated by learning the number of people estimated based on the third learned model and the number of people recorded in the second image information; good too. The second image information may be, for example, image information in which a person is recorded and used for learning.

The above-described first trained model may be generated by, for example, learning by the control unit 11 (for example, a learning unit (not shown), etc.).
Further, the first trained model may be generated by, for example, learning by a device (an external device (for example, a learning device (not shown)), etc.) outside the information processing device 1. The external device may When generating the first trained model, the information processing apparatus 1 may acquire the first trained model, for example, via the communication unit 21 or using a memory (not shown) or the like.

When the number of people estimated by the estimation unit 13 is equal to or greater than the threshold, the first acquisition unit 15 records the first image information based on the second trained model and the first image information received by the reception unit 12. Get the number of people. The second trained model may be, for example, a trained model generated by learning a heat map indicating the presence or absence of persons and the number of persons in the heat map.

That is, the second trained model, for example, has a range of locations where the existence of a person is highly likely to be estimated and a location where the existence of a person is unlikely to be estimated. It may be a trained model that has learned a heat map indicating the presence or absence of existence. In this case, for example, the first acquisition unit 15 selects a location where the existence of a person is likely to be estimated and a location where the possibility of the existence of a person is unlikely to be estimated. The number of persons may be estimated using a second trained model that has learned a heat map indicating the presence or absence of a person. The heat map may express the presence or absence of a person by, for example, color differences or color shading. A heat map can be created, for example, by various known methods.

The second trained model includes, for example, a heat map of a plurality of patterns in which a plurality of people are recorded (a pattern of the plurality of heat maps), and the number of people (number of people) recorded in each heat map of each pattern. It may be a trained model that has learned
The above-described second trained model may be generated by, for example, learning by the control unit 11 (for example, a learning unit (not shown), etc.).
Further, the second trained model may be generated, for example, by performing learning by a device (an external device (for example, a learning device (not shown)), etc.) outside the information processing device 1. The external device may When generating the second trained model, the information processing apparatus 1 may acquire the second trained model, for example, via the communication unit 21 or using a memory (not shown) or the like.

The first acquisition unit 15 may generate a heat map indicating presence/absence of a person based on the first image information received by the reception unit 12, for example. The first acquisition unit 15 can generate a heat map based on the first image information using various known methods, for example. As an example, the first acquisition unit 15 may generate a heat map indicating the presence or absence of the person based on the first image information received by the reception unit 12 and the characteristics of the person specified in advance. The features of a person may be, for example, the outline (outline) of a person's body, or may be various other features. For example, after generating the heat map as described above, the first acquisition unit 15 may estimate the number of people based on the heat map and the second learned model.

When the number of people estimated by the estimation unit 13 is less than the threshold, the second acquisition unit 16 records the first image information based on the third trained model and the first image information received by the reception unit 12. Recognize each person and acquire the number of persons based on the recognition result. The third trained model may be, for example, a trained model for recognizing a person based on an image. That is, the third trained model may be, for example, a trained model generated by learning an image in which a person is recorded and the person. The third trained model may be, for example, a trained model that has learned a plurality of patterns of images (plurality of images) in which a person is recorded and a person recorded in each pattern image. As an example, the third trained model may be one that has learned characteristics of a person as a person. The features of the person in this case are, for example, the outline of the body of the person, the face (features of the face (e.g., eyes, nose, mouth, etc.)), the back view of the person, the side view of the person, the skeleton of the person, and Various features such as feature points (for example, joints, etc.) of a person's body may be used.

The above-described third trained model may be generated by, for example, learning by the control unit 11 (for example, a learning unit (not shown), etc.).
Further, the third trained model may be generated by, for example, learning by a device (an external device (for example, a learning device (not shown)), etc.) outside the information processing device 1. The external device may When generating the third trained model, the information processing apparatus 1 may acquire the third trained model, for example, via the communication unit 21 or using a memory (not shown) or the like.

The second acquisition unit 16 recognizes one or more persons recorded in the first image information, for example, based on the first image information and the above-described third learned model. The second acquisition unit 16 acquires the number of persons (number of persons) recorded in the first image information, for example, based on the number of recognized persons.

The above-described threshold values used in the first acquisition unit 15 and the second acquisition unit 16 may be the same (or different) values, for example. Which is more accurate when estimating the number of people using a heat map (using the second trained model) or when estimating the number of people using person recognition (third trained model)? It may be set by performing an experiment or the like in advance to determine whether or not an appropriate number of people can be output.
In general, when the number of people recorded in the first image information (number of people) is relatively large, it is more preferable to estimate the number of people using a heat map (using the second trained model). Accurate number of people can be output. On the other hand, when the number of people recorded in the first image information (number of people) is relatively small, estimating the number of people using person recognition (third trained model) outputs a more accurate number of people. be able to. As an example, the threshold may be various values such as 50, 60, 70, 80, and 90, although appropriate values may vary depending on the situation.

The output control unit 17 may control the output unit to output the estimation result by the first acquisition unit 15 and the estimation result by the second acquisition unit 16 . The output unit may be, for example, the communication unit 21, the storage unit 22, the display unit 23, and the like.
That is, the output control unit 17 may control the communication unit 21 to transmit information about the estimation result obtained by the first obtaining unit 15 and information about the estimation result obtained by the second obtaining unit 16 to the outside (external device). . The external device in this case may be, for example, a server and a user terminal (not shown). The user terminal may be, for example, a terminal used by the user of the information processing apparatus 1, and may be a desktop, a laptop, a tablet, a smartphone, or the like as a specific example.
The output control unit 17 may control the storage unit 22 to store information about the estimation result obtained by the first obtaining unit 15 and information about the estimation result obtained by the second obtaining unit 16 .
The output control unit 17 may control the display unit 23 to display the estimation result by the first acquisition unit 15 and the estimation result by the second acquisition unit 16 .

[Information processing method]
Next, an information processing method according to one embodiment will be described.

First, an example of a learning method will be described as an information processing method according to an embodiment.
FIG. 3 is a first flowchart for explaining an information processing method according to one embodiment.

In step ST101, the control unit 11 acquires second image information in which a person is recorded.

In step ST102, the control unit 11 estimates the number of persons recorded in the second image information based on the second image information acquired in step ST101 and the second trained model. The second trained model may be, for example, a trained model generated by learning a heat map indicating the presence or absence of persons and the number of persons in the heat map. That is, for example, the second trained model can determine the presence of a person between a location where the presence of a person is highly likely to be estimated and a location where the possibility of a person is unlikely to be estimated. It may be a trained model that has learned a heat map indicating the presence/absence.

In step ST103, the control unit 11 estimates the number of persons recorded in the second image information based on the second image information obtained in step ST101 and the third learned model. The third trained model may be, for example, a trained model for recognizing a person based on an image. That is, the third trained model may be, for example, a trained model generated by learning an image in which a person is recorded and the person. The control unit 11 recognizes one or more persons recorded in the second image information, for example, based on the second image information and the third learned model. The control unit 11 acquires the number of persons recorded in the second image information (number of persons), for example, based on the number of recognized persons.

In step ST104, the control unit 11 learns the second image information acquired in step ST101, the number of people estimated in step ST102, and the number of people estimated in step ST103 to generate a first trained model. In this case, the control unit 11 may further learn which of the number of people estimated in step ST102 and the number of people estimated in step ST103 is more accurate.

Note that steps ST101 to ST104 described above may be performed by a device outside the information processing device 1 (external device). In this case, the information processing apparatus 1 acquires the first trained model generated in step ST104, the second trained model used in step ST102, and the third trained model used in step ST103. good too.

Next, an example of a number-of-people estimation method will be described as an information processing method according to an embodiment.
FIG. 4 is a second flowchart for explaining the information processing method according to one embodiment.

In step ST201, the reception unit 12 receives first image information in which a person is recorded.

In step ST202, the estimation unit 13 estimates the number of persons recorded in the first image information based on the first learned model and the first image information received in step ST201. The first trained model may be, for example, the trained model generated in step ST104, and is generated by learning an image in which a plurality of persons are recorded and the number of persons recorded in the image. It may be a trained model that is

In step ST203, the control section 11 (for example, the estimation section 13, etc.) selects one of the first acquisition section 15 and the second acquisition section 16 based on the estimation result in step ST202. That is, the control unit 11 (for example, the estimation unit 13 or the like) estimates the number of people using the second learned model (first acquisition unit 15), estimates the number of people using the third learned model (second acquisition unit 16 ), the function that can more accurately estimate the number of people is selected according to the estimation result of step ST202. When the second trained model (first acquisition unit 15) is selected, the process proceeds to step ST204. When the third trained model (second acquisition unit 16) is selected, the process proceeds to step ST205.

In step ST204, the first acquisition unit 15 acquires the number of persons recorded in the first image information based on the second trained model and the first image information received in step ST201.
The second trained model may be, for example, a trained model generated by learning a heat map indicating the presence or absence of persons and the number of persons in the heat map. That is, for example, the second trained model can determine the presence of a person between a location where the presence of a person is highly likely to be estimated and a location where the possibility of a person is unlikely to be estimated. It may be a trained model that has learned a heat map indicating the presence/absence. In other words, the second trained model, for example, determines the presence of a person between a location where the possibility of a person being estimated is high and a location where a person is unlikely to be estimated. It may be a trained model generated by learning a heat map indicating the presence or absence of.
The first acquisition unit 15 may generate a heat map indicating the presence or absence of a person based on the first image information received in step ST201 and the characteristics of the person specified in advance. For example, after generating the heat map as described above, the first acquisition unit 15 may estimate the number of people based on the heat map and the second learned model.

In step ST205, the second acquisition unit 16 recognizes each person recorded in the first image information based on the third learned model and the first image information received in step ST201, and to get the number of people. The third trained model may be, for example, a trained model for recognizing a person based on an image. That is, the third trained model may be, for example, a trained model generated by learning an image in which a person is recorded and the person.

In step ST206, the output control section 17 may control the output section to output the estimation result in step ST204 and the estimation result in step ST205. The output unit may be, for example, the communication unit 21, the storage unit 22, the display unit 23, and the like.

[Modification]
Next, a modified example of this embodiment will be described.

(Modification 1)
First, a first modified example will be described. In the first modified example, a “selection unit 14” described later may be arranged instead of the “estimation unit 13” described above. Alternatively, the “estimating unit 13” and the “selecting unit 14” described later may be collectively configured as one function.

In the first modified example, the reception unit 12 may have the same configuration as the configuration described above.

The selection unit 14 selects which one of the first acquisition unit 15 and the second acquisition unit 16 should estimate a person based on the first trained model and the first image information received by the reception unit 12 . In this case, the first trained model learns, for example, an image in which a plurality of people are recorded, and the selection of either estimation of the number of people using a heat map or estimation of the number of people using person recognition. It may be a trained model generated by That is, the first trained model includes, for example, an image in which a person is recorded, the number of people estimated by inputting the image into a second trained model described later, and the third trained model described later using the image. It may be a trained model generated by learning the number of people estimated using the number of people. In this case, for example, the first trained model further determines which of the population estimation using the second trained model and the population estimation using the third trained model is more correct (the appropriate estimation result is obtained). Either the first acquisition unit 15 or the second acquisition unit 16 may acquire a learned model that has been learned. The selection unit 14 uses the first learned model to estimate the number of persons recorded in the first image information (the number of people). output one of

When the first acquisition unit 15 is output (selected) by the selection unit 14, the first acquisition unit 15 records the first image information based on the second trained model and the first image information. Estimate the number of people (number of people). That is, when the selection unit 14 selects the first acquisition unit 15, the first acquisition unit 15 generates a The number of persons recorded in the first image information is acquired based on the second learned model that is obtained and the first image information that is received by the receiving unit 12 . The second trained model may have the same configuration as described in the above embodiment.

When the second acquisition unit 16 is output (selected) by the selection unit 14, the second acquisition unit 16 records the first image information based on the third trained model and the first image information. recognize the person The second acquisition unit 16 estimates the number of persons recorded in the first image information (number of persons) based on the number of persons recognized as such. That is, when the selection unit 14 selects the second acquisition unit 16, the second acquisition unit 16 includes an image in which a person is recorded, a third learned model generated by learning the person, Each person recorded in the first image information is recognized based on the first image information received by the unit 12, and the number of persons is acquired based on the recognition result. The third trained model may have the same configuration as described in the above embodiment.

(Modification 2)
Next, a second modified example will be described.
In the embodiment and the first modified example described above, the number of persons recorded in the first image information is estimated. It may be the contents of various objects without limitation. The contents of various objects may be recorded in the image information (third image information), and examples include indoor and indoor situations, and seasonal situations (spring, summer, autumn and winter). good too.

The reception unit 12 receives third image information in which an object is recorded. The reception unit 12 may have a configuration similar to that of the embodiment described above.

The estimation unit 13 estimates the content recorded in the third image information based on the fourth trained model and the third image information received by the reception unit 12 . The fourth trained model may be, for example, a trained model generated by learning an image in which a plurality of objects are recorded and the content of the objects recorded in the image. That is, the fourth trained model is, for example, a plurality of patterns of images (plurality of images) in which one or more objects are recorded, and the content of the object recorded in each pattern image. It may be a finished model.

Further, in the second modification, the selection unit 14 may be arranged instead of the estimation unit 13, as in the first modification described above. That is, based on the fourth trained model and the third image information received by the receiving unit 12, the selecting unit 14 estimates the content of the object by either the first obtaining unit 15 or the second obtaining unit 16. may be selected. The fourth trained model in this case is, for example, an image in which one or more objects are recorded, and either state estimation of the content of the object using a heat map or state estimation of the content using recognition of the object. It may be a trained model generated by learning selection. That is, the fourth trained model includes, for example, an image in which one or more objects are recorded, the state of the content of the object estimated by inputting the image into a fifth trained model described later, and the image. It may be a trained model generated by learning the state of the content of the object estimated using a sixth trained model described later. In this case, the fourth trained model further determines which of the estimation of the state of the object content using the fifth trained model and the estimation of the state of the object content using the sixth trained model is more correct. Alternatively, it may be a learned model (either the first acquisition unit 15 or the second acquisition unit 16 that provides an appropriate estimation result).

When the content of the object estimated by the estimation unit 13 is the first, the first acquisition unit 15 obtains the third image information based on the fifth trained model and the third image information received by the reception unit 12. Get the content of the object being recorded. The fifth trained model may be, for example, a trained model generated by learning a heat map indicating the state of the contents of the object and the contents of the object in the heat map.
Further, when the first acquisition unit 15 is output (selected) by the selection unit 14, the first acquisition unit 15 obtains the fourth image information based on the fifth trained model and the third image information. Infer the content (state) of the object being recorded.

The second acquisition unit 16, in a second case where the content of the object estimated by the estimation unit 13 is different from the first case, based on the sixth trained model and the third image information received by the reception unit 12 , each object recorded in the third image information is recognized, and the content of the object is acquired based on the recognition result. The sixth trained model may be, for example, a trained model generated by learning an image in which an object is recorded and the object.
Further, when the second acquisition unit 16 is output (selected) by the selection unit 14, the second acquisition unit 16 obtains the third image information based on the sixth trained model and the third image information. Recognize the object being recorded. The second acquisition unit 16 estimates the content (state) of the object recorded in the first image information based on the object thus recognized.

Each unit of the information processing apparatus 1 described above may be implemented as a function of an arithmetic processing unit of a computer or the like. That is, the receiving unit 12, the estimating unit 13, the selecting unit 14, the first obtaining unit 15, the second obtaining unit 16, and the output control unit 17 (control unit 11) of the information processing device 1 are received by an arithmetic processing unit of a computer. They may each be implemented as a function, an estimation function, a selection function, a first acquisition function, a second acquisition function, and an output control function (control function).
The information processing program can cause the computer to implement each function described above. The information processing program may be recorded in a non-temporary computer-readable recording medium such as a memory, solid state drive, hard disk drive, or optical disc.
Further, as described above, each part of the information processing device 1 may be realized by an arithmetic processing device of a computer or the like. The arithmetic processing unit or the like is configured by an integrated circuit or the like, for example. Therefore, each part of the information processing device 1 may be implemented as a circuit that constitutes an arithmetic processing device or the like. That is, the reception unit 12, the estimation unit 13, the selection unit 14, the first acquisition unit 15, the second acquisition unit 16, and the output control unit 17 (control unit 11) of the information processing device 1 constitute an arithmetic processing unit of a computer. A receiving circuit, an estimating circuit, a selecting circuit, a first obtaining circuit, a second obtaining circuit, and an output control circuit (control circuit).
Further, the communication unit 21, the storage unit 22, and the display unit 23 (output unit) of the information processing device 1 may be implemented as, for example, a communication function including functions of an arithmetic processing unit, a storage function, and a display function (output function). good. Further, the communication unit 21, the storage unit 22, and the display unit 23 (output unit) of the information processing device 1 are realized as a communication circuit, a storage circuit, and a display circuit (output circuit) by being configured by an integrated circuit or the like, for example. may Further, the communication unit 21, the storage unit 22, and the display unit 23 (output unit) of the information processing device 1, for example, are configured as a communication device, a storage device, and a display device (output device) by being configured by a plurality of devices. may

The information processing apparatus 1 can combine any one of the plurality of units described above or an arbitrary plurality thereof.
Although the term "information" is used in this disclosure, the term "information" can be interchanged with "data" and the term "data" can be interchanged with "information."

[Aspects and effects of the present embodiment]
Next, one aspect of the present embodiment and effects produced by each aspect will be described. It should be noted that the present embodiment is not limited to each aspect described below, and may be realized by appropriately combining each unit described above. Also, the effects described below are merely examples, and the effects of each aspect are not limited to those described below.

(Aspect 1)
An information processing apparatus according to one aspect is generated by learning a reception unit that receives first image information in which a person is recorded, an image in which a plurality of persons are recorded, and the number of persons recorded in the image. an estimating unit for estimating the number of persons recorded in the first image information based on the first trained model received by the receiving unit and the first image information received by the receiving unit; In this case, based on a second trained model generated by learning a heat map indicating the presence or absence of a person, the number of people in the heat map, and the first image information received by the reception unit, When the number of people estimated by the first acquisition unit that acquires the number of people recorded in the first image information and the estimation unit is less than a threshold value, the image is generated by learning the image in which the people are recorded and the people. second acquisition for recognizing each person recorded in the first image information based on the third learned model received by the reception unit and the first image information received by the reception unit, and acquiring the number of persons based on the recognition result; and
Thereby, the information processing device can estimate the number of persons (number of people) based on the first image information.
In addition, when the number of people recorded in the first image information (number of people) is relatively large, it is more accurate to estimate the number of people using a heat map (using the second trained model). The number of people can be output. On the other hand, when the number of people recorded in the first image information (number of people) is relatively small, estimating the number of people using person recognition (third trained model) outputs a more accurate number of people. be able to. Therefore, since the information processing apparatus selects and uses one of the first acquisition unit and the second acquisition unit according to the number of people recorded in the first image information, it is possible to estimate the number of people more accurately.

(Aspect 2)
In the information processing device according to one aspect, the first acquisition unit obtains a Alternatively, the number of persons may be estimated using a second trained model that has learned a heat map indicating the presence or absence of persons.
Accordingly, even when the number of persons recorded in the first image information (number of persons) is relatively large, the information processing apparatus can acquire the number of persons more accurately using the heat map.

(Aspect 3)
In the information processing device according to one aspect, the first acquisition unit generates a heat map indicating the presence or absence of the person based on the first image information received by the reception unit and the characteristics of the person specified in advance. The number of persons may be estimated based on the heat map and the second trained model.
Accordingly, even when the number of persons recorded in the first image information (number of persons) is relatively large, the information processing apparatus can acquire the number of persons more accurately using the heat map.

(Aspect 4)
In the information processing device of one aspect, the estimating unit includes the number of persons estimated based on the second image information in which persons are recorded and the second trained model, and the number of persons estimated based on the second image information and the third trained model. The number of persons may be estimated using a first trained model generated by learning the number of persons estimated by the second image information and the number of persons recorded in the second image information.
Accordingly, the information processing apparatus can estimate which of the first acquisition unit and the second acquisition unit is more appropriate for estimating the number of people, based on the first image information. That is, the information processing apparatus can select a function more suitable for estimating the number of people, from the first acquisition unit and the second acquisition unit.

(Aspect 5)
An information processing apparatus according to one aspect includes a reception unit that receives first image information in which persons are recorded, an image in which a plurality of persons are recorded, estimation of the number of people using a heat map, and estimation of the number of people using recognition of the people. Based on the first trained model generated by learning any selection of estimation and the first image information received by the reception unit, either the first acquisition unit or the second acquisition unit identifies the person. A selection unit that selects whether to estimate, and when the selection unit selects the first acquisition unit, a heat map indicating the presence or absence of a person and the number of people in the heat map are generated by learning. a first acquisition unit for acquiring the number of persons recorded in the first image information based on the second learned model received by the reception unit and the first image information received by the reception unit; is recorded in the first image information based on the image in which the person is recorded, the third trained model generated by learning the person, and the first image information received by the reception unit a second acquisition unit that recognizes each person and acquires the number of persons based on the recognition result.
Thereby, the information processing apparatus can achieve the same effect as the above-described aspect.

(Aspect 6)
An information processing apparatus according to one aspect is generated by learning a reception unit that receives third image information in which an object is recorded, an image in which a plurality of objects are recorded, and the content of the objects recorded in the image. an estimating unit for estimating the content recorded in the third image information based on the fourth trained model, the third image information received by the receiving unit, and the content of the object estimated by the estimating unit is the first based on a fifth trained model generated by learning a heat map indicating the state of the content of the object, the content of the object in the heat map, and the third image information received by the receiving unit, a first acquiring unit for acquiring the content of the object recorded in the third image information; and an image in which the object is recorded in a second case where the content of the object estimated by the estimating unit is different from the first case. , the object, and the third image information received by the reception unit, each of the objects recorded in the third image information is recognized, and based on the recognition result and a second obtaining unit for obtaining the content of the object.
Thereby, the information processing apparatus can achieve the same effect as the above-described aspect.

(Aspect 7)
In one aspect of the information processing method, the computer learns a receiving step of receiving first image information in which persons are recorded, an image in which a plurality of persons are recorded, and the number of persons recorded in the image. an estimation step for estimating the number of people recorded in the first image information based on the first trained model generated by and the first image information received by the reception step; If it is equal to or greater than the threshold, the second trained model generated by learning the heat map indicating the presence or absence of the person and the number of people in the heat map, and the first image information received by the receiving step. a first acquiring step of acquiring the number of persons recorded in the first image information, and learning an image in which the persons are recorded and the persons when the number of persons estimated by the estimating step is less than a threshold based on the Recognizing each person recorded in the first image information based on the third learned model generated by and the first image information received by the receiving step, and acquiring the number of persons based on the recognition result and a second obtaining step.
Accordingly, the information processing method can achieve the same effect as the information processing apparatus of one aspect described above.

(Aspect 8)
An information processing program according to one aspect provides a computer with a reception function for receiving first image information in which a person is recorded, an image in which a plurality of persons are recorded, and the number of persons recorded in the image. An estimation function for estimating the number of people recorded in the first image information based on the first trained model generated by and the first image information received by the reception function, and the number of people estimated by the estimation function If it is equal to or greater than the threshold, the second trained model generated by learning the heat map indicating the presence or absence of the person and the number of people in the heat map, and the first image information received by the reception function a first acquisition function for acquiring the number of persons recorded in the first image information, and an image in which the persons are recorded when the number estimated by the estimation function is less than a threshold value, and the persons are learned based on the Recognize each person recorded in the first image information based on the third learned model generated by and the first image information received by the reception function, and acquire the number of persons based on the recognition result a second acquisition function;
Accordingly, the information processing program can achieve the same effect as the information processing apparatus of one aspect described above.

1 information processing device 11 control unit 12 reception unit 13 estimation unit 14 selection unit 15 first acquisition unit 16 second acquisition unit 17 output control unit 21 communication unit 22 storage unit 23 display unit

Claims (7)

a reception unit that receives first image information in which a person is recorded;
Based on a first trained model generated by learning an image in which a plurality of persons are recorded and the number of persons recorded in the image, and the first image information received by the reception unit, a first an estimation unit for estimating the number of persons recorded in one piece of image information;
a second trained model generated by learning a heat map indicating presence/absence of persons and the number of persons in the heat map when the number of persons estimated by the estimation unit is equal to or greater than a threshold; a first acquisition unit that acquires the number of persons recorded in the first image information based on the first image information received by the reception unit;
When the number of people estimated by the estimation unit is less than the threshold, an image in which people are recorded, a third trained model generated by learning the people, and first image information received by the reception unit. a second acquisition unit that recognizes each person recorded in the first image information based on and acquires the number of persons based on the recognition result;
Information processing device. The first acquisition unit determines the presence or absence of a person between a range of a location where the existence of a person is highly likely to be estimated and a location where the existence of a person is unlikely to be estimated. 2. The information processing apparatus according to claim 1, wherein the number of persons is estimated by using a second trained model obtained by learning the heat map shown. The first acquisition unit generates a heat map indicating presence/absence of a person based on first image information received by the reception unit and characteristics of a person specified in advance. 3. The information processing apparatus according to claim 1, wherein the number of persons is estimated based on the learned model. The estimation unit includes: the number of people estimated based on second image information in which people are recorded and the second trained model; the number of people estimated based on the second image information and the third trained model; 4. The information processing apparatus according to any one of claims 1 to 3, wherein the number of persons is estimated by using a first trained model generated by learning the number of persons recorded in the second image information. . a reception unit that receives first image information in which a person is recorded;
a first trained model generated by learning an image in which a plurality of people are recorded, and selection of one of estimation of the number of people using a heat map and estimation of the number of people using recognition of people; a selection unit that selects whether to estimate a person by either the first acquisition unit or the second acquisition unit based on the first image information received by the reception unit;
a second trained model generated by learning a heat map indicating presence/absence of a person and the number of persons in the heat map when the first acquisition unit is selected by the selection unit; the first acquisition unit for acquiring the number of persons recorded in the first image information based on the first image information received by the reception unit;
When the second acquisition unit is selected by the selection unit, an image in which a person is recorded, a third trained model generated by learning the person, and first image information received by the reception unit. a second acquisition unit that recognizes each person recorded in the first image information based on the above, and acquires the number of people based on the recognition result;
Information processing device. the computer
a receiving step of receiving first image information in which a person is recorded;
Based on a first trained model generated by learning an image in which a plurality of persons are recorded and the number of persons recorded in the image, and the first image information received in the receiving step, a first an estimation step of estimating the number of persons recorded in one piece of image information;
a second trained model generated by learning a heat map indicating the presence or absence of persons and the number of persons in the heat map when the number of persons estimated by the estimation step is equal to or greater than a threshold; a first obtaining step of obtaining the number of persons recorded in the first image information based on the first image information received by the receiving step;
When the number of people estimated by the estimation step is less than the threshold, an image in which people are recorded, a third trained model generated by learning the people, and the first image information received by the reception step. a second acquiring step of recognizing each person recorded in the first image information based on and acquiring the number of persons based on the recognition result;
Information processing method that performs to the computer,
a receiving function for receiving first image information in which a person is recorded;
Based on a first trained model generated by learning an image in which a plurality of persons are recorded and the number of persons recorded in the image, and the first image information received by the reception function, a first an estimation function for estimating the number of persons recorded in one image information;
a second trained model generated by learning a heat map indicating the presence or absence of persons and the number of persons in the heat map when the number of persons estimated by the estimation function is equal to or greater than a threshold; a first acquisition function for acquiring the number of persons recorded in the first image information based on the first image information received by the reception function;
When the number of people estimated by the estimation function is less than the threshold, an image in which people are recorded, a third trained model generated by learning the people, and first image information received by the reception function. a second acquisition function for recognizing each person recorded in the first image information based on and acquiring the number of persons based on the recognition result;
Information processing program that realizes

Images