JP7393270B2 - Information processing device, information processing method, and information processing program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

2. Description of the Related Art Conventionally, there has been a technique for tracking a subject recorded in an image captured by a camera. The technology described in Patent Document 1 detects a subject from frames captured at the same time by each of a plurality of cameras, and calculates the position of the object in real space based on the position of the object within the frame. The technique described in Patent Document 1 predicts the position of a subject in the next frame based on the calculated position of the subject.

JP 2019-109765 Publication

The technique described in Patent Document 1 tracks a subject recorded in a frame, and cannot acquire any other information other than tracking the subject. Further, the technique described in Patent Document 1 requires a plurality of cameras, and thus requires the cost and effort of installing the plurality of cameras. Furthermore, depending on the circumstances of the installation location, it may not be possible to install multiple cameras.

An object of the present invention is to provide an information processing device, an information processing method, and an information processing program that can acquire information regarding a subject recorded in an image.

An information processing device according to one embodiment includes a fixed-point imaging unit that temporally continuously images a subject to generate image data, and at least two temporally different images recorded in the image data captured by the fixed-point imaging unit. a recognition unit that recognizes each subject based on the subject and generates a first recognition frame surrounding the subject at each time, a plurality of first recognition frames, a center position of the plurality of first recognition frames, and a fixed-point imaging unit. and a reference line of the image based on the image data captured by the recognition unit, the second recognition encloses the subject at a predetermined time that is temporally earlier than the recording time of the image when the first recognition frame is generated by the recognition unit. a prediction unit that predicts the frame and the center position of the second recognition frame; a front frame that corresponds to the outer shape of the front of the subject, and a front frame that partially touches the second recognition frame generated by the prediction unit; A generation unit that generates a back frame according to the outer shape of the back surface, a distance from a fixed-point imaging unit to a subject, an image size based on image data, and at least one of the front frame and the back frame generated by the generation unit. and a size acquisition unit that acquires the size of the subject based on the size.

An information processing device according to one aspect includes a center of gravity acquisition unit that acquires a center of gravity position of a subject based on a front frame and a back frame generated by a generation unit, and a center of gravity position acquired by the center of gravity acquisition unit in a predetermined coordinate system. The camera may also include a position acquisition unit that acquires the position of the subject by converting the image.

The information processing device in one embodiment may include a speed acquisition unit that acquires the speed of the subject based on changes in a plurality of positions of the center of gravity of the subject over time acquired by the center of gravity acquisition unit.

In one embodiment, the information processing device outputs at least one of the size of the subject acquired by the size acquisition unit, the position of the subject acquired by the position acquisition unit, and the speed of the subject acquired by the speed acquisition unit. It is also possible to include an output section that does this.

In one embodiment of the information processing device, the recognition unit may generate a rectangular frame surrounding the subject on the image as the first recognition frame.

In one embodiment of the information processing device, the prediction unit may generate a rectangular frame surrounding the subject on the image as the second recognition frame.

In one aspect of the information processing device, the prediction unit sets a line extending in the horizontal direction of an image based on image data as a reference line, sets a predetermined position of the reference line as an end point, and performs a plurality of first recognition processes from the end point. The second recognition frame may be generated with the center position being on the center line passing through each center position of the frame.

In one embodiment of the information processing device, when the subject is located on the left side in the vertical direction of the image based on the image data with respect to the predetermined position as a reference, the generation unit generates a vertex at the lower left of the second recognition frame with respect to the image. A rectangular front frame may be generated so as to be in contact with the second recognition frame, and a rectangular back frame may be generated so as to be in contact with the top right vertex of the second recognition frame with respect to the image.

In one embodiment of the information processing device, when the subject is located on the right side in the vertical direction of the image based on the image data with respect to the predetermined position as a reference, the generation unit may be configured to A rectangular front frame may be generated so as to be in contact with the vertex, and a rectangular back frame may be generated so as to be in contact with the apex at the upper left of the second recognition frame with respect to the image.

In one embodiment of the information processing device, the size acquisition unit may perform the following steps based on a result of associating in advance a position in the image based on image data generated by the fixed-point imaging unit and a distance from the fixed-point imaging unit at the position. The distance from the fixed-point imaging unit to the subject may also be acquired.

In one aspect of the information processing method, a computer sequentially images a subject to generate image data; and at least two temporal a recognition step of recognizing each subject based on different images and generating a first recognition frame surrounding the subject at each time; a plurality of first recognition frames; and a center position of the plurality of first recognition frames; enclosing the subject at a predetermined time that is temporally earlier than the recording time of the image when generating the first recognition frame in the recognition step, based on the reference line of the image based on the image data taken in the fixed-point imaging step; a prediction step of predicting a second recognition frame and a center position of the second recognition frame; and a prediction step of predicting a second recognition frame and a center position of the second recognition frame; , a generation step that generates a back frame according to the outer shape of the back of the subject, a distance from the fixed point imaging step to the subject, an image size based on image data, and a front frame and a back frame generated in the generation step. and a size obtaining step of obtaining the size of the subject based on at least one of the sizes.

An information processing program in one embodiment provides a computer with a fixed point imaging function that temporally continuously images a subject to generate image data, and at least two temporal a recognition function that recognizes each subject based on different images and generates a first recognition frame surrounding the subject at each time; a plurality of first recognition frames; and a center position of the plurality of first recognition frames; enclosing the subject at a predetermined time that is temporally earlier than the recording time of the image when the first recognition frame is generated by the recognition function, based on the reference line of the image based on the image data captured by the fixed-point imaging function; A prediction function that predicts a second recognition frame and the center position of the second recognition frame, and a front frame that corresponds to the front outline of the subject so that a part of the frame is in contact with the second recognition frame generated by the prediction function. , a generation function that generates a back frame according to the external shape of the back of the subject, the distance from the fixed point imaging function to the subject, the size of the image based on image data, and the front frame and back frame generated by the generation function. and a size acquisition function for acquiring the size of a subject based on at least one of the sizes.

An information processing device according to one aspect includes a recognition unit that generates a first recognition frame surrounding a subject based on at least two temporally different images, a plurality of first recognition frames, and a plurality of first recognition frames. , and the reference line of the image based on the image data captured by the fixed-point imaging unit, at a predetermined time that is temporally earlier than the recording time of the image when the recognition unit generates the first recognition frame. a prediction unit that predicts a second recognition frame surrounding the subject at a time and a center position of the second recognition frame, and generates a front frame and a back frame according to the front outline of the subject according to the second recognition frame. Obtaining the size of the subject based on the generation unit, the distance from the fixed-point imaging unit to the subject, the size of the image based on the image data, and the size of at least one of the front frame and the back frame generated by the generation unit. Since the size acquisition unit is provided, information regarding the subject recorded in the image can be acquired.
Further, the information processing method and the information processing program of one embodiment can achieve the same effects as the information processing apparatus of one embodiment.

FIG. 1 is a block diagram for explaining an information processing device according to an embodiment. FIG. 3 is a diagram for use in explaining the case of acquiring image position information. FIG. 3 is a diagram for explaining an example of an image captured by a fixed-point imaging unit. It is a graph showing the relationship between the Y coordinate and the number of pixels. It is a graph for explaining an example of image position information. It is a graph showing the relationship between the Y coordinate and the width ratio of an image. FIG. 3 is a block diagram for explaining first to ninth setting units. It is a figure for explaining an example of a 1st recognition frame and a 2nd recognition frame. FIG. 6 is a diagram for explaining the center positions of each of the first recognition frame and the second recognition frame. It is a figure for explaining an example of a front frame and a back frame. FIG. 3 is a diagram for explaining the center of gravity position. FIG. 6 is a diagram for explaining a change in the center of gravity position when acquiring the speed of a subject. 1 is a flowchart illustrating a method of acquiring image position information, which is an information processing method according to an embodiment. 1 is a flowchart illustrating an information processing method according to an embodiment, which is a method of acquiring information regarding a subject.

An embodiment of the present invention will be described below.
Although the wording "information" is used in this specification, the wording "information" can be rephrased as "data" and the wording "data" can be rephrased as "information."

FIG. 1 is a block diagram for explaining an information processing device 1 according to an embodiment.
The information processing device 1 images a subject 201 (see FIGS. 8 and 10) based on image data generated by capturing moving images or temporally continuous still images using the fixed-point imaging unit 21. The subject 201 is, for example, an object that moves in a constant manner. Objects that move in a constant manner also include objects that move in a substantially constant manner, such as vehicles, people, animals, etc. that move linearly (substantially straight). As a specific example, the subject 201 may be an object moving on a straight road or the like. Note that the subject 201 is not limited to a moving object, but may be a stationary object (for example, a vehicle, a person, an animal, etc.).

For example, the information processing device 1 performs object recognition on the same subject 201 that is imaged by the fixed-point imaging unit 21 and that differs in time, and creates a recognition frame (first recognition frame 210 (see FIG. 8)) surrounding the subject 201 ( A plurality of ROIs (Regions of Interest) are generated for each time. The information processing device 1 uses a plurality of first recognition frames 210 to determine the time (for example, past time) at which an image (frame or still image) was captured as a basis for generating the first recognition frames 210. ) is predicted, and a recognition frame (second recognition frame 220 (see FIG. 8)) is generated at the predicted position. The information processing device 1 uses the first recognition frame 210 and the second recognition frame 220 to determine the size of the subject 201 (for example, the length in the vertical, horizontal and depth directions) and the position of the subject 201 (for example, the current , past position, etc.), the distance from the fixed-point imaging unit 21 to the subject 201, and the speed of the subject 201.

The information processing device 1 will be described in detail below.
The information processing device 1 includes a fixed point imaging section 21, a storage section 22, a setting section 12, a recognition section 13, a prediction section 14, a generation section 15, a size acquisition section 16, a center of gravity acquisition section 17, a position acquisition section 18, and a speed acquisition section 19. and an output control section 20. The setting unit 12 , the recognition unit 13 , the prediction unit 14 , the generation unit 15 , the size acquisition unit 16 , the center of gravity acquisition unit 17 , the position acquisition unit 18 , the speed acquisition unit 19 and the output control unit 20 are the control unit 11 of the information processing device 1 (As an example, it may be realized as a function of an arithmetic processing device). In addition to the storage section 22 described above, a communication section 23 and a display section 24, which will be described later, may constitute an embodiment of the "output section" of the present invention.

The fixed-point imaging unit 21 temporally continuously images a subject and generates image data. The fixed-point imaging unit 21 generates moving image data by capturing moving images or continuous images at predetermined time intervals. The fixed point imaging unit 21 is arranged at a predetermined location and captures an image in a predetermined direction.
The fixed-point imaging unit 21 is installed on a pole or the like fixed to the ground, or on a movable tripod or the like. When the fixed point imaging unit 21 is installed on a movable tripod or the like, it can be moved together with the tripod. Thereby, the fixed-point imaging unit 21 can be easily placed at a location desired by the user, and can be placed at a desired location only for a period required by the user.

The fixed-point imaging unit 21 is supplied with power, for example, from at least one of a power generation panel (not shown) or a power source (not shown).
Power generation panels generate electricity by receiving light. The power generation panel may be a device that utilizes energy harvesting technology, ie, a device that harvests energy from the surrounding environment and converts it into electricity. As a specific example, the power generation panel may be a solar panel. The power generated by the power generation panel may be stored in, for example, a battery, and the power may be supplied to the fixed-point imaging unit 21 from the battery.
The power source is, for example, a primary battery, a secondary battery, a commercial power source, or the like. When a primary battery or a secondary battery is used as a power source, the primary battery or secondary battery is arranged in the fixed-point imaging unit 21 so as to be replaceable. Further, when a secondary battery is used as a power source, the secondary battery may be arranged in the fixed-point imaging section 21 so as to be rechargeable.
Note that the power generation panel or the power source may supply power to the control unit and the like in addition to the fixed-point imaging unit 21.

The storage unit 22 is a device that stores various information, programs, and the like. The storage unit 22 stores a correspondence relationship (learning model) that associates a subject with a feature of the subject. The correspondence relationship is used when the recognition unit 13 recognizes the subject. The correspondence relationship is generated by performing deep learning or the like, for example. That is, the correspondence relationship is generated by learning the characteristics of the object based on deep learning or the like, and by associating the relationship between the object and the characteristics of the object. The learning model may be generated by the control unit 11 based on image data generated by the fixed-point imaging unit 21, and may be acquired from an external device (for example, a server, etc.) (not shown) via the communication unit 23, which will be described later. You may.
The storage unit 22 also stores a relationship (an image location information). A method for generating image position information will be described later.

The communication unit 23 is a device capable of transmitting and receiving information to and from an external device (for example, a server, etc.) (not shown).

The setting unit 12 performs preliminary settings in order to obtain the size of the subject 201, the position of the subject 201, the distance from the fixed-point imaging unit 21 to the subject 201, and the speed of the subject 201 in each unit (function) described later. conduct. That is, a relationship is set in which a position in an image (frame or still image) based on image data generated by the fixed-point imaging section 21 is associated in advance with a distance from the fixed-point imaging section 21 at that position.

First, a configuration for acquiring image position information will be described. By acquiring image position information in advance, the information processing device 1 obtains subject information regarding the subject 201, such as the size of the subject 201, the position of the subject 201, and the distance from the fixed-point imaging unit 21 to the subject 201, as described later. It becomes possible to obtain the speed of the subject 201.

First, an object (for example, a person, etc.) is moved within the imaging range of the fixed-point imaging unit 21. In the following, a "person" will be described as an example of an object for which settings are made by the setting unit 12. At this time, the setting unit 12 sets image position information in the image data based on the height of the person recorded in the image data generated by the fixed-point imaging unit 21 and the angle of view of the fixed-point imaging unit 21. That is, the setting unit 12 acquires information regarding a position within the image (image position information) based on the image captured by the fixed-point imaging unit 21. Specifically, when the recognition unit 13 (described later) recognizes a person based on the image data captured by the fixed-point imaging unit 21, the setting unit 12 determines the height of the person and the angle of view of the fixed-point imaging unit 21. , acquires positional information (image position information (information related to the distance from the fixed-point imaging unit 21 to the person)) in the image captured by the fixed-point imaging unit 21.

FIG. 2 is a diagram for use in explaining the case of acquiring image position information.
FIG. 3 is a diagram for explaining an example of an image captured by the fixed-point imaging unit 21.
FIG. 4 is a graph showing the relationship between the Y coordinate and the number of pixels. The horizontal axis in FIG. 4 indicates the Y coordinate, and the vertical axis indicates the number of pixels per 1 m. Here, the depth direction in the image captured by the fixed-point imaging unit 21 is the Y coordinate, and the width direction of the image is the X coordinate.
FIG. 5 is a graph for explaining an example of image position information. The horizontal axis in FIG. 5 indicates the Y coordinate, and the vertical axis indicates the distance from the ground position where the fixed-point imaging unit 21 is arranged to an arbitrary Y coordinate position.
FIG. 6 is a graph showing the relationship between the Y coordinate and the width ratio of the image. The horizontal axis in FIG. 6 shows the Y coordinate, and the vertical axis shows the ratio of the width of the image.
FIG. 7 is a block diagram for explaining the first to ninth setting units 121 to 129.

Here, in FIG. 2, the Y-axis direction (Y-coordinate direction) is the depth direction. Further, in FIG. 2, when a person 110 at a predetermined position in the Y-axis direction (Y-coordinate direction) is imaged by the fixed-point imaging unit 21, the image at the predetermined position is indicated by reference numeral 100a. The width (size in the X-axis direction) of the image 100a is indicated by the symbol w. If the center position of the image 100a is designated by the symbol Pc, the distance from the fixed point imaging unit 21 to the center position Pc is a first distance L1, and the distance from the center position Pc to the ground is a second distance L2. Further, the angle between the ground and a straight line along the Y-axis direction from the fixed-point imaging unit 21 to a position corresponding to the ground in the image 100a is denoted by θ. Furthermore, the height of the fixed-point imaging unit 21 from the ground is defined as H, the ground position at which the fixed-point imaging unit 21 is arranged is defined as P0, and the distance from the ground position P0 along the Y-axis direction to the image 100a is a third Let the distance be L3.
The setting section 12 obtains image position information by obtaining the third distance L3 and associating the position in the image captured by the fixed point imaging section 21 with the third distance L3.

Specifically, as shown in FIG. 7, the setting section 12 includes a first setting section 121, a second setting section 122, a third setting section 123, a fourth setting section 124, a fifth setting section 125, and a sixth setting section 122. section 126 , a seventh setting section 127 , an eighth setting section 128 , and a ninth setting section 129 .

The first setting unit 121 obtains in advance the height of the subject, the angle of view of the fixed-point imaging unit 21, and the width of the image captured by the fixed-point imaging unit 21.
In order to obtain image position information, first, the person 110 is moved within the imaging range of the fixed-point imaging unit 21. In this case, for example, the height of the person is input using an input device (not shown). The first setting unit 121 obtains the height of the inputted person.
The first setting unit 121 also obtains the angle of view θ2 taken by the fixed-point imaging unit 21. The angle of view θ2 is input using, for example, an input device (not shown). The first setting unit 121 obtains the input angle of view θ2. The angle of view θ2 is determined by the planar size of the image sensor (not shown) of the fixed-point imaging section 21, the focal length of the imaging lens (not shown) of the fixed-point imaging section 21, and the like.
The first setting unit 121 also acquires the width (for example, the number of pixels) of the image obtained by the fixed-point imaging unit 21. The width of the image (image width) is input using, for example, an input device (not shown).

The second setting unit 122 obtains the number of pixels forming the image data per meter based on the height of the person 110 obtained by the first setting unit 121.
The second setting unit 122 acquires the number of pixels per meter in the image captured by the fixed-point imaging unit 21 at an arbitrary position within the image by making the person 110 walk within the imaging range. That is, the second setting unit 122 obtains the size (length) of 1 m in the image 100a based on the height of the person obtained by the first setting unit 121, and further calculates the size of the image per 1 m obtained Get the number of pixels.
In the case illustrated in FIG. 3, the size of the image sensor used in the fixed-point imaging section 21 (the image obtained by the fixed-point imaging section 21) is 1280 pixels x 960 pixels. When the person 110 (not shown in FIG. 3) is recorded in the image illustrated in FIG. 3, the second setting unit 122 sets the second setting unit 122 to Obtain the number of pixels per meter.
Note that when the person 110 moves in a direction away from the fixed-point imaging unit 21, there is an infinite point P1 (see FIG. 3) at a predetermined position in the image at which the person does not move further away from that position. A horizontal line including the infinite point P1 in the image becomes the reference line LS.

The third setting unit 123 acquires the relationship between the Y coordinate, which is the coordinate in the depth direction of the image based on the image data, and the number of pixels per meter of the subject acquired by the second setting unit 122.
In the example shown in FIG. 3, the vertical direction of the image corresponds to the depth direction of the real space, and the vertical coordinate of the image is the Y coordinate. Further, in the example shown in FIG. 3, the horizontal direction of the image corresponds to the width direction of the real space, and the horizontal coordinate of the image is the X coordinate. If the upper left pixel in FIG. 3 is the coordinate (0,0) of the origin, then the coordinates of the upper right pixel are (1280,0). The coordinates of the pixel at the center position in FIG. 3 are (640, 480). The coordinates of the lower left pixel in FIG. 3 are (0,960), and the coordinates of the lower right pixel are (1280,960).

Within the imaging range of the fixed-point imaging unit 21 (within the range surrounded by coordinates (0,0), coordinates (1280,0), coordinates (1280,960), and coordinates (0,960)), for example, a subject (person 110) ), the third setting unit 123 obtains the number of pixels per meter at a plurality of Y coordinate positions.

As an example is shown in FIG. 4, the relationship between the Y coordinate acquired by the third setting unit 123 and the number of pixels per 1 m is linear. Here, as shown in FIG. 2, the value of the Y coordinate increases as it approaches the fixed-point imaging section 21. That is, as the Y coordinate increases (as the subject approaches the fixed-point imaging unit 21), the number of pixels per meter increases. In other words, as the Y coordinate decreases (as the subject moves away from the fixed-point imaging unit 21), the number of pixels per meter decreases.

The fourth setting section 124 configures the image 100a in which the person 110 is detected at a predetermined position based on the width of the image obtained by the first setting section 121 and the number of pixels per meter obtained by the second setting section 122. Get the actual width w.
As illustrated in FIG. 2, consider an image 100a at a Y-coordinate position (a predetermined position in the Y-coordinate direction) where a person 110 is located. The fourth setting unit 124 determines that the number of pixels in the width direction of the image captured by the fixed-point imaging unit 21 (1280 pixels in the case illustrated in FIG. 3) is acquired by the first setting unit 121, so the number of pixels per 1 m is Based on this, the actual length of the width w of the image 100a at the predetermined position of the Y coordinate is obtained.

The fifth setting unit 125 determines the width of the image 100a from the fixed-point imaging unit 21 based on the actual width w of the image acquired by the fourth setting unit 124 and the angle of view θ2 acquired by the first setting unit 121. A first distance, which is the distance to the center position Pc, is acquired, and a second distance L2, which is the distance from the ground to the center position Pc, is acquired.
The fifth setting unit 125 determines the center position of the image 100a from the fixed-point imaging unit 21 based on the actual length of the width w of the image 100a at the predetermined position of the Y coordinate described above and the angle of view θ2 of the fixed-point imaging unit 21. Obtain the actual distance (first distance L1) to Pc. Further, the fifth setting unit 125 determines, based on the number of pixels of the image, the number of pixels per meter, the actual length of the width w of the image 100a at a predetermined position of the Y coordinate, and the angle of view of the fixed-point imaging unit 21. The actual distance (second distance L2) from the center position Pc of the image 100a to the ground is acquired.

The sixth setting unit 126 determines the angle θ between the center line of the fixed-point imaging unit 21 in the imaging direction and the ground based on the first distance L1 and the second distance L2 acquired by the fifth setting unit 125. That is, the sixth setting section 126 uses trigonometric functions to determine the angle θ between the ground and the fixed-point imaging section 21 based on the first distance L1 and the second distance L2 acquired by the fifth setting section 125. seek.

The seventh setting unit 127 determines whether the fixed-point imaging unit 21 is located from the ground based on the first distance L1 and second distance L2 acquired by the fifth setting unit 125 and the angle θ acquired by the sixth setting unit 126. Get the height H up to. That is, the seventh setting unit 127 uses trigonometric functions to obtain the height H from the ground to the fixed-point imaging unit 21 based on the first distance L1, the second distance L2, and the angle θ.

The eighth setting unit 128 sets the fixed-point imaging unit 21 based on the height H from the ground to the fixed-point imaging unit 21 acquired by the seventh setting unit 127 and the angle θ acquired by the sixth setting unit 126. The image position information is acquired based on acquiring the position from the imaging position where the camera is placed to the subject, and then acquiring a plurality of positions of the subject.
That is, the eighth setting unit 128 determines the position from the ground position P0 where the fixed-point imaging unit 21 is arranged based on the height H from the ground to the fixed-point imaging unit 21 and the angle θ between the fixed-point imaging unit 21 and the ground. A distance L3 to a predetermined Y coordinate position (image 100a) is obtained. The eighth setting unit 128 obtains information regarding the position within the image (image position information) by associating the distance L3 with the Y coordinate at a predetermined position within the image. Further, the eighth setting unit 128 moves the person 110 within the imaging range of the fixed-point imaging unit 21, and moves the subject (person) 110 from a ground position P0 where the fixed-point imaging unit 21 is arranged at an arbitrary position within the imaging range. Image position information is obtained based on obtaining the distance to. The eighth setting unit 128 stores the image position information in the storage unit 22.

The image position information is information indicating how many meters away from the fixed-point imaging unit 21 the position (coordinates) of an arbitrary pixel in the image captured by the fixed-point imaging unit 21 is. As a specific example of the image position information, the pixel coordinates (640, 480) shown in FIG. The information indicates that the pixel coordinates (640, 840) are 20 m away from the fixed-point imaging unit 21.

As shown in FIG. 5, the image position information is information indicating that as the Y coordinate increases, the distance from the ground position P0 where the fixed-point imaging unit 21 is arranged becomes shorter. In other words, the image position information is information indicating that as the Y coordinate becomes smaller, the distance from the ground position P0 where the fixed-point imaging unit 21 is arranged becomes longer.

Here, the ninth setting unit 129 determines the width at the reference position in the Y-coordinate direction and the width at a predetermined position in the Y-coordinate direction among the actual width w of each of the plurality of images acquired by the fourth setting unit 124. The width ratio to the Y coordinate is obtained based on the width.
That is, the fourth setting unit 124 obtains the actual width w of the image. At a position where the person 110 is not present (Y coordinate corresponding to a position where no person is walking), the fourth setting unit 124 cannot acquire the actual width w of the image. Therefore, when the fourth setting unit 124 obtains the width w of the image at a plurality of positions on the Y coordinate, the ninth setting unit 129 selects an arbitrary value on the Y coordinate based on the width w of the image at the plurality of positions. Calculate the width of the image at the position. For example, the ninth setting unit 129 sets one of the widths of the plurality of images acquired by the fourth setting unit 124 as the width of the image at the reference position of the Y coordinate (reference width), and The ratio (width ratio) between the widths of other images excluding the reference width and the reference width is calculated. The ninth setting unit 129 calculates the ratio (width ratio) to the reference width based on each of the plurality of images acquired by the fourth setting unit 124. A graph of this width ratio is shown in FIG. 6.

As illustrated in FIG. 6, as the Y coordinate increases (as the distance from the fixed-point imaging unit 21 decreases), the width ratio of the image decreases. In other words, the width ratio of the image increases as the Y coordinate decreases (as the distance from the fixed-point imaging unit 21 increases).

The fifth setting section 125 described above uses the width ratio of the image obtained by the ninth setting section 129 even at a position where the width w of the image is not obtained by the fourth setting section 124, so that the width w of the image can be set at any Y coordinate. Since the width of the image can be acquired, the first distance L1 and the second distance L2 can be acquired.
This completes the acquisition of image position information.

Next, a configuration for acquiring subject information regarding the subject 201 using the above-mentioned image position information will be described.
FIG. 8 is a diagram for explaining an example of the first recognition frame 210 and the second recognition frame 220.
FIG. 9 is a diagram for explaining the center positions 210a and 220a of the first recognition frame 210 and the second recognition frame 220, respectively.
FIG. 10 is a diagram for explaining an example of the front frame 220b and the back frame 220c.
FIG. 11 is a diagram for explaining the center of gravity position PG.
FIG. 12 is a diagram for explaining changes in the center of gravity position PG when acquiring the speed of the subject 201.

When the subject 201 is recorded in the image data generated by the fixed-point imaging unit 21, the recognition unit 13 recognizes the subject 201 based on the correspondence relationship (learning model) stored in the storage unit 22. That is, the recognition unit 13 extracts the features of the subject 201 from the image based on the image data generated by the fixed-point imaging unit 21 based on the correspondence relationship (learning model), and recognizes the subject 201 from the extracted features. Furthermore, the recognition unit 13 may not only recognize the subject 201 using the learning model, but may also recognize the subject 201 by, for example, performing pattern matching.

The recognition unit 13 recognizes each subject 201 (vehicle in the case shown in FIGS. 8 and 10) based on at least two temporally different images recorded in the image data captured by the fixed-point imaging unit 21, and A plurality of first recognition frames 210 (see FIG. 8) surrounding the subject 201 are generated at the time . In this case, the recognition unit 13 may generate a rectangular frame surrounding the subject 201 on the image as the first recognition frame 210. When a moving image is captured by the fixed-point imaging unit 21, the recognition unit 13 recognizes the subject 201 based on temporally consecutive frames or frames at predetermined time intervals as temporally different images, for example. When a still image is captured by the fixed-point imaging unit 21, the recognition unit 13 determines whether the fixed-point imaging unit 21 captures a still image based on a still image captured continuously in time or a still image captured at predetermined time intervals as a temporally different image. The subject 201 is recognized.

The first recognition frame 210 may be, for example, a rectangular frame surrounding the subject 201, or may be a rectangular frame that touches the outer shape of the subject 201. The recognition unit 13 generates a first recognition frame 210 for each temporally different image (frame or still image). The recognition unit 13 generates a first recognition frame 210 for each subject 201 when a plurality of subjects 201 are recorded in the image.

The prediction unit 14 uses the plurality of first recognition frames 210, the center positions 210a of the plurality of first recognition frames 210, and the reference line LS of the image based on the image data captured by the fixed-point imaging unit 21, to A second recognition frame 220 (see FIG. 8) surrounding the subject 201 at a predetermined time that is temporally earlier than the recording time of the image when the first recognition frame 210 is generated by the recognition unit 13, and the second recognition frame 220 The center position 220a (see FIG. 9) is predicted. That is, as shown in FIG. 9, the prediction unit 14 sets a line extending in the horizontal direction of the image based on the image data as the above-mentioned reference line LS, and sets a predetermined position of the reference line LS as an end point (intersection point PS). However, the second recognition frame 220 may be generated with the center position 220a being on the center line passing through each center position 210a of the plurality of first recognition frames 210 from the end point (intersection point PS). In this case, the prediction unit 14 may generate a rectangular frame surrounding the subject 201 on the image as the second recognition frame 220.

The prediction unit 14 generates a line (first extension line 230) passing through the center position 210a of each of the plurality of first recognition frames 210, and extends it toward the reference line LS. The prediction unit 14 may obtain an intersection PS on the image between the first extension line 230 and the reference line LS. Further, the prediction unit 14 extends the first extension line 230 in the opposite direction to the reference line LS side, and extends the second recognition frame 220 so that the center position 220a of the second recognition frame 220 is on the first extension line 230. A frame 220 is generated. As an example, the prediction unit 14 generates a second recognition frame 220 temporally ahead by a predetermined time based on four first recognition frames 210 for each predetermined time. Note that the second recognition frame 220 may be the expected position of the subject 201 at the current time. This makes it possible to recognize the subject 201 (predict the position of the subject 201) even if the subject 201 is temporarily hidden in whole or in part by another object.

Note that the prediction unit 14 uses the intersection point PS between the first extension line 230 and the reference line LS as a reference, and extends the line on the second extension line 240 passing through at least one of the four corners of each of the plurality of first recognition frames 210. , the corresponding corners of the second recognition frame 220 may be arranged. As an example, when the subject 201 moves to the lower left on the image with respect to the intersection PS of the first extension line 230 and the reference line LS, the prediction unit 14 predicts the upper left and lower right of each of the plurality of first recognition frames 210. Generate two second extension lines 240 that pass through the corners of , place the upper left corner of the second recognition frame 220 on one second extension line 240, and place the second The lower right corner of the recognition frame 220 may be placed.

Similarly, when the subject 201 moves to the lower right from the intersection PS, as an example, two second extension lines 240 passing through the upper right and lower left corners of the plurality of first recognition frames 210 are generated, and one of the second extension lines 240 is generated. The upper right corner of the second recognition frame 220 may be placed on the second extension line 240, and the lower left corner of the second recognition frame 220 may be placed on the other second extension line 240.

Similarly, when the subject 201 moves downward from the intersection PS, for example, two second extension lines 240 passing through the upper right and upper left corners of the plurality of first recognition frames 210 are generated, and one The upper left corner of the second recognition frame 220 may be placed on the second extension line 240, and the upper right corner of the second recognition frame 220 may be placed on the other second extension line 240.

As shown in FIG. 10, the generation unit 15 creates a front frame 220b corresponding to the front outline of the subject 201 and a rear face of the subject 201 so that a part thereof is in contact with the second recognition frame 220 generated by the prediction unit 14. A back frame 220c is generated according to the outer shape of the back frame 220c. The recognition unit 13 described above can also recognize the shape of the subject 201 by recognizing the subject 201. The generation unit 15 can use the recognition of the shape of the subject 201 by the recognition unit 13 to obtain the outer shape of the subject 201 in the width direction and height direction. The generation unit 15 generates a rectangular front frame 220b according to the outer shape of the subject 201 in the width direction and height direction. The generation unit 15 also generates a second recognition frame 220 on a connection line 250 connecting the intersection PS of the first and second extension lines 230, 240 and the reference line LS and the four corners of the front frame 220b. The rear frame 220c is set so that the corner of the rear frame 220c coincides with the point where the rear frame 220c intersects.

The generation unit 15 determines the position of the subject 201 on the left side with respect to the vertical direction of the image based on the image data, based on the intersection point (intersection point PS between the first and second extension lines 230 and 240 and the reference line LS) as a predetermined position. In this case, a rectangular front frame 220b is generated so as to touch the lower left vertex of the second recognition frame 220 with respect to the image, and a rectangular front frame 220b is generated so as to touch the upper right vertex of the second recognition frame 220 with respect to the image. A back frame 220c is generated. That is, the generation unit 15 matches the lower left corner of the second recognition frame 220 with the lower left corner of the front frame 220b, and aligns the two sides connected to the lower left corner of the second recognition frame 220 with the front The two sides connected to the lower left corner of the frame 220b are overlapped. In this case, the generation unit 15 matches the upper right corner of the second recognition frame 220 with the upper right corner of the back frame 220, and creates two sides connected to the upper right corner of the second recognition frame 220, The two sides connected to the upper right corner of the back frame 220c are overlapped.

When the subject 201 is located on the right side in the vertical direction of the image based on the image data with respect to the predetermined position as a reference, the generation unit 15 generates an image so that the subject 201 is in contact with the vertex at the lower right of the second recognition frame 220 with respect to the image. A rectangular front frame 220b is generated, and a rectangular rear frame 220c is generated so as to be in contact with the upper left vertex of the second recognition frame 220 with respect to the image. That is, the generation unit 15 matches the lower right corner of the second recognition frame 220 with the lower right corner of the front frame 220b, and generates two sides connected to the lower right corner of the second recognition frame 220. and two sides connected to the lower left corner of the front frame 220b are overlapped. In this case, the generation unit 15 aligns the upper left corner of the second recognition frame 220 with the upper left corner of the back frame 220c, and aligns the two sides connected to the second recognition frame 220 with the upper left corner of the back frame 220c. Overlap the two sides that connect to.

The size acquisition unit 16 determines the size based on the distance from the fixed-point imaging unit 21 to the subject 201, the size of the image based on the image data, and the size of at least one of the front frame 220b and the back frame 220c generated by the generation unit 15. , obtain the size of the subject 201. Since the size of the subject 201 in the width direction and height direction changes depending on the position of the intersection point PS between the first and second extension lines 230 and 240 and the reference line LS, the size acquisition unit 16 calculates the size of the subject 201 as appropriate. Get the size of. Although any calculation method may be used, the size acquisition unit 16 may acquire the size of the subject 201 as follows, for example.

When determining the size of the subject 201 in the width direction on the image, the size acquisition unit 16 calculates the coordinate (X coordinate) on the image of the intersection point PS of the first and second extension lines 230 and 240 and the reference line LS, The first value is acquired based on the coordinate (X coordinate) of the center position 220a of the second recognition frame 220 on the image and the image size (for example, the number of pixels in the horizontal direction of the image). The size acquisition unit 16 acquires a second value by adding a predetermined first coefficient to the first value and adding a second coefficient to the value. The size acquisition unit 16 multiplies the second value by the size of the second recognition frame 220 in the width direction on the image, thereby obtaining the size of the subject 201 in the width direction on the image (width of the front frame 220b). Get the direction size).
The size acquisition unit 16 can obtain the height of the subject 201 on the image in the same manner as the above-described size of the subject 201 in the width direction on the image.

Here, the size acquisition unit 16 obtains a result (obtained by the setting unit 12 The distance from the fixed-point imaging unit 21 to the subject 201 is acquired based on the results obtained (results obtained). The size acquisition unit 16 acquires the size of the subject 201 in the width direction and height direction based on the acquired distance and the size of the subject 201 on the image described above. Further, the size acquisition unit 16 acquires the size of the subject 201 in the depth direction based on the result obtained by the setting unit 12.

The center of gravity acquisition unit 17 acquires the center of gravity position PG of the subject 201 (see FIG. 11) based on the front frame 220b and the back frame 220c generated by the generation unit 15. For example, the center of gravity acquisition unit 17 obtains two corners on the ground surface side (for example, the coordinates of the corners on the image) among the four corners of the front frame 220b, and four corners of the back frame 220c. The center of gravity position PG (coordinates on the image) of the subject 201 is obtained using the two corners on the ground surface side (for example, the coordinates of the corners on the image). In this case, the center of gravity acquisition unit 17 may obtain, for example, a result (setting unit 12 (image position information)), the distance from the fixed point imaging unit 21 to the center of gravity position PG (as an example, the distance in the horizontal direction (X direction) and vertical direction (Y direction) on the image Note that the distance in the X direction may be the distance from the center line in the horizontal direction on the image.

The position acquisition unit 18 acquires the position of the subject 201 by converting the gravity center position PG acquired by the gravity center acquisition unit 17 into a predetermined coordinate system. The position acquisition unit 18 acquires the position of the subject 201 by converting the gravity center position PG (the gravity center position on the image) acquired by the gravity center acquisition unit 17 into a global coordinate system, for example. For example, the position acquisition unit 18 acquires the imaging direction (azimuth and angle toward the ground surface (angle between the perpendicular and the optical axis of the fixed-point imaging unit 21)) of the fixed-point imaging unit 21, and the global coordinates of the fixed-point imaging unit 21. The position of the subject 201 in the global coordinate system is obtained by using trigonometric functions.

The speed acquisition unit 19 acquires the speed of the subject 201 based on the change in the plurality of gravity center positions PG (PG1, PG2) of the subject 201 at each time acquired by the gravity center acquisition unit 17. As illustrated in FIG. 12, the speed acquisition unit 19 calculates how far the center of gravity position PG (PG1, PG2) has moved during a predetermined time dt based on at least two temporally different center of gravity positions PG1, PG2. By doing so, it is possible to obtain the speed of the subject 201. In this case, the velocity acquisition unit 19 may acquire the velocity Vx in the x direction and the velocity Vy in the Y direction based on the displacement dx in the X direction and the displacement dy in the y direction during the predetermined time dt. Further, the speed acquisition unit 19 may acquire the average speed of the subject 201 based on the centroid positions PG of three or more temporally different subjects 201.

The output control section 20 controls the output section. The above-mentioned output unit outputs the size of the subject 201 acquired by the size acquisition unit 16, the position of the subject 201 acquired by the position acquisition unit 18, and the velocity acquisition unit 19 based on the control of the output control unit 20. At least one of the speeds of the subject 201 thus determined is output. The output unit may be, for example, the communication unit 23 and the display unit 24 in addition to the storage unit 22 described above. The output control unit 20 controls the size of the subject 201 acquired by the size acquisition unit 16, the center of gravity position PG of the subject 201 acquired by the center of gravity acquisition unit 17, the position and speed of the subject 201 acquired by the position acquisition unit 18. Subject information regarding the speed of the subject 201 acquired by the acquisition unit 19 is stored in the storage unit 22 . Further, the output control section 20 may, for example, control the communication section 23 to transmit the subject information to an external device (for example, a server, etc.). The output control unit 20 may display subject information on the display unit 24, for example.

Next, an information processing method according to an embodiment will be described.
First, a method for acquiring image position information will be explained.
FIG. 13 is an information processing method according to an embodiment, and is a flowchart for explaining a method of acquiring image position information.

In step ST11, the fixed-point imaging unit 21 images the subject.

In step ST12, the recognition unit 13 extracts the features of the subject imaged in step ST11 based on the results learned in advance, and recognizes the subject based on the extraction results.

In step ST13, the setting unit 12 acquires image position information based on the height of the subject imaged in step ST11, that is, the subject recognized in step ST12, and the angle of view of the fixed-point imaging unit 21.
That is, the setting section 12 performs the following processing using the first to ninth setting sections 121 to 129.
The first setting unit 121 obtains in advance the height of the subject, the angle of view of the fixed-point imaging unit 21, and the width of the image captured by the fixed-point imaging unit 21.
When the subject moves, the second setting unit 122 obtains the number of pixels per meter based on the height of the subject obtained by the first setting unit 121.
The third setting unit 123 acquires the relationship between the Y coordinate, which is the coordinate in the depth direction of the image, and the number of pixels per meter of the subject acquired by the second setting unit 122.
The fourth setting unit 124 obtains the actual width w of the image at a predetermined position based on the width of the image obtained by the first setting unit 121 and the number of pixels per meter obtained by the second setting unit 122. do.
The fifth setting unit 125 determines the center of the image from the fixed-point imaging unit 21 based on the width w of the actual image acquired by the fourth setting unit 124 and the angle of view θ2 acquired by the first setting unit 121. A first distance L1, which is the distance to the position Pc, and a second distance L2, which is the distance from the ground to the center position Pc, are acquired.
Based on the first distance L1 and the second distance L2 acquired by the fifth setting unit 125, the sixth setting unit 126 determines the center line (line along the Y-axis direction) between the ground and the fixed-point imaging unit 21 in the imaging direction. Find the angle θ formed by
The seventh setting unit 127 determines whether the fixed-point imaging unit 21 is located from the ground based on the first distance L1 and second distance L2 acquired by the fifth setting unit 125 and the angle θ acquired by the sixth setting unit 126. Get the height H up to.
The eighth setting unit 128 sets the fixed-point imaging unit 21 based on the height H from the ground to the fixed-point imaging unit 21 acquired by the seventh setting unit 127 and the angle θ acquired by the sixth setting unit 126. The position from the imaging position where the camera is placed to the subject is acquired, and image position information is acquired.
Here, the ninth setting unit 129 calculates, based on the width at the reference position in the Y-coordinate direction and the width at a predetermined position in the Y-coordinate direction, among the widths of the image acquired by the fourth setting unit 124. It is also possible to obtain the width ratio to the Y coordinate.

In step ST14, the setting unit 12 stores the image position information acquired in step ST13 in the storage unit 22.

Next, a method for acquiring information regarding the subject 201 will be described.
FIG. 14 is an information processing method according to an embodiment, and is a flowchart for explaining a method of acquiring information regarding the subject 201.

In step ST21, the fixed-point imaging unit 21 temporally continuously images the subject 201 and generates image data. The fixed-point imaging unit 21 generates moving image data by capturing moving images or continuous images at predetermined time intervals. The subject 201 is, for example, an object that moves in a constant (substantially constant) manner. One type of object that moves in a constant motion is an object that moves linearly (substantially linearly). Note that the object may be stationary. Specific examples of objects may be vehicles, people, animals, etc.

In step ST22, the recognition unit 13 recognizes the subject 201 from at least two temporally different images (frames or still images) based on the image data generated in step ST21, and identifies a plurality of images surrounding the subject 201 at each time. A first recognition frame 210 is generated. In this case, the recognition unit 13 may generate a rectangular frame surrounding the subject 201 (for example, touching the outline of the subject 201) on the image as the first recognition frame 210.

In step ST23, the prediction unit 14 determines, based on the plurality of first recognition frames 210 generated in step ST22, a predetermined time that is temporally earlier than the recording time of the image when generating the first recognition frame 210. A second recognition frame 220 surrounding the subject 201 is predicted. Specifically, the prediction unit 14 determines the plurality of first recognition frames 210, the center position 210a of each of the plurality of first recognition frames 210, and the reference line LS of the image based on the image data generated in step ST21. Based on this, the second recognition frame 220 and the center position 220a of the second recognition frame 220 are predicted. In this case, the prediction unit 14 may generate a rectangular frame surrounding the subject 201 on the image as the second recognition frame 220.
The predetermined time mentioned above may be the current time. That is, the second recognition frame 220 may be the expected position of the subject 201 at the current time. This makes it possible to recognize the subject 201 (predict the position of the subject 201) even if the subject 201 is temporarily hidden in whole or in part by another object.

In step ST24, the generation unit 15 generates a front frame 220b corresponding to the front outline of the subject 201 and a front frame 220b corresponding to the rear outline of the subject 201 so that a part thereof touches the second recognition frame 220 generated in step ST23. A rear frame 220c is generated.
When the subject 201 is located on the left side in the vertical direction of the image based on the intersection point PS of the first and second extension lines 230 and 240 and the reference line LS, the generation unit 15 generates a second recognition frame for the image. A rectangular front frame 220b is generated so as to touch the lower left vertex of the second recognition frame 220, and a rectangular rear frame 220c is generated so as to touch the upper right vertex of the second recognition frame 220 with respect to the image.
Alternatively, when the subject 201 is located on the right side with respect to the vertical direction of the image based on the intersection point PS of the first and second extension lines 230 and 240 and the reference line LS, the generation unit 15 generates a second A rectangular front frame 220b is generated so as to touch the lower right vertex of the recognition frame 220, and a rectangular rear frame 220c is generated so as to touch the upper left vertex of the second recognition frame 220 with respect to the image.

In step ST25, the size acquisition unit 16 acquires the distance from the fixed-point imaging unit 21 to the subject 201, the size of the image based on the image data (for example, the number of pixels), and the front frame 220b and back frame generated by the generation unit 15. The size of the subject 201 is acquired based on the size of at least one of the images 220c and 220c.

In step ST26, the center of gravity acquisition unit 17 obtains the center of gravity position PG of the subject 201 based on the front frame 220b and back frame 220c generated in step ST24.
In this case, the center of gravity acquisition unit 17 may obtain, for example, a result (setting unit 12 (image position information)), the distance from the fixed point imaging unit 21 to the center of gravity position PG (as an example, the distance in the horizontal direction (X direction) and vertical direction (Y direction) on the image) It is also possible to obtain Note that the distance in the X direction may be a distance from the center line in the horizontal direction on the image.

In step ST27, the position acquisition unit 18 acquires the position of the subject 201 by converting the center of gravity position PG acquired in step ST26 into a predetermined coordinate system. The predetermined coordinate system is, for example, a global coordinate system.

In step ST28, the velocity acquisition unit 19 acquires the velocity of the subject 201 based on the changes in the plurality of gravity center positions PG of the subject 201 acquired in step ST26 or step ST27.

Note that in the embodiment described above, an example was described in which information regarding the subject 201 is acquired based on the first recognition frame 210 and the second recognition frame 220. However, in the present invention, if the subject 201 at the current time can be recognized, the second recognition frame 220 may be used as the first recognition frame 210 to acquire information regarding the subject 201.

Next, the effects of this embodiment will be explained.
The information processing device 1 includes a fixed-point imaging unit 21 that temporally continuously images a subject 201 to generate image data, and at least two temporally different images recorded in the image data captured by the fixed-point imaging unit 21. A recognition unit 13 that each recognizes a subject 201 based on an image and generates a plurality of first recognition frames 210 surrounding the subject 201 at each time, a plurality of first recognition frames 210, and the plurality of first recognition frames. Based on the center position 210a of 210 and the reference line LS of the image based on the image data captured by the fixed point imaging unit 21, the time is longer than the recording time of the image when the recognition unit 13 generates the first recognition frame 210. A prediction unit 14 that predicts a second recognition frame 220 surrounding the subject 201 and a center position 220a of the second recognition frame 220 at a predetermined time that is the next predetermined time, and a second recognition frame 220 generated by the prediction unit 14. A generating unit 15 that generates a front frame 220b corresponding to the front outline of the subject 201 and a back frame 220c corresponding to the rear outline of the subject 201 so that the parts from the fixed-point imaging unit 21 to the subject 201 are in contact with each other. a size acquisition unit 16 that acquires the size of the subject 201 based on the distance, the size of the image based on the image data, and the size of at least one of the front frame 220b and the back frame 220c generated by the generation unit 15; Equipped with
Thereby, the information processing apparatus 1 can acquire information regarding the subject 201 recorded in the image.

The information processing device 1 includes a center of gravity acquisition unit 17 that acquires the center of gravity position PG of the subject 201 based on the front frame 220b and the back frame 220c generated by the generation unit 15, and a center of gravity position PG acquired by the center of gravity acquisition unit 17. The camera may also include a position acquisition unit 18 that acquires the position of the subject 201 by converting the image into a predetermined coordinate system.
The predetermined coordinate system may be, for example, a global coordinate system. Thereby, the information processing device 1 can acquire the position (coordinates) of the subject 201.

The information processing device 1 may include a speed acquisition unit 19 that acquires the speed of the subject 201 based on changes in a plurality of barycenter positions PG of the subject 201 over time acquired by the centroid acquisition unit 17.
Thereby, the information processing device 1 can acquire the speed based on the center of gravity position PG of the subject 201 when the subject 201 moves.

The information processing device 1 determines at least the size of the subject 201 acquired by the size acquisition unit 16, the position of the subject 201 acquired by the position acquisition unit 18, and the speed of the subject 201 acquired by the speed acquisition unit 19. It is also possible to include an output section that outputs one.
Thereby, the information processing device 1 can transmit the acquired information regarding the subject 201 to the outside through the communication unit 23, display it on the display unit 24, and store it in the storage unit 22.

In the information processing device 1, the recognition unit 13 may generate a rectangular frame surrounding the subject 201 on the image as the first recognition frame 210.
Thereby, the information processing device 1 can recognize the subject 201 based on a learning model, pattern matching, etc., and can generate the first recognition frame 210 (ROI) based on the recognized subject 201. can.

In the information processing device 1, the prediction unit 14 may generate a rectangular frame surrounding the subject 201 on the image as the second recognition frame 220.
Thereby, the information processing apparatus 1 can generate the second recognition frame 220 (ROI) of the subject 201 based on the plurality of first recognition frames 210.

In the information processing device 1, the prediction unit 14 sets a line extending in the horizontal direction of the image based on the image data as the reference line LS, sets a predetermined position of the reference line LS as an end point, and calculates a plurality of first The second recognition frame 220 may be generated by setting the center position 220a on the center line passing through each center position 210a of the recognition frame 210.
Thereby, the information processing device 1 can generate the second recognition frame 220 according to the moving direction of the subject 201 based on the plurality of first recognition frames 210.

In the information processing device 1, when the subject 201 is located on the left side with respect to the vertical direction of the image based on the image data with respect to the predetermined position as a reference, the generation unit 15 generates an image that is located at the lower left of the second recognition frame 220 with respect to the image. The rectangular front frame 220b may be generated so as to be in contact with the apex, and the rectangular back frame 220c may be generated so as to be in contact with the apex at the upper right of the second recognition frame 220 with respect to the image.
Thereby, the information processing device 1 can generate the front frame 220b and the back frame 220c when there is a leftward component with respect to the image in the moving direction of the subject 201.

In the information processing device 1, when the subject 201 is located on the right side with respect to the vertical direction of the image based on the image data with respect to the predetermined position as a reference, the generation unit 15 generates an image at the bottom right of the second recognition frame 220 with respect to the image. The rectangular front frame 220b may be generated so as to touch a certain vertex, and the rectangular back frame 220c may be generated so as to touch the upper left vertex of the second recognition frame 220 with respect to the image.
Thereby, the information processing apparatus 1 can generate the front frame 220b and the back frame 220c when there is a rightward component with respect to the image in the movement direction of the subject 201.

In the information processing device 1, the size acquisition unit 16 uses the result of associating in advance the position in the image based on the image data generated by the fixed-point imaging unit 21 and the distance from the fixed-point imaging unit 21 at that position. , the distance from the fixed-point imaging unit 21 to the subject 201 may be acquired.
Thereby, the information processing device 1 can use the distance as a basis for acquiring information regarding the subject 201.

In the information processing method, a fixed point imaging step in which a computer sequentially images the subject 201 to generate image data, and at least two temporally different images recorded in the image data captured in the fixed point imaging step. a recognition step of respectively recognizing the subject 201 based on the image and generating a plurality of first recognition frames 210 surrounding the subject 201 at each time; a plurality of first recognition frames 210; Based on the center position 210a of 210a and the reference line LS of the image based on the image data captured in the fixed point imaging step, the image is temporally earlier than the recording time of the image when the first recognition frame 210 is generated in the recognition step. A prediction step of predicting a second recognition frame 220 surrounding the subject 201 and a center position 220a of the second recognition frame 220 at a predetermined time to be , a generation step of generating a front frame 220b according to the front outline of the subject 201 and a back frame 220c according to the rear outline of the subject 201, the distance from the fixed point imaging step to the subject 201, and the image data. A size obtaining step of obtaining the size of the subject 201 is performed based on the size of the based image and the size of at least one of the front frame 220b and the rear frame 220c generated in the generation step.
Thereby, the information processing method can acquire information regarding the subject 201 recorded in the image.

The information processing program provides the computer with at least two temporally different functions: a fixed-point imaging function that sequentially images the subject 201 to generate image data, and a fixed-point imaging function that generates image data by sequentially imaging the subject 201; A recognition function that recognizes each subject 201 based on an image and generates a plurality of first recognition frames 210 surrounding the subject 201 at each time, a plurality of first recognition frames 210, and the plurality of first recognition frames 210. based on the center position 210a of the image and the reference line LS of the image based on the image data captured by the fixed-point imaging function, the image is temporally earlier than the recording time of the image when the first recognition frame 210 is generated by the recognition function. A prediction function that predicts a second recognition frame 220 surrounding the subject 201 at a predetermined time and a center position 220a of the second recognition frame 220, and a prediction function that partially touches the second recognition frame 220 generated by the prediction function. In addition, a generation function that generates a front frame 220b according to the front outline of the subject 201 and a back frame 220c according to the rear outline of the subject 201, the distance from the fixed point imaging function to the subject 201, and the image data are provided. A size acquisition function that acquires the size of the subject 201 based on the size of the image based on the image size and the size of at least one of the front frame 220b and the rear frame 220c generated by the generation function is realized.
Thereby, the information processing program can acquire information regarding the subject 201 recorded in the image.

Each part of the information processing device 1 described above may be realized as a function of a computer processing device or the like. That is, the setting section 12, the recognition section 13, the prediction section 14, the generation section 15, the size acquisition section 16, the center of gravity acquisition section 17, the position acquisition section 18, the speed acquisition section 19, and the output control section 20 of the information processing device 1 are configured by a computer. Each of the functions may be realized as a setting function, a recognition function, a prediction function, a generation function, a size acquisition function, a center of gravity acquisition function, a position acquisition function, a speed acquisition function, and an output control function by the arithmetic processing device or the like.
The information processing program can cause a computer to realize each of the functions described above. The information processing program may be recorded on a computer-readable non-transitory recording medium such as an external memory or an optical disc.
Further, as described above, each part of the information processing device 1 may be realized by a calculation processing device of a computer or the like. The arithmetic processing device and the like are constituted by, for example, an integrated circuit or the like. Therefore, each part of the information processing device 1 may be realized as a circuit that constitutes an arithmetic processing device or the like. That is, the setting section 12, the recognition section 13, the prediction section 14, the generation section 15, the size acquisition section 16, the center of gravity acquisition section 17, the position acquisition section 18, the speed acquisition section 19, and the output control section 20 of the information processing device 1 are configured by a computer. It may be realized as a setting circuit, a recognition circuit, a prediction circuit, a generation circuit, a size acquisition circuit, a center of gravity acquisition circuit, a position acquisition circuit, a speed acquisition circuit, and an output control circuit that constitute an arithmetic processing device or the like.
Further, the fixed-point imaging unit 21 and the output unit (storage unit 22, communication unit 23, and display unit 24) of the information processing device 1 have a fixed-point imaging function including functions such as an arithmetic processing unit, and an output function (storage function, communication function and display function). Further, the fixed point imaging section 21 and the output section (the storage section 22, the communication section 23, and the display section 24) of the information processing device 1 are configured by, for example, an integrated circuit or the like. (communication circuit and display circuit). Further, the fixed-point imaging unit 21 and the output unit (storage unit 22, communication unit 23, and display unit 24) of the information processing device 1 are configured by a plurality of devices, so that the fixed-point imaging unit and the output unit (storage unit, (communication device and display device).

1 Information processing device 11 Control section 12 Setting section 13 Recognition section 14 Prediction section 15 Generation section 16 Size acquisition section 17 Center of gravity acquisition section 18 Position acquisition section 19 Speed acquisition section 20 Output control section 21 Fixed point imaging section 22 Storage section 23 Communication section 24 Display section

Claims (12)

a fixed-point imaging unit that sequentially images a subject and generates image data;
a recognition unit that recognizes the subject based on at least two temporally different images recorded in image data captured by the fixed-point imaging unit, and generates a first recognition frame surrounding the subject at each time;
A recognition unit generates a first recognition frame based on a plurality of first recognition frames, a center position of the plurality of first recognition frames, and a reference line of an image based on image data captured by the fixed-point imaging unit. a prediction unit that predicts a second recognition frame surrounding the subject and a center position of the second recognition frame at a predetermined time that is temporally earlier than the recording time of the image when the image is recorded;
a generation unit that generates a front frame according to the front outline of the subject and a back frame according to the rear outline of the subject so that a part thereof touches the second recognition frame generated by the prediction unit;
Size acquisition for obtaining the size of the subject based on the distance from the fixed point imaging unit to the subject, the size of the image based on image data, and the size of at least one of the front frame and the back frame generated by the generation unit. Department and
An information processing device comprising: a center of gravity acquisition unit that acquires the position of the center of gravity of the subject based on the front frame and the back frame generated by the generation unit;
a position acquisition unit that acquires the position of the subject by converting the gravity center position acquired by the gravity center acquisition unit into a predetermined coordinate system;
The information processing device according to claim 1, comprising: The information processing apparatus according to claim 2, further comprising a speed acquisition section that acquires the speed of the subject based on changes in a plurality of positions of the center of gravity of the subject over time acquired by the center of gravity acquisition section. an output unit that outputs at least one of the size of the subject acquired by the size acquisition unit, the position of the subject acquired by the position acquisition unit, and the speed of the subject acquired by the speed acquisition unit. The information processing device according to claim 3. The information processing device according to claim 1, wherein the recognition unit generates a rectangular frame surrounding the subject on the image as the first recognition frame. The information processing device according to claim 1, wherein the prediction unit generates a rectangular frame surrounding the subject on the image as the second recognition frame. The prediction unit is
Set a line extending in the horizontal direction of the image based on the image data as the reference line,
According to any one of claims 1 to 6, wherein a predetermined position of the reference line is set as an end point, and the second recognition frame is generated with the center position being on a center line passing from the end point to the center position of each of the plurality of first recognition frames. The information processing device described. The generation unit is
When the subject is located on the left side in the vertical direction of the image based on the image data based on the predetermined position, a rectangular front frame is generated so as to be in contact with the lower left vertex of the second recognition frame with respect to the image. 8. The information processing apparatus according to claim 7, wherein a rectangular back frame is generated with respect to the image so as to be in contact with a top right corner of the second recognition frame. The generation unit is
When the subject is located on the right side in the vertical direction of the image based on the image data based on the predetermined position, a rectangular front frame is generated for the image so as to be in contact with the lower right vertex of the second recognition frame. The information processing apparatus according to claim 7, wherein the information processing apparatus also generates a rectangular back frame so as to be in contact with the top left vertex of the second recognition frame with respect to the image. The size acquisition unit selects a subject from the fixed-point imaging unit based on a result of associating in advance a position in the image based on image data generated in the fixed-point imaging unit and a distance from the fixed-point imaging unit at the position. The information processing device according to any one of claims 1 to 9, wherein the information processing device acquires a distance to. The computer is
a fixed point imaging step of generating image data by imaging a subject continuously over time;
a recognition step of respectively recognizing the subject based on at least two temporally different images recorded in the image data captured in the fixed-point imaging step, and generating a first recognition frame surrounding the subject at each time;
A first recognition frame is generated by a recognition step based on a plurality of first recognition frames, a center position of the plurality of first recognition frames, and a reference line of an image based on image data imaged by the fixed-point imaging step. a prediction step of predicting a second recognition frame surrounding the subject and the center position of the second recognition frame at a predetermined time that is temporally earlier than the recording time of the image when the image is recorded;
a generation step of generating a front frame according to the front outline of the subject and a back frame according to the rear outline of the subject so that a part thereof touches the second recognition frame generated in the prediction step;
Size acquisition for obtaining the size of the subject based on the distance from the fixed point imaging step to the subject, the size of the image based on image data, and the size of at least one of the front frame and the back frame generated in the generation step. step and
An information processing method that performs. to the computer,
A fixed-point imaging function that captures images of a subject continuously over time and generates image data;
a recognition function that recognizes a subject based on at least two temporally different images recorded in image data captured by the fixed-point imaging function, and generates a first recognition frame surrounding the subject at each time;
A first recognition frame is generated by a recognition function based on a plurality of first recognition frames, a center position of the plurality of first recognition frames, and a reference line of an image based on image data captured by the fixed point imaging function. a prediction function that predicts a second recognition frame surrounding the subject and the center position of the second recognition frame at a predetermined time that is temporally earlier than the recording time of the image when the image is recorded;
a generation function that generates a front frame according to the front outline of the subject and a back frame according to the rear outline of the subject so that a part thereof touches the second recognition frame generated by the prediction function;
Size acquisition for obtaining the size of the subject based on the distance from the fixed point imaging function to the subject, the size of the image based on image data, and the size of at least one of the front frame and the back frame generated by the generation function. function and
An information processing program that realizes.

Images