JP2023100805A - Imaging apparatus, imaging method, and imaging program - Google Patents

Abstract

To provide an imaging apparatus which is preferable for imaging the same subject by using a plurality of imaging apparatuses, an imaging method, and an imaging program.SOLUTION: An imaging apparatus comprises: a plurality of fixed-point imaging parts that generates image data by continuously imaging a subject in time-variant; a recognition part that recognizes the subject from an image on the basis of image data generated by each fixed-point imaging part; an acquisition part that acquires position information related to a position of the subject to be recognized by the recognition part; a prediction part that predicts a movement destination of the subject on the basis of the position information to be acquired by the acquisition part; a selection part that selects the fixed-point imaging part nearest to the movement destination to be predicted by the prediction part to make the subject imaging by the fixed-point imaging part selected; a correspondence part that corresponds the image on the basis of the image data imaged by each fixed-point imaging part and the position information acquired by the acquisition part on the basis of the image; and a display part that continuously acquires a plurality of images to be corresponded by the correspondence part and the position information in time-variant to display the image and a locus of the subject on the basis of the position information.SELECTED DRAWING: Figure 1

Description

The present invention relates to an imaging device, an imaging method, and an imaging program.

2. Description of the Related Art Conventionally, it has been common practice to shoot moving images using an imaging device. Under such circumstances, there is a technique for displaying the moving trajectory of a subject on a display unit together with a moving image in order to select a desired scene from captured moving images (see Patent Document 1). According to the technique described in Patent Document 1, when one point on the trajectory is selected, the scene corresponding to that time is displayed.

JP 2014-132748 A

As an example of the application in which moving images are captured using an imaging device, a monitoring camera or the like is arranged to capture a person or the like moving on a road or the like. A plurality of surveillance cameras are often arranged, and there are cases where the same subject is photographed by different surveillance cameras. In this case, there is a possibility that the viewer of the moving image captured by the monitoring camera cannot determine whether or not the subject captured by the different monitoring cameras is the same.

The present invention relates to an imaging device, an imaging method, and an imaging program suitable for imaging the same subject using a plurality of imaging devices.

An imaging device of one aspect includes a plurality of fixed-point imaging units that capture images of a subject continuously in time to generate image data, and a recognition unit that recognizes a subject from an image based on the image data generated by each of the fixed-point imaging units. an acquisition unit for acquiring position information about the position of the subject recognized by the recognition unit; a prediction unit for predicting the destination of the subject based on the position information acquired by the acquisition unit; and the destination predicted by the prediction unit. a selection unit that selects a fixed-point imaging unit that is closest to and causes the selected fixed-point imaging unit to capture an image of the subject; an image based on the image data captured by the fixed-point imaging unit; an associating unit that associates position information with location information; a plurality of images and location information associated by the associating unit are successively obtained in time series; and a display unit that displays

In one aspect of the imaging device, the associating unit associates the image based on the image data captured by each of the plurality of fixed-point imaging units with the position information, and captures the image data with respect to the correspondence information indicating the association. When displaying the image and the trajectory, the display unit adds the fixed-point imaging based on the feature information to the image based on the correspondence information and the trajectory based on the position information. It is also possible to attach and display a different characteristic image for each copy.

An imaging device according to one aspect includes a time information addition unit that adds time information synchronized with an image based on image data generated by a fixed-point imaging unit and position information acquired by an acquisition unit; and the display unit synchronizes the image based on the image data read from the storage unit and the trajectory based on the position information based on the time information added by the time information adding unit. may be displayed in

In one aspect of the imaging device, the acquisition unit associates in advance a position in an image captured by the fixed-point imaging unit with the actual position of the subject, and when the recognition unit recognizes a new subject, It is good also as acquiring a positional information.

In one aspect of the imaging device, the acquisition unit associates a position in an image captured by the fixed-point imaging unit with a distance from the fixed-point imaging unit at that position, a distance from the fixed-point imaging unit, and an actual image. Position information may be acquired when a new subject is recognized by the recognition unit based on the result of associating it with the position in space.

In one aspect of the imaging device, the acquisition unit includes a first acquisition unit that acquires in advance the height of the subject, the angle of view of the fixed-point imaging unit, and the width of the image captured by the fixed-point imaging unit; , a second acquisition unit that acquires the number of pixels constituting image data per meter based on the height of the subject acquired by the first acquisition unit; a third acquisition unit that acquires the relationship between the coordinates and the number of pixels per meter of the subject acquired by the second acquisition unit; the width of the image acquired by the first acquisition unit; a fourth acquisition unit that acquires an actual image width at a predetermined position based on the number of pixels per meter, and the actual image width acquired by the fourth acquisition unit; a fifth acquisition unit that acquires a first distance, which is the distance from the fixed-point imaging unit to the center position in the image, and a second distance, which is the distance from the ground to the center position, based on the angle of view; Based on the first distance and the second distance obtained by the fifth obtaining unit, a sixth obtaining unit obtains the angle between the ground and the center line in the imaging direction of the fixed-point imaging unit; a seventh acquisition unit that acquires the height from the ground to the fixed-point imaging unit based on the first distance, the second distance, and the angle acquired by the sixth acquisition unit; Based on the height from the ground to the fixed-point imaging unit and the angle acquired by the sixth acquisition unit, the position from the imaging position where the fixed-point imaging unit is arranged to the subject is acquired, and further, a plurality of positions of the subject are acquired. and an eighth acquisition unit that acquires the image position information based on the acquisition.

In one aspect of the imaging device, the obtaining unit performs the width based on the width at the reference position in the Y-coordinate direction and the width at the predetermined position in the Y-coordinate direction, out of the width of the actual image obtained by the fourth obtaining unit. and a ninth acquisition unit that acquires the width ratio with respect to the Y coordinate.

In one aspect of the imaging method, a computer performs a fixed-point imaging step in which a plurality of fixed-point imaging units successively images a subject in time to generate image data; a recognition step of recognizing a subject from the object; an obtaining step of obtaining position information about the position of the subject recognized in the recognizing step; a prediction step of predicting the destination of the subject based on the position information obtained in the obtaining step; selecting a fixed-point imaging unit closest to the destination predicted in the step and causing the selected fixed-point imaging unit to capture an image of the subject; a step of associating positional information obtained in the obtaining step with the positional information acquired in the obtaining step; and a display step of displaying the trajectory of the subject based on the information on the display unit.

An image capturing program of one aspect provides a computer with a fixed point capturing function of capturing images of a subject continuously in time by a plurality of fixed point capturing units to generate image data, and an image based on the image data generated by each of the fixed point capturing units. a recognition function that recognizes a subject from a target, an acquisition function that acquires position information related to the position of the subject recognized by the recognition function, a prediction function that predicts the destination of the subject based on the position information acquired by the acquisition function, and a prediction A selection function that selects the fixed-point imaging unit closest to the destination predicted by the function and captures the subject with the selected fixed-point imaging unit, an image based on the image data captured by the fixed-point imaging unit, and a A function for associating location information obtained by an acquisition function with a corresponding image, and a plurality of images and location information associated by the associating function are acquired in succession in time, and the images and their locations are obtained in chronological order. and a display function for displaying the trajectory of an object based on information on a display unit.

In the imaging device of one aspect, the destination of the subject is predicted based on the acquisition of the position information of the subject by the acquisition unit, and the selection unit selects the nearest fixed-point imaging unit capable of imaging the destination. , it is possible to provide an imaging apparatus suitable for imaging the same subject using a plurality of fixed-point imaging units.
Further, the imaging method and the imaging program of one aspect can achieve the same effect as the imaging apparatus of one aspect described above.

1 is a block diagram for explaining an imaging device according to an embodiment; FIG. FIG. 10 is a diagram for explaining prediction of a subject; FIG. FIG. 4 is a diagram for explaining association between an image and position information; FIG. FIG. 3 is a diagram for explaining a display screen of a display unit; FIG. FIG. 10 is a diagram for use in explaining a case of acquiring image position information; FIG. 4 is a diagram for explaining an example of an image captured by a fixed-point imaging section; FIG. It is a graph which shows the relationship between a Y-coordinate and the number of pixels. 7 is a graph for explaining an example of image position information; 4 is a graph showing the relationship between the Y coordinate and the width ratio of an image; 4 is a first flowchart for explaining an imaging method according to one embodiment; 4 is a second flow chart for explaining an imaging method according to one embodiment;

An embodiment of the present invention will be described below.
An imaging device images a subject with a plurality of fixed-point imaging units arranged at different points. The imaging device predicts the destination of the subject by the prediction section, and captures the subject by the nearest fixed-point imaging section that can capture the subject. Further, the imaging device acquires the trajectory of the movement of the subject based on the acquisition of the position information of the subject by the acquisition unit. Then, the imaging device displays the image of the subject imaged by the fixed-point imaging section and the movement trajectory of the subject acquired by the acquisition section on the display section so as to be temporally synchronized.

A specific configuration of the imaging apparatus will be described below.
FIG. 1 is a block diagram for explaining an imaging device according to one embodiment.
FIG. 2 is a diagram for explaining subject prediction.
FIG. 3 is a diagram for explaining the correspondence between images and position information.
FIG. 4 is a diagram for explaining the display screen of the display unit.

As shown in FIG. 1 , the imaging device 1 includes a fixed-point imaging unit 11 , a recognition unit 12 , an acquisition unit 13 , a prediction unit 14 , a selection unit 15 , an association unit 16 and a display unit 17 .

A plurality of fixed-point imaging units 11 are provided, for example, at different locations. In the imaging device 1, information (positional information) regarding the position where the fixed-point imaging section 11 is arranged is obtained in advance. A plurality of fixed-point imaging units 11 may capture images of different locations, and the imaging range of one fixed-point imaging unit 11 may be part of the imaging range of another fixed-point imaging unit 11 .

The fixed-point imaging unit 11 continuously images a subject in time to generate image data. The fixed-point imaging unit 11 may generate image data by capturing moving images, or may generate image data by capturing still images at predetermined time intervals. The fixed-point imaging unit 11 (time information adding unit 18 described later) may or may not attach the imaging time (time information) to the frames or still images that make up the moving image. The fixed-point imaging unit 11 may output the generated image data to the recognition unit 12 described later, or may store it in the storage unit 19 . If time information is not attached to a frame or still image in the fixed-point imaging unit 11, the storage unit 19 (time information addition unit 18 described later) adds the storage time (time information) to the frame or still image when storing the image data. may be attached to When image data is generated in the fixed-point imaging section 11, the image data is immediately stored in the storage section 19. FIG. Therefore, the photographing time and the stored time almost match.
Note that each of the plurality of fixed-point imaging units 11 may be connected to an NTP server so that the clock times are synchronized. Alternatively, the plurality of fixed-point imaging units 11 may be connected to an NTP server with a GPS (Global Positioning System) antenna in an environment not connected to a communication network.
In addition, the fixed-point imaging section 11 may be an imaging section in which there is little change in brightness in the captured image and little change in white balance.

The recognition unit 12 recognizes a subject from an image based on image data captured by each fixed-point imaging unit 11 . For example, the recognition unit 12 may learn the features of the subject in advance, extract the features of the subject from the image data based on the learning result, and recognize the subject from the extracted features. Alternatively, the recognition unit 12 may recognize the subject based on the image data by using pattern matching, for example. For example, the recognizing unit 12 recognizes a person, a passenger car, a truck, a motorcycle, a bicycle, etc. as a subject. The recognition unit 12 may recognize a plurality of subjects from one image (moving image, frame, still image).
Note that the recognition unit 12 may receive image data from the fixed-point imaging unit 11 or may receive image data read from the storage unit 19 .

The acquisition unit 13 acquires position information regarding the position of the subject recognized by the recognition unit 12 . In the image obtained by the fixed-point imaging section 11, the position within the image is set in advance.
That is, the acquisition unit 13 is set in advance so that the position in the image and the distance from the fixed-point imaging unit 11 at that position can be known. The acquisition unit 13 associates the acquired distance with the actual spatial positional coordinates (positional information). By specifying the position of the subject in the image, the acquisition section 13 can acquire the position information of the subject based on the distance from the fixed-point imaging section 11 and the position information associated with the distance. . A method of associating the position in the image with the distance from the fixed-point imaging unit 11 at that position will be described later.

Note that the acquisition unit 13 is not limited to the above example, and the acquisition unit 13 repeats the measurement work that associates the position in the image captured by the fixed-point imaging unit 11 with the actual spatial position coordinates (position information). The obtained correspondence relationship (correspondence relationship) may be stored. In this case, when an image is obtained by the fixed-point imaging section 11, the obtaining section 13 may obtain the actual spatial position information from the position on the image based on the corresponding relationship.

When the subject is recorded in the image obtained by the fixed-point imaging section 11, that is, when the recognition section 12 recognizes the subject, the obtaining section 13 obtains the position of the subject in the image and the position from the fixed-point imaging section 11 at that position. The position information of the subject (the distance from the fixed-point imaging unit 11 to the subject) is acquired based on the distance of .
Since the fixed-point imaging unit 11 images the subject continuously or at predetermined time intervals, the acquisition unit 13 can acquire a plurality of pieces of position information for the same subject. The fixed-point imaging section 11 may store the acquired position information in the storage section 19 .
The fixed-point imaging unit 11 (time information addition unit 18 described later) may associate the acquired position information with the time information and position information recorded in the frame or still image. In this case, the fixed point imaging section 11 may store the position information in the storage section 19 in association with the time information.

The prediction unit 14 predicts the destination of the subject based on the position information acquired by the acquisition unit 13 . As described above, the acquisition unit 13 can acquire a plurality of pieces of position information about the same subject in time series, so the prediction unit 14 predicts the destination of the subject based on the plurality of pieces of position information. . That is, the prediction unit 14 obtains the velocity vector of the subject at each position based on a plurality of pieces of position information that precede or follow each other in time. The fixed-point imaging unit 11 captures moving images at a predetermined frame rate, or captures still images at preset time intervals. Therefore, the prediction unit 14 can know the temporal interval between two different images captured by the fixed-point imaging unit 11 . In addition, since the acquisition unit 13 acquires the position information of the subject in the image, the prediction unit 14 can know the amount of movement of the subject from the two temporally consecutive images. Therefore, the prediction unit 14 obtains the velocity vector of the subject based on the multiple images. The prediction unit 14 predicts the future destination of the subject based on the determined velocity vector and the position information of the subject. In this case, the prediction unit 14 can acquire the position coordinates (position information) of the movement destination of the subject.

As illustrated in FIG. 2, when the subject T (passenger car) travels on the road, the acquisition unit 13 acquires a plurality of positions P1 (position information) of the subject T based on the image obtained by the fixed-point imaging unit 11. . The prediction unit 14 predicts the future position P2 (position information) of the subject based on the multiple pieces of position information acquired by the acquisition unit 13 .

The selection unit 15 selects the fixed-point imaging unit 11 closest to the destination predicted by the prediction unit 14 and causes the selected fixed-point imaging unit 11 to pick up an image of the subject. Since the position information of the fixed-point imaging unit 11 is acquired in advance, the selection unit 15 can capture an image of the destination (position information) of the subject predicted by the prediction unit 14, and the destination (position information) can be captured. ), and the selected fixed-point imaging unit 11 is caused to image the subject.

In the case illustrated in FIG. 2, two fixed-point imaging units 11a and 11b are arranged. The selection unit 15 selects the nearest fixed-point imaging unit 11b capable of imaging the future position P2 of the subject predicted by the prediction unit 14. FIG. Since the selection unit 15 obtains in advance the position information of the fixed-point imaging unit 11 and also obtains in advance the position coordinates of the range that the fixed-point imaging unit 11 can image, the selection unit 15 selects one fixed-point imaging from among the plurality of fixed-point imaging units 11. Part 11 can be selected.

The associating unit 16 associates an image based on image data captured by the fixed-point imaging unit 11 with position information acquired by the acquiring unit 13 based on the image. That is, the associating unit 16 associates the subject image obtained by the fixed-point imaging unit 11 with the subject position information obtained based on the image. In this case, when the image is a still image, the associating unit 16 associates the still image with the subject position information obtained from the still image. Alternatively, when the image is a moving image, the associating unit 16 associates a moving image (frame as an example) at a predetermined time with subject position information obtained from the moving image (frame as an example) at the predetermined time. As illustrated in FIG. 3, the associating unit 16 associates one image (frame or still image) with subject position information obtained from the one image. The associating unit 16 may associate all images with subject position information.
Thereby, the associating unit 16 can associate the image at the predetermined time with the position information of the subject at the predetermined time. The associating unit 16 associates the images with the positional information continuously or intermittently in terms of time, thereby obtaining data in which the images and the positional information are associated in chronological order.

The display unit 17 temporally continuously acquires a plurality of images and position information associated by the association unit 16, and displays the images and the trajectory of the subject based on the position information in chronological order.
As an example, the display unit 17 may display a display screen that displays an image and a display screen that displays the trajectory of the subject. The display unit 17 can display the image captured by the fixed-point imaging unit 11 and the trajectory (positional information) of the subject in real time. Alternatively, the display unit 17 can display past images stored in the storage unit 19 and the trajectory (positional information) of the subject obtained from the images.

The display unit 17 displays the image of the subject captured by the fixed-point imaging unit 11 and the trajectory of the subject based on the positional information of the subject obtained by the acquisition unit 13 . The display unit 17 temporally synchronizes and displays the image of the subject and the position (trajectory) of the subject.
That is, the display unit 17 continuously displays a moving image of the subject or a still image of the subject in chronological order, and also displays the position of the subject on a map, for example. When displaying the position of the subject on the map, the display unit 17 may display the trajectory of the movement of the subject so that the movement of the subject can be understood.

As exemplified in FIG. 4, the display unit 17 displays a map and displays the trajectory of the subject on the map. In addition, when a plurality of fixed-point imaging units 11 are arranged, the display unit 17 displays an image picked up by each fixed-point imaging unit 11 . In this case, the associating unit 16 associates the image captured by one fixed-point imaging unit 11 with the position (trajectory) of the subject obtained based on the image. Display so that the correspondence can be understood. In the example shown in FIG. 4, the display unit 17 adds a display area for displaying an image captured by one fixed-point imaging unit 11c and a solid-line enclosure around the trajectory so that the correspondence can be understood. . Similarly, the display unit 17 adds a display area for displaying the image captured by the other fixed-point image capturing unit 11d and a broken-line enclosure around the trajectory so that the correspondence can be understood.

In the imaging apparatus 1 described above, the associating unit 16 associates images based on image data captured by each of the plurality of fixed-point imaging units 11 with position information. It is also possible to attach feature information indicating different features to each fixed-point imaging unit 11 that has captured the . In this case, when displaying the image and the trajectory, the display unit 17 displays the image based on the correspondence information and the trajectory based on the position information with a different characteristic image for each fixed-point imaging unit 11 based on the characteristic information. It is also possible to

The associating unit 16 associates an image showing a subject with the trajectory of the subject obtained from the image, and adds features to the association. The associating unit 16 assigns the same feature even when a plurality of subjects appear in one image and there are a plurality of trajectories of the subjects acquired from the one image. Note that when a plurality of subjects are recorded in one image, the associating unit 16 may set the trajectory of each subject to a different shape, for example, a circle, triangle, square, or the like. .
That is, the associating unit 16 attaches different features to each fixed-point imaging unit 11 that acquires an image. A feature is, for example, a color, a character, a symbol, or the like. As a specific example, when the association unit 16 assigns a red feature as the feature information, the display unit 17 displays red around the display screen that displays the image captured by the fixed-point imaging unit 11. , and the trajectory of the subject displayed on the display screen is colored red. Further, when the association unit 16 assigns a blue feature as the feature information, the display unit 17 assigns blue around the display screen that displays the image captured by the other fixed-point imaging unit 11, and displays the image. Add blue to the trajectory of the subject displayed on the screen. As a result, the association unit 16 can make associations that are easier to understand visually than distinguishing solid lines and dashed lines as illustrated in FIG. 4 .

The imaging device 1 described above may include the time information adding section 18 and the storage section 19 .
The time information addition unit 18 adds synchronized time information to the image based on the image data generated by the fixed point imaging unit 11 and the position information acquired by the acquisition unit 13 . The storage unit 19 stores image data and position information. In this case, the display unit 17 displays the image based on the image data read from the storage unit 19 and the trajectory based on the position information so as to be synchronized based on the time information added by the time information adding unit 18. may be

As an example, the time information adding unit 18 attaches the imaging time (time information) to the image data captured by the fixed-point imaging unit 11 . In this case, the time information adding section 18 may be arranged inside the fixed-point imaging section 11 or may be arranged outside the fixed-point imaging section 11 . Alternatively, when the image data is stored in the storage unit 19, the time information addition unit 18 attaches the storage time (time information). In this case, the time information addition unit 18 may be arranged inside the storage unit 19 or outside the storage unit 19 . When the image data generated by the fixed-point imaging section 11 is stored in the storage section 19 , the image data is stored in the storage section 19 as soon as it is generated by the fixed-point imaging section 11 . Therefore, the storage time and the imaging time are almost the same time.
Further, when the position information is acquired by the acquisition unit 13, the time information adding unit 18 associates the time information attached to the image used for acquiring the position information with the position information.
As a result, the time information adding section 18 can synchronize the image with the subject's position information obtained from the image. The display unit 17 can synchronize and display an image and the trajectory (positional information) of the subject appearing in the image based on the time information.

Next, a method of associating a position in an image obtained by the fixed-point imaging section 11 with a distance from the fixed-point imaging section 11 at that position will be described.
FIG. 5 is a diagram for use in explaining the acquisition of image position information.
FIG. 6 is a diagram for explaining an example of an image captured by a fixed-point imaging unit.
FIG. 7 is a graph showing the relationship between the Y coordinate and the number of pixels. The horizontal axis of FIG. 7 indicates the Y coordinate, and the vertical axis indicates the number of pixels per meter. Here, the depth direction in the image captured by the fixed-point imaging unit is the Y coordinate, and the width direction of the image is the X coordinate.
FIG. 8 is a graph for explaining an example of image position information. The horizontal axis of FIG. 8 indicates the Y coordinate, and the vertical axis indicates the distance from the position of the ground where the fixed-point imaging section is arranged to an arbitrary Y coordinate position.
FIG. 9 is a graph showing the relationship between the Y coordinate and the width ratio of the image. The horizontal axis of FIG. 9 indicates the Y coordinate, and the vertical axis indicates the width ratio of the image.

Specifically, as shown in FIG. 1, the acquisition unit 13 includes a first acquisition unit 131, a second acquisition unit 132, a third acquisition unit 133, a fourth acquisition unit 134, a fifth acquisition unit 135, a sixth acquisition unit It includes a unit 136 , a seventh acquisition unit 137 , an eighth acquisition unit 138 and a ninth acquisition unit 139 .

The first acquisition unit 131 acquires in advance the height of the subject, the angle of view of the fixed-point imaging unit 11 , and the width of the image captured by the fixed-point imaging unit 11 .
In order to acquire position information (image position information) within an image, first, the subject 110 is moved within the imaging range of the fixed-point imaging section 11 . A subject 110 (see FIG. 5) is, for example, a person. When the subject 110 is a person, for example, the height of the person (the height of the subject) is input using an input device (not shown). The first acquisition unit 131 acquires the input height of the person.
Also, the first acquisition unit 131 acquires the angle of view θ2 captured by the fixed-point imaging unit 11 . The angle of view θ2 is input using, for example, an input device (not shown). The first acquisition unit 131 acquires the input angle of view θ2. The angle of view θ2 is determined by the planar size of the imaging element (not shown) of the fixed-point imaging section 11, the focal length of the imaging lens (not shown) of the fixed-point imaging section 11, and the like.
The first acquisition unit 131 also acquires the width of the image obtained by the fixed-point imaging unit 11 (for example, the number of pixels). The width of the image (image width) is input using, for example, an input device (not shown).

The second acquisition unit 132 acquires the number of pixels forming the image data per 1 m based on the height of the subject acquired by the first acquisition unit 131 .
The second acquisition unit 132 acquires the number of pixels per 1 m in the image captured by the fixed-point imaging unit 11 at any position in the image by making the subject (person) 110 walk within the imaging range. . That is, the second acquisition unit 132 acquires the size (length) of 1 m in the image 100a based on the height of the person acquired by the first acquisition unit 131, and further acquires the image per size of 1 m. Get the number of pixels in the .
In the case illustrated in FIG. 6, the size of the imaging element (the image obtained by the fixed-point imaging section 11) used for the fixed-point imaging section 11 is 1280 pixels×960 pixels. When the subject 110 (not shown in FIG. 6) is recorded in the image illustrated in FIG. 6, the second acquisition unit 132, based on the height of the subject 110 and the number of pixels of the imaging device (image), Get the number of pixels per meter size.

The third acquisition unit 133 acquires the relationship between the Y coordinate, which is the coordinate in the depth direction of the image based on the image data, and the number of pixels per meter of the object acquired by the second acquisition unit 132 .
In the case of FIG. 6, the vertical direction of the image corresponds to the depth direction of the real space, and the vertical coordinate of the image is the Y coordinate. Also, in the case of FIG. 6, the horizontal direction of the image corresponds to the width direction of the real space, and the horizontal coordinate of the image is the X coordinate. Assuming that the upper left pixel in FIG. 6 has coordinates (0, 0) of the origin, the coordinates of the upper right pixel are (1280, 0). The coordinates of the pixel at the center position in FIG. 6 are (640, 480). The coordinates of the lower left pixel in FIG. 6 are (0,960), and the coordinates of the lower right pixel are (1280,960).
Since the fixed-point imaging unit 11 takes an image looking down from above, the upper side of the screen shown in FIG. Therefore, the Y coordinate of the origin shown in FIG.

Within the imaging range of the fixed-point imaging unit 11 (within the range surrounded by coordinates (0, 0), coordinates (1280, 0), coordinates (1280, 960), and coordinates (0, 960)), for example, a subject (person) walks vertically and horizontally, the third acquisition unit 133 acquires the number of pixels per meter at a plurality of Y-coordinate positions.

As an example is shown in FIG. 7, the relationship between the Y coordinate acquired by the third acquisition unit 133 and the number of pixels per meter is linear. Here, as shown in FIG. 5, the numerical value of the Y coordinate increases as the fixed-point imaging section 11 is approached. That is, as the Y coordinate increases (as the subject approaches the fixed-point imaging section 11), the number of pixels per meter increases. In other words, as the Y coordinate becomes smaller (as the subject moves away from the fixed-point imaging section 11), the number of pixels per meter decreases.

The fourth obtaining unit 134 obtains the actual width w of the image at the predetermined position based on the width of the image obtained by the first obtaining unit 131 and the number of pixels per meter obtained by the second obtaining unit 132. do.
As illustrated in FIG. 5, consider an image 100a at the Y coordinate position (predetermined position in the Y coordinate direction) where the subject 110 is present. Since the number of pixels in the width direction of the image captured by the fixed-point imaging unit 11 (1280 pixels in the case illustrated in FIG. 6) is obtained by the first obtaining unit 131, the fourth obtaining unit 134 obtains the number of pixels per 1 m. , the actual length of the width w of the image 100a at the predetermined position of the Y coordinate is obtained.

Based on the width w of the actual image acquired by the fourth acquisition unit 134 and the angle of view θ2 acquired by the first acquisition unit 131, the fifth acquisition unit 135 obtains the A first distance, which is the distance to the center position Pc, is acquired, and a second distance, which is the distance from the ground to the center position Pc, is acquired.
The fifth acquisition unit 135 obtains the center position of the image 100a from the fixed-point imaging unit 11 based on the actual length of the width w of the image 100a at the predetermined position of the Y coordinate and the angle of view θ2 of the fixed-point imaging unit 11. Acquire the actual distance (first distance L1) to Pc. In addition, the fifth acquisition unit 135, based on the number of pixels of the image, the number of pixels per 1 m, the actual length of the width w of the image 100a at the predetermined position of the Y coordinate, and the angle of view of the fixed-point imaging unit 11, An actual distance (second distance L2) from the center position Pc of the image 100a to the ground is acquired.

Based on the first distance L1 and the second distance L2 obtained by the fifth obtaining unit 135, the sixth obtaining unit 136 obtains the angle θ between the center line in the imaging direction of the fixed-point imaging unit 11 and the ground. That is, the sixth acquisition unit 136 uses a trigonometric function to obtain the angle θ Ask for

Based on the first distance L1 and the second distance L2 obtained by the fifth obtaining unit 135 and the angle θ obtained by the sixth obtaining unit 136, the seventh obtaining unit 137 calculates the fixed-point imaging unit 11 from the ground. Get the height H to That is, the seventh acquisition unit 137 acquires the height H from the ground to the fixed-point imaging unit 11 based on the first distance L1, the second distance L2, and the angle θ by using a trigonometric function.

Based on the height H from the ground to the fixed-point imaging unit 11 obtained by the seventh obtaining unit 137 and the angle θ obtained by the sixth obtaining unit 136, the eighth obtaining unit 138 is obtained from the imaging position where is arranged to the subject, and image position information is obtained based on obtaining a plurality of the positions of the subject.
In other words, the eighth acquisition unit 138 obtains an image from the ground position P0 where the fixed-point imaging unit 11 is arranged based on the height H from the ground to the fixed-point imaging unit 11 and the angle θ between the fixed-point imaging unit 11 and the ground. A distance L3 to a predetermined Y-coordinate position (image 100a) is obtained. The eighth acquisition unit 138 acquires information (image position information) regarding the position within the image by associating the distance L3 with the Y coordinate at the predetermined position within the image. Further, the eighth acquisition unit 138 moves the subject 110 within the imaging range of the fixed-point imaging unit 11, and obtains the subject (person) 110 from the ground position P0 where the fixed-point imaging unit 11 is arranged at an arbitrary position within the imaging range. Image position information is obtained based on obtaining the distance to . The eighth acquiring section 138 stores the image position information in the storage section 19 .

The image position information is information indicating the distance in meters from the fixed-point imaging unit 11 to the position (coordinates) of any pixel in the image captured by the fixed-point imaging unit 11 . As a specific example of the image position information, the pixel coordinates (640, 480) shown in FIG. and the pixel coordinates (640, 840) are information that the distance from the fixed-point imaging unit 11 is 20 m.

As shown in FIG. 8, the image position information is information indicating that as the Y coordinate increases, the distance from the ground position P0 where the fixed-point imaging section 11 is arranged decreases. In other words, the image position information is information indicating that the distance from the ground position P0 where the fixed-point imaging section 11 is arranged increases as the Y coordinate decreases.

Here, the ninth obtaining unit 139 obtains the actual width w of each of the plurality of images obtained by the fourth obtaining unit 134 at the reference position in the Y coordinate direction and the width at the predetermined position in the Y coordinate direction Obtain the width ratio to the Y coordinate based on the width.
That is, the fourth acquisition unit 134 acquires the actual width w of the image. At a position where the subject 110 does not exist (the Y coordinate corresponding to the position where no person is walking), the fourth acquisition unit 134 cannot acquire the actual width w of the image. Therefore, when the fourth obtaining unit 134 obtains the width w of the image at a plurality of positions of the Y coordinate, the ninth obtaining unit 139 obtains an arbitrary width w of the Y coordinate based on the width w of the image at the plurality of positions. Calculate the width of the image at the position of . For example, the ninth obtaining unit 139 sets one of the widths of the plurality of images obtained by the fourth obtaining unit 134 as the width of the image at the reference position of the Y coordinate (reference width), and sets the width of the plurality of images. A ratio (width ratio) between the width of the image other than the reference width and the reference width is calculated. The ninth acquisition unit 139 obtains a ratio (width ratio) to the reference width based on each of the plurality of images acquired by the fourth acquisition unit 134 . A graph showing this width ratio is shown in FIG.

Next, an imaging method according to one embodiment will be described.
First, a method for acquiring image data will be described.
FIG. 10 is a first flowchart for explaining an imaging method according to one embodiment.

In step ST11, the fixed-point imaging section 11 images a subject and generates image data. An image obtained from image data is, for example, a moving image or a still image.

In step ST12, the fixed-point imaging section 11 (time information addition section 18) attaches time information to the image generated in step ST11. The time information is, for example, an imaging time when an object is imaged by the fixed-point imaging unit 11, or a storage time when image data is stored in the storage unit 19, or the like. The imaging time and the storage time are almost the same time.

In step ST13, the recognition unit 12 recognizes the subject from the image based on the image data generated in step ST11. The recognition unit 12, for example, recognizes the subject based on a result learned in advance, or recognizes the subject using pattern matching.

In step ST14, the acquisition unit 13 acquires the position information of the subject recognized in step ST13. Since the position of the subject in the image is previously associated with the distance from the fixed-point imaging section 11 to that position, the recognition section 12 recognizes the position of the subject based on the position of the subject in the image generated in step ST11. Get information about location (location information).

In step ST15, the acquisition unit 13 (time information addition unit 18) associates time information with the position information acquired in step ST14. The time information is the same time as the time information attached to the image (image data) from which the position information is acquired.

In step ST16, the prediction unit 14 predicts the destination of the subject appearing in the image based on the position information and the like acquired in step ST14.

In step ST17, the selection unit 15 selects the nearest fixed-point imaging unit 11 capable of imaging the movement destination of the subject predicted in step ST16.

In step ST18, the fixed-point imaging section 11 determines whether to end generation of image data. If the generation of image data is not finished (No), the process returns to step ST11, and the fixed-point imaging unit 11 selected in step ST17 images the subject and generates image data. If the generation of image data is to end (Yes), the process ends.

Next, an image display method will be described.
FIG. 11 is a second flowchart for explaining the imaging method according to one embodiment.

In step ST21, the association unit 16 acquires image data and position information. For example, the associating unit 16 acquires the image data to which the time information is added in step ST12 and the position information of the subject to which the time information is associated in step ST15. Note that the associating unit 16 may acquire the image data to which the time information stored in the storage unit 19 is added and the position information of the subject associated with the time information stored in the storage unit 19. good.

In step ST22, the associating section 16 associates the image data with the position information of the subject based on the time information. In this case, the associating unit 16 may attach different features to each fixed-point imaging unit 11 that has acquired the image data. That is, the associating unit 16 assigns features to the image data and position information for each fixed-point imaging unit 11 .

In step ST23, the display unit 17 displays the image based on the image data acquired in step ST21 and the trajectory of the subject based on the position information acquired in the same step ST21. In this case, the display unit 17 displays the image and the trajectory of the subject together with the features added in step ST22.

According to this embodiment, the following effects are obtained.
The imaging device 1 includes a plurality of fixed-point imaging units 11, a recognition unit 12 for recognizing a subject from an image based on image data generated by each of the fixed-point imaging units 11, and position information about the position of the subject recognized by the recognition unit 12. The acquisition unit 13 that acquires, the prediction unit 14 that predicts the destination of the subject based on the position information acquired by the acquisition unit 13, and the fixed-point imaging unit 11 that is closest to the destination predicted by the prediction unit 14 are selected. Then, the selection unit 15 that causes the selected fixed-point imaging unit 11 to capture an image of the subject, the image based on the image data captured by the fixed-point imaging unit 11, and the position information acquired by the acquisition unit 13 based on the image are associated. An associating unit 16, and a plurality of images and position information associated by the associating unit 16 are successively acquired in time, and the images and the trajectory of the subject based on the position information are displayed in chronological order. and a display unit 17 .
The imaging device 1 predicts the destination of the subject and captures the subject with the same or different fixed-point imaging unit 11, so that the subject can be continuously captured even when the subject moves. That is, the imaging device 1 is suitable for imaging the same subject using a plurality of fixed-point imaging units 11 .
Further, since the imaging device 1 associates the image with the position information, it is possible to synchronize the two when displaying the image and the position information.

In the imaging device 1, the associating unit 16 associates the image based on the image data captured by each of the plurality of fixed-point imaging units 11 with the position information, and captures the image data with respect to the correspondence information indicating the association. When the display unit 17 displays an image and a trajectory, the display unit 17 displays the image based on the correspondence information and the trajectory based on the position information based on the feature information. A different characteristic image may be attached to each fixed-point imaging unit 11 and displayed.
As a result, the imaging apparatus 1 can determine which fixed-point imaging unit 11 among the plurality of fixed-point imaging units 11 captured the image, and which fixed-point imaging unit 11 captured the subject's position information (trajectory). can indicate whether

The imaging apparatus 1 includes a time information addition unit 18 that adds synchronized time information to the image based on the image data generated by the fixed point imaging unit 11 and the position information acquired by the acquisition unit 13; and a storage unit 19 that stores information. In this case, the display unit 17 displays the image based on the image data read from the storage unit 19 and the trajectory based on the position information so as to be synchronized based on the time information added by the time information adding unit 18. may be
Thereby, the imaging device 1 can synchronize the image and the position (trajectory) of the subject appearing in the image based on the time information.

In the imaging device 1, the acquisition unit 13 determines that a new subject is recognized by the recognition unit 12 based on the result of previously associating the position in the image captured by the fixed-point imaging unit 11 with the actual position of the subject. , the position information may be acquired.
Thereby, the imaging device 1 can acquire the position information of the subject.

In the imaging device 1, the acquisition unit 13 obtains the result of associating the position in the image captured by the fixed-point imaging unit 11 with the distance from the fixed-point imaging unit 11 at that position, and the distance from the fixed-point imaging unit 11. , and the result of associating the position in the real space with the position information when the recognition unit 12 recognizes a new subject, the position information may be acquired.
Accordingly, the imaging device 1 can acquire the position information of the subject based on the acquisition of the distance to the subject from the fixed-point imaging section 11 .

In the imaging device 1, the acquisition unit 13 includes a first acquisition unit 131 that acquires in advance the height of the subject, the angle of view of the fixed-point imaging unit 11, and the actual width of the image captured by the fixed-point imaging unit 11; When moving, the second acquisition unit 132 acquires the number of pixels forming the image data per 1 m based on the height of the subject acquired by the first acquisition unit 131, and the depth direction of the image based on the image data. and the number of pixels per meter of the subject acquired by the second acquisition unit 132, the width of the image acquired by the first acquisition unit 131, and , the number of pixels per meter obtained by the second obtaining unit 132, and the width of the actual image obtained by the fourth obtaining unit 134. Based on the width and the angle of view acquired by the first acquisition unit 131, the first distance, which is the distance from the fixed-point imaging unit 11 to the center position in the image, is acquired, and the distance from the ground to the center position is obtained. A fifth acquisition unit 135 that acquires a certain second distance, and the angle between the ground and the center line in the imaging direction of the fixed-point imaging unit 11 based on the first distance and the second distance acquired by the fifth acquisition unit 135 from the ground to the fixed point imaging unit 11 based on the sixth acquisition unit 136 that obtains , the first distance and the second distance acquired by the fifth acquisition unit 135, and the angle acquired by the sixth acquisition unit 136 , the height from the ground to the fixed-point imaging unit 11 obtained by the seventh obtaining unit 137, and the angle obtained by the sixth obtaining unit 136. an eighth acquiring unit 138 that acquires the position from the imaging position where the imaging unit 11 is arranged to the subject, and further acquires the distance from the fixed-point imaging unit 11 based on acquiring a plurality of the positions of the subject; It may be prepared.
As a result, the imaging device 1 acquires the distance from the fixed-point imaging unit 11 based on the height of the subject and the angle of view of the fixed-point imaging unit 11 through the processing performed by the first to eighth acquisition units 131 to 138, respectively. can do.

In the imaging device 1, the obtaining unit 13 obtains the width of the actual image obtained by the fourth obtaining unit 134 based on the width at the reference position in the Y-coordinate direction and the width at the predetermined position in the Y-coordinate direction. Therefore, a ninth acquisition unit 139 that acquires the width ratio with respect to the Y coordinate may be provided.
The imaging apparatus 1 can acquire the width of the image even if the subject (for example, a person) is not walking on the Y coordinate by acquiring the width ratio by the ninth acquiring unit 139. The imaging device 1 can acquire the distance from the fixed-point imaging section 11 at any Y coordinate.

In the imaging method, a computer performs a fixed-point imaging step in which a plurality of fixed-point imaging units 11 take images of a subject successively in time to generate image data, and an image based on the image data generated by each of the fixed-point imaging units 11. a recognition step for recognizing a subject; an acquisition step for acquiring position information relating to the position of the subject recognized in the recognition step; a prediction step for predicting a destination of the subject based on the position information acquired in the acquisition step; a selection step of selecting the fixed-point imaging unit 11 closest to the predicted moving destination and causing the selected fixed-point imaging unit 11 to capture an image of the subject; an image based on the image data captured by the fixed-point imaging unit 11; an associating step of associating the positional information obtained in the obtaining step based on, acquiring a plurality of the images and the positional information associated in the associating step consecutively in time, and chronologically arranging the images with the positional information; and a display step of displaying the trajectory of the subject based on the position information on the display unit 17 .
The imaging method predicts the destination of the subject and images the subject by the same or different fixed-point imaging unit 11. Therefore, even when the subject moves, the subject can be continuously imaged. That is, the imaging method is a method suitable for imaging the same subject using a plurality of fixed-point imaging units 11 .
In addition, since the imaging method associates the image with the position information, the two can be synchronized when displaying the image and the position information.

The imaging program provides a computer with a fixed-point imaging function for generating image data by continuously imaging a subject with a plurality of fixed-point imaging units 11, and an image based on the image data generated by each of the fixed-point imaging units 11. A recognition function that recognizes a subject, an acquisition function that acquires position information about the position of the subject recognized by the recognition function, a prediction function that predicts the destination of the subject based on the position information acquired by the acquisition function, and a prediction function a selection function of selecting the fixed-point imaging unit 11 closest to the predicted destination and imaging the subject with the selected fixed-point imaging unit 11; an image based on the image data captured by the fixed-point imaging unit 11; an associating function that associates location information acquired by the acquiring function based on the above, acquires a plurality of images and location information that are associated by the associating function successively in time, and chronologically obtains the images and the location information; and a display function of displaying the trajectory of the subject on the display unit 17 based on the position information.
The imaging program predicts the destination of the subject and images the subject with the same or different fixed-point imaging unit 11, so that the subject can be continuously imaged even when the subject moves. That is, the imaging program is a program suitable for imaging the same subject using a plurality of fixed-point imaging units 11 .
In addition, since the imaging program associates the image with the position information, it is possible to synchronize the two when displaying the image and the position information.

1 imaging device 11 fixed-point imaging unit 12 recognition unit 13 acquisition unit 131 first acquisition unit 132 second acquisition unit 133 third acquisition unit 134 fourth acquisition unit 135 fifth acquisition unit 136 seventh acquisition unit 138 eighth acquisition unit 139 9 acquisition unit 14 prediction unit 15 selection unit 16 association unit 17 display unit 18 time information addition unit 19 storage unit

Claims (9)

a plurality of fixed-point imaging units that capture images of a subject continuously in time and generate image data;
a recognition unit that recognizes the subject from an image based on image data generated by each of the fixed-point imaging units;
an acquisition unit that acquires position information about the position of the subject recognized by the recognition unit;
a prediction unit that predicts a destination of the subject based on the position information acquired by the acquisition unit;
a selection unit that selects the fixed-point imaging unit closest to the destination predicted by the prediction unit and causes the selected fixed-point imaging unit to image the subject;
an associating unit that associates an image based on image data captured by the fixed-point imaging unit with the position information acquired by the acquisition unit based on the image;
a display unit that acquires a plurality of the images associated by the associating unit and the position information consecutively in time, and displays the images and the trajectory of the subject based on the position information in chronological order;
An imaging device comprising: The associating unit associates an image based on image data captured by each of the plurality of fixed-point imaging units with position information, and the fixed-point imaging unit captures image data with respect to correspondence information indicating the association. Attach feature information indicating different features for each
When displaying the image and the trajectory, the display unit displays the image based on the correspondence information and the trajectory based on the position information with a different characteristic image for each fixed-point imaging unit based on the characteristic information. The imaging device according to claim 1. a time information addition unit that adds time information synchronized with the image based on the image data generated by the fixed point imaging unit and the position information acquired by the acquisition unit;
a storage unit that stores the image data and the position information,
The display unit displays an image based on the image data read from the storage unit and a trajectory based on the position information so as to be synchronized based on the time information added by the time information addition unit. 3. The imaging device according to 1 or 2. The acquisition unit
Position information is acquired when a new object is recognized by the recognition unit based on a result of previously associating a position within an image captured by the fixed-point imaging unit with an actual position of the object. 4. The imaging device according to any one of 3. The acquisition unit
a result of associating a position in an image captured by the fixed-point imaging unit with a distance from the fixed-point imaging unit at that position;
a result of associating the distance from the fixed-point imaging unit with a position in real space;
5. The imaging apparatus according to claim 4, wherein position information is acquired when a new subject is recognized by the recognition unit based on. The acquisition unit
a first acquisition unit that acquires in advance the height of the subject, the angle of view of the fixed-point imaging unit, and the width of the image captured by the fixed-point imaging unit;
a second obtaining unit that obtains the number of pixels constituting image data per meter based on the height of the subject obtained by the first obtaining unit when the subject moves;
a third acquiring unit for acquiring a relationship between a Y coordinate, which is a coordinate in the depth direction of an image based on image data, and the number of pixels per meter of the subject acquired by the second acquiring unit;
a fourth obtaining unit that obtains an actual image width at a predetermined position based on the width of the image obtained by the first obtaining unit and the number of pixels per meter obtained by the second obtaining unit;
A first distance, which is the distance from the fixed-point imaging unit to the center position in the image, based on the width of the actual image acquired by the fourth acquisition unit and the angle of view acquired by the first acquisition unit. and a fifth acquisition unit that acquires a second distance that is the distance from the ground to the center position;
a sixth obtaining unit that obtains an angle between the ground and a center line in the imaging direction of the fixed-point imaging unit based on the first distance and the second distance that are obtained by the fifth obtaining unit;
acquiring a height from the ground to the fixed point imaging unit based on the first distance and the second distance acquired by the fifth acquisition unit and the angle acquired by the sixth acquisition unit; a seventh acquisition unit;
Based on the height from the ground to the fixed-point imaging unit acquired by the seventh acquisition unit and the angle acquired by the sixth acquisition unit, from the imaging position where the fixed-point imaging unit is arranged to the subject an eighth acquisition unit that acquires the position of the subject and further acquires the distance from the fixed point imaging unit based on acquiring a plurality of the positions of the subject;
The imaging device according to claim 5, comprising: Based on the width at a reference position in the Y-coordinate direction and the width at a predetermined position in the Y-coordinate direction, among the widths of the actual image acquired by the fourth acquisition unit, the acquisition unit acquires the 7. The imaging apparatus according to claim 6, further comprising a ninth acquisition unit that acquires a width ratio with respect to the Y coordinate. the computer
a fixed-point imaging step in which a plurality of fixed-point imaging units capture images of a subject continuously in time to generate image data;
a recognition step of recognizing the subject from an image based on image data generated by each of the fixed-point imaging units;
an acquisition step of acquiring position information about the position of the subject recognized in the recognition step;
a prediction step of predicting a destination of the subject based on the position information acquired in the acquisition step;
a selection step of selecting the fixed-point imaging unit closest to the destination predicted in the prediction step and causing the selected fixed-point imaging unit to image the subject;
an associating step of associating an image based on image data captured by the fixed-point imaging unit with position information obtained in the obtaining step based on the image;
A display step of acquiring a plurality of images and location information associated in the association step successively in time, and displaying the images and the trajectory of the subject based on the location information in chronological order on a display unit. and,
An imaging method that performs to the computer,
a fixed-point imaging function for generating image data by capturing images of a subject continuously in time by a plurality of fixed-point imaging units;
a recognition function for recognizing the subject from an image based on image data generated by each of the fixed-point imaging units;
an acquisition function for acquiring position information about the position of the subject recognized by the recognition function;
a prediction function for predicting a destination of the subject based on the position information acquired by the acquisition function;
a selection function that selects the fixed-point imaging unit closest to the destination predicted by the prediction function and causes the selected fixed-point imaging unit to image the subject;
an associating function that associates an image based on image data captured by the fixed-point imaging unit with position information acquired by the acquisition function based on the image;
A display function that acquires a plurality of images and location information associated by the association function successively in time, and displays the images and the trajectory of the subject based on the location information in chronological order on the display unit. and,
An imaging program that realizes

Images