JP2021189587A - Display device, display method, and display program - Google Patents

Abstract

To provide a display device, a display method, and a display program capable of appropriately grasping a social distance to prevent infection of an infectious disease such as a new coronavirus infection.SOLUTION: A display device 10 includes: a detection unit 11 which detects a social distance range to prevent infection of infectious diseases of a person included in an image acquired from a camera; and a display unit 12 which displays the social distance range detected by the detection unit 11 in a display format corresponding to the risk of infection of infectious diseases.SELECTED DRAWING: Figure 1

Description

The present invention relates to a display device, a display method, and a display program.

The spread of the new coronavirus infection is spreading all over the world, and it has a great impact on society and the economy. In order to prevent droplet infection, which is one of the transmission routes such as the new coronavirus infection, it is necessary to secure a social distance (social distance), which is the physical distance between people.

As a related technique, for example, Patent Document 1 is known. Patent Document 1 discloses a technique for controlling the reach of droplets emitted from a person by adjusting the environment in the space.

International Publication No. 2020/044826

According to the technique described in Patent Document 1, it is possible to suppress droplet infection by adjusting the environment in the space. However, in the related technology such as Patent Document 1, there is a problem that it is difficult to properly grasp the social distance because the method of making the social distance visible is not considered.

In view of such problems, it is an object of the present disclosure to provide a display device, a display method, and a display program capable of appropriately grasping a social distance.

The display device according to the present disclosure has a detection unit that detects a social distance range for preventing infection of a person included in an acquired image, and the detected social distance range of the infectious disease. It is provided with a display unit for displaying in a display mode according to the risk of infection.

The display device according to the present disclosure includes a detection unit that detects a social distance range for preventing infection of a person included in the acquired image, and the foot of the person as the detected social distance range. It is provided with a display unit for displaying an area in a predetermined distance range centered on a higher position.

The display method according to the present disclosure detects a social distance range for preventing infection of a person included in an acquired image, and sets the detected social distance range as the risk of infection with the infectious disease. It is displayed in a display mode according to the sex.

The display method according to the present disclosure detects a social distance range for preventing infection of a person included in the acquired image, and the detected social distance range is higher than the feet of the person. It displays an area within a predetermined distance range centered on a position.

The display program according to the present disclosure detects a social distance range for preventing the transmission of an infectious disease of a person included in the acquired image, and the detected social distance range is the risk of infection of the infectious disease. It is a display program for causing a computer to execute a process, which is displayed in a display mode according to gender.

The display program according to the present disclosure detects a social distance range for preventing the transmission of an infectious disease of a person included in the acquired image, and the detected social distance range is higher than the feet of the person. It is a display program for causing a computer to execute a process that displays an area in a predetermined distance range centered on a position.

According to the present disclosure, it is possible to provide a display device, a display method, and a display program capable of appropriately grasping a social distance.

It is a block diagram which shows the outline of the display device which concerns on Embodiment 1. FIG. It is a block diagram which shows the structure of the social distance analysis system which concerns on Embodiment 3. It is a flowchart which shows the operation of the social distance analysis apparatus which concerns on Embodiment 3. It is a figure which shows the example of the image processed by the social distance analysis apparatus which concerns on Embodiment 3. FIG. It is a figure which shows the detection example of the joint detection process which concerns on Embodiment 3. It is a flowchart which shows the risk determination process which concerns on Embodiment 3. It is a flowchart which shows the determination result display processing which concerns on Embodiment 3. It is a figure which shows the display example of the social distance analysis apparatus which concerns on Embodiment 3. FIG. It is a figure which shows the display example of the social distance analysis apparatus which concerns on Embodiment 3. FIG. It is a figure which shows the display example of the social distance analysis apparatus which concerns on Embodiment 3. FIG. It is a figure which shows the display example of the social distance analysis apparatus which concerns on Embodiment 3. FIG. It is a block diagram which shows the structure of the social distance analysis system which concerns on Embodiment 4. It is a figure which shows the display example of the social distance analysis apparatus which concerns on Embodiment 4. It is a block diagram which shows the structure of the social distance analysis system which concerns on Embodiment 5. It is a figure which shows the display example of the social distance analysis apparatus which concerns on Embodiment 5. It is a figure which shows the display example of the social distance analysis apparatus which concerns on Embodiment 5. It is a figure which shows the display example of the social distance analysis apparatus which concerns on Embodiment 5. It is a block diagram which shows the outline of the hardware of the computer which concerns on embodiment.

Hereinafter, embodiments will be described with reference to the drawings. In each drawing, the same elements are designated by the same reference numerals, and duplicate explanations are omitted as necessary.
(Embodiment 1)
FIG. 1 shows the configuration of the display device according to the first embodiment. As shown in FIG. 1, the display device 10 according to the present embodiment includes a detection unit 11 and a display unit 12.

The detection unit 11 detects a social distance range for preventing the transmission of an infectious disease of a person included in the acquired image. The social distance range is a range (area) that defines the social distance required for the prevention of infection of a person's infectious disease. The display unit 12 displays the detected social distance range in a display mode according to the risk of infection of an infectious disease. For example, as a display mode, any one of the color, shape, size, etc. of the social distance range may be changed.

As described above, in the present embodiment, when the social distance necessary for the prevention of infectious disease is visible, the display mode of the social distance range is changed according to the risk of infection. It becomes possible to properly grasp the social distance of.

(Embodiment 2)
The display device according to the second embodiment has the same configuration as that of FIG. Similar to the first embodiment, the display device 10 according to the present embodiment includes a detection unit 11 and a display unit 12.

Similar to the first embodiment, the detection unit 11 detects a social distance range for preventing infection of a person included in the acquired image with an infectious disease. On the other hand, in the present embodiment, the display unit 12 displays a region of a predetermined distance range centered on a position higher than the feet of the person as the detected social distance range. For example, the color, shape, size, etc. of the social distance range may be fixed in advance, and a circular area of a predetermined size may be displayed in a top view centered on the height of the waist of the person. good.

As described above, in the present embodiment, when the social distance necessary for the prevention of infectious disease is visible, the social distance range is a predetermined distance range such as a circular shape at a position higher than the feet of the person. By displaying the area of, the social distance can be displayed more easily visually, and the display can be made according to the actual danger.

(Embodiment 3)
Hereinafter, the third embodiment will be described with reference to the drawings. FIG. 2 shows a configuration example of the social distance analysis system 1 according to the present embodiment. The social distance analysis system 1 measures whether a social distance is taken between people walking or the like from a camera image, and displays the measurement result to spread the spread of infectious diseases in public spaces and the like. It is a system to prevent. As shown in FIG. 2, the social distance analysis system 1 includes a social distance analysis device 100 and a camera 200.

The camera 200 is an imaging unit such as a surveillance camera that is installed at a predetermined monitoring position or the like and captures a person in the monitoring space (imaging space). The camera 200 may be a camera having a fixed orientation and installation location, a camera whose orientation can be changed such as a PTZ (Pan Tilt Zoom) camera, or a movable camera mounted on a moving body such as a drone. But it may be. The camera 200 and the social distance analyzer 100 are communicably connected via an arbitrary network. The camera 200 and the social distance analysis device 100 may be combined into one device.

The social distance analysis device 100 is an analysis device that analyzes (detects) the social distance of a person based on the image of the camera 200, and is also a display device that displays the analysis result. The social distance analysis device 100 includes an input unit 101, an acquisition unit 102, a camera calibration unit 103, a human joint detection unit 104, a coordinate calculation unit 105, a storage unit 106, a social distance determination unit 107, and a display unit 108. There is. The configuration of each part (block) is an example, and may be composed of other parts as long as the method (operation) described later is possible. Further, the social distance analysis device 100 is realized by, for example, a computer device such as a personal computer or a server that executes a program, but it may be realized by one device or by a plurality of devices. .. For example, the storage unit 106 or the display unit 108 may be used as an external device.

The input unit 101 is an input interface for acquiring information input from a user who operates the social distance analysis device 100. For example, the user is a monitor who monitors a person whose social distance is not secured from the image of the surveillance camera. The input unit 101 is, for example, a GUI (Graphical User Interface), and information according to a user's operation is input from an input device such as a keyboard, a mouse, or a touch panel. For example, the input unit 101 receives information necessary for the camera parameter calculation process of the camera calibration unit 103, and also receives a user operation for the display process of the display unit 108.

The acquisition unit 102 acquires a two-dimensional image including a person captured by the camera 200. The acquisition unit 102 acquires, for example, an image including a person (a video including a plurality of time-series images) captured by the camera 200 during a predetermined period. In addition to the acquisition from the camera 200, an image including a person prepared in advance may be acquired from a database or the like.

The camera calibration unit 103 calculates the camera parameters of the camera 200 based on the image acquired from the camera 200 (performs the camera calibration). The camera parameter (imaging parameter) is a parameter for converting a length in a three-dimensional real-world space into a length in a two-dimensional image plane. Camera parameters include internal parameters such as camera focal length, lens distortion, and optical center coordinates, and external parameters such as camera position (translation vector) and angle (rotation matrix). The camera calibration unit 103 stores the calculated camera parameters (position, angle, focal length, etc.) in the storage unit 106.

The camera parameters may be calculated from the acquired image, and the calculation method is not limited. For example, the camera calibration unit 103 calculates camera parameters based on the length of a normal vector (a line segment perpendicular to the ground) extracted from an image and a given reference length. The normal vector may be acquired from the bone line between the joint points of the person detected by the human joint detection unit 104, or may be specified by the input of the user. The bone line segment used for the normal vector may be any bone line segment contained in the skeleton of a person. A line segment of the bone at the feet (a line segment between the ankle and the knee) may be used, but since a line segment that is stable in a nearly vertical state when a person is walking is preferable, for example, a line segment of the spine (waist). The line segment between the neck and the neck) is used. Further, the given reference length is a length corresponding to a line segment used for calculation, and is, for example, the height of a person or the length of each bone (average value, etc.). The reference length used for the calculation may be input by the user or may be stored in the storage unit 106 in advance. Further, since the normal vector may be perpendicular to the ground, not only the skeleton of a person but also a vertical line such as a car or a truck may be used. Further, the camera parameters may be calculated by taking an image of a calibration object (an object whose length is known in advance) with the movement of a person or the like in the image stopped. Further, when the camera 200 is a camera such as a PTZ camera in which the angle and zoom value of the camera can be changed, the difference value changed from the reference position may be acquired and reflected in the camera parameters.

The human joint detection unit 104 detects the joint points of a person in a two-dimensional image based on the image acquired from the camera 200. For example, the human joint detection unit 104 uses a joint estimation technique (skeleton estimation technique) such as OpenPose using machine learning. The human joint detection unit 104 detects a joint point (key point) presumed to be a human joint based on the characteristics of each part of the person recognized in the image by using the joint estimation technique, and further detects it. Extract the bone line segment (bone link) that connects the cut seams. The detected joint points and bone line segments are the joint points and bone line segments estimated by the joint estimation technique, and may not match the joints and bones of an actual person. It can be said that the human joint detection unit 104 detects the skeletal structure including the joint points and the bone line segments from the feet to the head of the person. The human joint detection unit 104 recognizes a person in a plurality of time-series images (frames), and detects joint points and bone line segments of all the people recognized in each image.

The coordinate calculation unit 105 calculates the coordinates (position) of the person in the three-dimensional real-world space from the detected coordinates of the joint points and bone lines by using the calculated camera parameters. The coordinates to be calculated are the coordinates (center coordinates) of the center point of the social distance range of the person. The center point is an arbitrary point included in the bone line segment between the joint points detected by the human joint detection unit 104. The center point may be an arbitrary point of a joint point or a bone line segment from the foot of the person to the head, or may be a point of a vertical line that can be extracted in the person. The center point may be a point included in the line segment of the bone at the feet as in the case of camera calibration, but a point of the line segment that is stable in a state close to vertical when a person is walking is preferable, for example. It is a point included in the line segment of the spine. The coordinate calculation unit 105 calculates the center coordinates of all the persons in each image (frame), and stores the calculation results in the storage unit 106 in chronological order.

The storage unit 106 stores information (data) necessary for the operation (processing) of the social distance analyzer 100. For example, the storage unit 106 is a non-volatile memory such as a flash memory, a hard disk device, or the like. For example, the storage unit 106 stores the camera parameters calculated by the camera calibration unit 103 and the time-series coordinate calculation results calculated by the coordinate calculation unit 105. It can be said that the storage unit 106 is a camera parameter storage unit and also a coordinate calculation result storage unit. The storage unit 106 may use an image (video) acquired by the acquisition unit 102, information on joint points and bone line segments processed by the human joint detection unit 104, machine learning data for joint estimation, and a person's image as necessary. Memorize the height, the average value of the length of each bone, the length of social distance, etc.

The social distance determination unit (determination unit) 107 determines the state of securing the social distance of the person (risk of infection) based on the calculated position (center coordinates) of the person. For example, the social distance determination unit 107 determines a range of a predetermined length from the center coordinates (center point) of the person as the social distance range. The predetermined length of the social distance may be input by the user or may be stored in the storage unit 106 in advance. The social distance determination unit 107 determines the risk of infection based on the distance (social distance) between the positions (center coordinates) of the person. For example, the social distance determination unit 107 determines the risk of infection depending on whether or not the distance between persons is closer than a predetermined distance. In addition, the risk of infection may be determined according to whether or not the social distance range of the person overlaps with the social distance range of another person, or another person may be included in the social distance range of the person. The risk of infection may be determined depending on whether or not it is included. Further, the social distance may be determined by considering time in addition to the distance. The social distance determination unit 107 may determine whether or not a two-step alert is necessary based on one criterion, or may determine a multi-step alert level based on a plurality of criteria.

The display unit 108 displays the result of the operation (processing) of the social distance analyzer 100 on the display screen. For example, the display unit 108 is a display device such as a liquid crystal display or an organic EL (Electro Luminescence) display. The display unit 108 displays an image (video) captured by the camera 200, a determination result of the social distance determination unit 107, and the like on the display screen of the GUI. For example, the display unit 108 displays the social distance range of a person according to a determination result (alert necessity or alert level) by a two-dimensional top view (plan view) or a three-dimensional bird's-eye view. The display unit 108 changes the display mode of the social distance range of the person according to the determination result of the risk of infection (the state of securing the social distance) by the social distance determination unit 107. For example, as a display mode, the color, shape, size, etc. of the social distance range may be changed according to the risk of infection. Further, the blinking interval may be changed when the social distance range is blinked and displayed. In this embodiment, as an example, the color of the social distance range is changed according to the distance between a plurality of people. For example, the social distance range is displayed in different colors so that it is emphasized as the risk of infection increases (according to the alert level). Further, the display unit 108 displays a region having a predetermined shape centered on the position of a person as a social distance range. Any area may be used as long as the social distance can be specified. For example, the area may be displayed in a circular shape, a semicircular shape, a fan shape, an elliptical shape, or the like when viewed from above. In the present embodiment, as an example, a circular region having a predetermined size (radius) is displayed.

The display unit 108 is an example of an output unit that outputs a determination result of a social distance securing state. For example, in addition to or instead of the screen display, the judgment result of the risk of infection (state of securing social distance) is output by sound or other notification method (email via network, etc.). May be good. When outputting a sound, the output mode (volume of warning sound, pitch, interval, type of voice message, etc.) may be changed according to the risk of infection, as in the display mode.

FIG. 3 shows the operation of the social distance analyzer 100 according to the present embodiment. As shown in FIG. 3, the social distance analyzer 100 acquires an image (video) from the camera 200 (S101). The acquisition unit 102 acquires an image of a person (imaging space) for camera calibration, and acquires an image of a person (imaging space) for social distance determination. The image for camera calibration and the image for social distance determination may be images captured by the same camera 200, and the imaging timings may be the same or different. That is, the camera may be calibrated from the image captured in the first period (about 10 to 60 seconds), and then the social distance may be determined from the image captured in the second period. The camera may be calibrated from the images captured during the period and the social distance may be determined. At least, the camera calibration should be performed before the camera parameters are used in the determination of the social distance (coordinate calculation). FIG. 4 shows an example of an image acquired from the camera 200. The example of FIG. 4 is an image when six people are walking in the imaging space.

Subsequently, camera calibration processing (S102 to S103) is performed, and social distance determination processing (S104 to S108) is performed. As an example of the camera calibration process, an example using a bone line segment between the joint points of a person will be described. In the camera calibration process, first, the social distance analyzer 100 detects the joint point of a person based on the image acquired from the camera 200 (S102). The human joint detection unit 104 detects the joint points of each person in the image plane by using a joint estimation technique such as OpenPose, and extracts the bone line segment connecting the detected joint points.

Subsequently, the social distance analyzer 100 calculates the camera parameter using the detected bone line segment (S103). For example, the camera calibration unit 103 selects a line segment of a person's spine as a normal vector used for calculation. The line segment of the spine of one person in the image may be selected, or the line segment of the spine of a plurality of people may be selected. When a plurality of spine line segments are selected, the average of the line segments (vectors) may be calculated. Further, a line segment of the spine may be used, or a line segment (center line segment) corresponding to the estimated height may be used by estimating the height (entire skeleton) of the person from the line segment of the spine. The camera calibration unit 103 calculates camera parameters from the average value of the line segment of the spine of a person and the length (or height) of the spine.

For example, a normal vector (a line segment of the spine) may be projected onto a projection plane perpendicular to the ground (reference plane), and camera parameters may be obtained based on the perpendicularity of the projected normal vector to the ground. The rotation matrix (camera angle) can be obtained by evaluating the verticality of the normal vector projected on the projection plane. Also, from the difference between the length of the normal vector of the two-dimensional image projected onto the three-dimensional space and the length in the real world (the average value of the bone length) using the rotation matrix, the translation vector (position of the camera). ) And the focal distance can be obtained.

On the other hand, in the social distance determination process, first, the social distance analyzer 100 detects the joint point of a person based on the acquired image (S104). Similar to S102, the human joint detection unit 104 detects the joint points and bone line segments of each person in the image acquired from the camera 200.

Subsequently, the social distance analyzer 100 calculates the coordinates (center coordinates) of the center point of the person in the detected bone line segment using the calculated camera parameters (S105). For example, the coordinate calculation unit 105 selects a point included in the line segment (waist to neck) of the spine of the person as the center point of the person. In this example, the hip joint point (the lower end point of the spine line segment) is selected as the center point. The social distance analyzer 100 uses camera parameters to convert the coordinates of the spine line segment (lumbar joint point) in the two-dimensional image plane into the coordinates (x, y, z) in the three-dimensional real-world space. do. The start point coordinates and end point coordinates of the line segment (vector) may be converted into three-dimensional coordinates to obtain the center coordinates, or the start point coordinates or end point coordinates of the line segment may be converted into three-dimensional coordinates to obtain the center coordinates. good.

FIG. 5 shows an example in which a person's joint point and bone line segment are detected from the image of FIG. As shown in FIG. 5, the joint points and bone line segments of each of the six persons in the image are detected. By detecting the joint points and bone line segments of a person from a plurality of time-series images (frames), the joint points and bone line segments of a person in a walking (moving) state are detected. In the example of FIG. 5, the three-dimensional coordinates of the center point of the waist are calculated from the line segment of the spine for each of the six persons in the image.

Subsequently, the social distance analyzer 100 determines the social distance range of the person from the calculated center coordinates of the person (S106). The social distance determination unit 107 sets a range of a three-dimensional space having a predetermined length (for example, 1 m to 2 m) from the center coordinates of a person as a social distance range. In this example, the social distance range is a circular region in top view with a predetermined length as the radius from the center point. In the example of FIG. 5, for each of the six people in the image, a circular region within a social distance range from the center coordinates is determined.

Subsequently, the social distance analyzer 100 determines the risk of infection of the person based on the center coordinates (position) of the person (S107). FIG. 6 shows a specific example of the risk determination process according to the present embodiment. FIG. 6 is an example of determining the necessity of a two-step alert based on one criterion. As shown in FIG. 6, the social distance determination unit 107 acquires a distance between persons (S201) and determines whether or not the acquired distance is shorter than a predetermined distance (S202). The social distance determination unit 107 calculates the distance in the three-dimensional real-world space between the center coordinates of a person and the center coordinates of another person.

If the distance between the persons is shorter than the predetermined distance, the social distance determination unit 107 determines that an alert is required (high risk of infection, social distance is not secured) (S203), and the person. If the distance between them is longer than the predetermined distance, it is determined that no alert is required (the risk of infection is low and the social distance is secured) (S204). The predetermined distance as a determination criterion may be the same as the distance used in the social distance range (for example, 1 m to 2 m), or may be a longer distance. In the example of FIG. 5, the distance between each of the six people in the image is calculated, and the necessity of an alert between each person is determined.

In addition, the necessity of alert may be determined based on the determined social distance range. For example, if the social distance range of a person overlaps with the social distance range of another person, it is judged that an alert is required, and if the social distance range of the person and the social distance range of another person do not overlap, an alert is required. It may be determined that it is unnecessary. Further, if another person is included in the social distance range of the person, it may be determined that an alert is required, and if the social distance range of the person does not include another person, it may be determined that an alert is unnecessary.

Further, the social distance determination unit 107 may determine the risk of infection from the distance between people for each image (frame), or analyze the positions of a plurality of images (frames) in the time series direction. The risk of infection may be determined in consideration of the movement of the analyzed person. For example, if the distance between the persons continues to be shorter than the predetermined distance for a certain period of time (when the person stays), it may be determined that an alert is required. This makes it possible to temporarily exclude people who pass each other from the judgment.

Subsequently, the social distance analyzer 100 displays the danger determination result (S108). FIG. 7 shows a specific example of the determination result display process according to the present embodiment. As shown in FIG. 7, the display unit 108 acquires the determination result of the social distance determination unit 107 (S301), and determines whether or not the acquired determination result requires an alert (S302). When the determination result requires an alert (when the risk of infection is high), the display unit 108 displays the social distance range of the person who requires the alert in red (S303), and when the determination result does not require an alert (S303). (When the risk of infection is low), the social distance range of the person who does not need an alert is displayed in blue (S304).

8 and 9 show display examples when an alert is not required and an alert is required. As shown in FIGS. 8 and 9, the display unit 108 displays a three-dimensional bird's-eye view V1 and a two-dimensional top view V2 on the display screen (display window) W1. That is, the display screen W1 has a bird's-eye view display area for displaying the bird's-eye view V1 and a top view display area for displaying the top view V2, and the display unit 108 displays the determination result of the social distance in each area. .. The bird's-eye view V1 and the top view V2 can display a plurality of images (videos) and social distance ranges in chronological order. The display unit 108 may display both the bird's-eye view V1 and the top view V2, or may display either one.

The bird's-eye view V1 is a three-dimensional display view of the imaging space as seen from the camera 200 (predetermined viewpoint). The display unit 108 displays the input video (image) on the bird's-eye view V1 and synthesizes the social distance range of the person with the video and displays it in a three-dimensional format. It should be noted that the viewpoint may be changed based on the images of a plurality of cameras. In the bird's-eye view V1, the center point P1 (for example, a circle) of the person is displayed at the three-dimensional position corresponding to the calculated center coordinates for all the people in the image, and the social distance range determined from the center point P1 is set. The social distance range D1 of the person is displayed in the corresponding three-dimensional range. In this example, the social distance range D1 is a circular (disk-shaped) region having a predetermined size centered on the center point P1 of the person, as in the above-mentioned social distance range determination process. As an example of displaying the social distance range, the contour line of the circular area is displayed, but the inside of the circular area may be displayed in the same color as the contour line or another pattern. The range of a predetermined distance is not limited to a disk shape, and may be three-dimensionally displayed in a spherical shape. In the bird's-eye view V1, the social distance range D1 which is circular in the top view is three-dimensionally displayed in the size and shape according to the position. Further, in the bird's-eye view V1, the center point P1 and the social distance range D1 are displayed according to the positions of the moving person in the image. The bird's-eye view V1 may display the joint points and bone line segments detected from each person.

The top view V2 is a two-dimensional plan view of the same imaging space as the bird's-eye view V1 as viewed from above. The display unit 108 displays an image of the position of a person in the input video (image) and the social distance range in a two-dimensional format on the top view V2. In addition to the person, an image of another object may be displayed, or an image may be displayed in the same manner as the bird's-eye view. Similar to the bird's-eye view, the top view V2 displays the center point P2 (for example, x mark) of the person at the two-dimensional position corresponding to the calculated center coordinates for all the people in the image, and is determined from the center point P2. A circular social distance range D2 is displayed in a two-dimensional range corresponding to the social distance range. Similar to the bird's-eye view V1, the contour line of the circular region is displayed as a display example of the social distance range, but the inside of the circular region may be displayed in the same color as the contour line or another pattern. Further, in the top view V2, as in the bird's-eye view V1, the center point P2 and the social distance range D2 are displayed according to the position of the moving person in the image.

For example, in FIGS. 8 and 9, the color (display mode) to be displayed is changed according to the distance between people. In FIG. 8, the distance between all the persons A1 to A6 is larger than the predetermined distance. Therefore, in the bird's-eye view V1 and the top view V2, the social distance ranges D1 and D2 and the center points P1 and P2 of the persons A1 to A6 are displayed in blue. On the other hand, in FIG. 9, although the distance between the person A3 and A6 is longer than the predetermined distance, the distance between the person A1 and the object A2 is shorter than the predetermined distance (part of the social distance range overlaps). Yes). Therefore, in the bird's-eye view V1 and the top view V2, the social distance ranges D1 and D2 of the persons A3 to A6 and the center points P1 and P2 are displayed in blue, and the social distance ranges D1 and D2 of the persons A1 and the person A2 are displayed. Center points P1 and P2 are displayed in red. In this example, red (highlighted color) / blue is used to clearly indicate that the risk of infection is high / low, but if the risk of infection can be identified, other colors are used. May be good. Further, the color is not limited to two colors, and gradation may be displayed in multiple stages according to the overlap of distances and social distance ranges. As the display mode, not only the color but also the blinking interval and the like may be changed. For example, the blinking display may be made when an alarm is required, and the blinking interval may be shortened as the distance becomes shorter (as the alert level increases). Further, a mark such as an arrow or a character string indicating that the risk of infection is high may be displayed. A numerical value of the distance between people may be displayed. In addition, the color of a person in a social distance range with a high risk of infection may be changed.

As in the above-mentioned risk determination process, the color (display mode) may be changed according to the risk based on the social distance range. For example, when the social distance range of the first person and the social distance range of the second person overlap, the social distance range of the first person and the second person is displayed in red, and the first person is displayed. If the social distance range of the first person and the social distance range of the second person do not overlap, the social distance range of the first person and the second person may be displayed in blue. In addition, the colors may be changed according to the amount of overlap of those social distance ranges. When the social distance range of the first person includes the second person, the social distance range of the first person is displayed in red, and the social distance range of the first person is the second person. If is not included, the social distance range of the first person may be displayed in blue. Further, the color may be changed according to the density of a plurality of people. For example, the color is changed according to the number of people included in a predetermined area, the number of overlapping social distance ranges of persons (overlapping area), and the number of other persons included in the social distance range of persons. You may.

Further, in the examples of FIGS. 8 and 9, in the bird's-eye view V1, the center point P1 and the circular social distance range D1 are displayed at the position (height) of the waist of the person. That is, the center point P1 and the social distance range D1 are displayed while floating at a position higher than the feet of the person. Since the closer to the face, the higher the possibility of droplet infection, displaying the social distance range at a higher position of the person can make the display more suitable for the risk of infection. For example, the waist height displaying the center point P1 and the social distance range D1 is the height of the hip joint point detected in the joint detection, and is the height obtained by calculating the coordinates of the person. Since the line segment of the spine is used to calculate the coordinates, the height of the lower end (waist) of the spine may be used, or any height of the spine (waist to neck) may be used. Since the positions of the spine and hips are stable when a person walks, displaying the spine and hips as the center makes it easier to grasp the social distance range.

For example, the center point P1 and the social distance range D1 may be displayed at an arbitrary position (height) higher than the feet of the person. It may be displayed at an arbitrary height between the feet and the waist, or may be displayed at an arbitrary height from the waist to the head. FIG. 10 is an example of displaying the center point P1 and the social distance range D1 at the position (height) of the knee of a person. For example, the line segment of the spine may be displayed at a height between the waist and the feet on an extension line extending from the waist to the feet. In addition, when the coordinates of a person are calculated using the bone line segment from the knee to the feet or the bone line segment from the knee to the waist, it is displayed at the height of the knee included in the line segment used for the calculation. You may. FIG. 11 is an example of displaying the center point P1 and the social distance range D1 at the position (height) of the face of a person. For example, the line segment of the spine may be displayed at a height near the head on an extension line extending from the neck to the head. Further, when the coordinates of a person are calculated using the bone line segment from the neck to the head, the height of the face included in the line segment used for the calculation may be displayed.

As described above, in the present embodiment, the camera is calibrated from the image captured by the camera, the position and social distance of the person are calculated, and the calculated position and social distance of the person are shown in a society having a predetermined shape. The target distance range is displayed on the display screen. In particular, on this display screen, the display mode such as the color of the social distance range is changed according to the risk of infection. As a result, the social distance of the person in the image can be grasped at a glance, and the person at high risk of infection (the social distance is not secured) can be clearly identified.

(Embodiment 4)
Hereinafter, the fourth embodiment will be described with reference to the drawings. In the present embodiment, an example in which the display mode of the social distance range is changed according to the direction of the face of the person in the social distance analyzer according to the third embodiment will be described.

FIG. 12 shows a configuration example of the social distance analysis system 1 (social distance analysis device 100) according to the present embodiment. As shown in FIG. 12, the social distance analyzer 100 according to the present embodiment further includes a face direction detecting unit 109 in addition to the configuration of FIG. 2 of the third embodiment.

The face direction detection unit 109 detects the direction of the face of a person included in the image acquired from the camera 200. The face direction detection unit 109 recognizes the face of a person in an image by using, for example, a face recognition technique using machine learning, and detects the direction of the face from the positions of the eyes and nose of the recognized face. For example, the risk of droplet infection varies before and after a person's face, with a high risk of infection in front of the face and a low risk of infection behind the face. Therefore, in the present embodiment, the danger determination criteria and the display of the social distance range are changed according to the direction of the face.

When determining the social distance range from the center point of the person, the social distance determination unit 107 determines the social distance range so that the front side is wider (longer distance) than the rear side of the person's face. For example, the social distance range is a semicircular or fan-shaped area extending in front of the face. For example, the social distance determination unit 107 determines the risk of infection based on the determined social distance range. In addition, the social distance determination unit 107 changes the determination criteria according to the orientation of the person's face, determines the risk of infection by a long threshold value (longer threshold value than the rear side) in front of the person's face, and determines the risk of infection of the person. Behind the face, a short threshold may be used to determine the risk of infection.

The display unit 108 changes the display mode of the social distance range according to the orientation of the face of the person. That is, when displaying the social distance range of the person, the display unit 108 is centered on the position of the person so that the front side of the person's face is wider than the back side as in the determination of the social distance range. Display a semi-circular or fan-shaped area in top view. FIG. 13 shows a display example in the present embodiment. In the example of FIG. 13, in the bird's-eye view V1 and the top view V2, as the social distance ranges D1 and D2, a semicircular region is displayed in a top view extending only to the front of the face excluding the back of the face. The social distance range D1 similar to that of the third embodiment may be displayed, and the area defined as the social distance range in the present embodiment may be displayed as an easily infected area.

As described above, in the present embodiment, the display mode of the social distance range is changed according to the orientation of the face of the person as the display mode according to the risk of infection of the infectious disease. This makes it possible to display the social distance range according to the risk of infection. In addition, the risk of infection can be determined more appropriately by changing the criteria for determining the social distance according to the orientation of the person's face.

(Embodiment 5)
Hereinafter, the fifth embodiment will be described with reference to the drawings. In the present embodiment, an example in which the display mode of the social distance range is changed according to the moving direction and speed of the person in the social distance analyzer according to the third or fourth embodiment will be described.

FIG. 14 shows a configuration example of the social distance analysis system 1 (social distance analysis device 100) according to the present embodiment. As shown in FIG. 14, the social distance analyzer 100 according to the present embodiment further includes a movement detection unit 110 in addition to the configuration of FIG. 2 of the third embodiment.

The movement detection unit 110 detects the movement of a person included in the image acquired from the camera 200. For example, the movement detection unit 110 analyzes the coordinates of the person stored in the storage unit 106 in time series, and detects the movement direction and the movement speed of the person. For example, when an infected person is running, the infected person's droplets may fly over a wider area than when walking, especially behind the person. Therefore, in the present embodiment, the danger determination criteria and the display of the social distance range are changed according to the moving direction and speed of the person.

When the social distance determination unit 107 determines the social distance range from the center point of the person, the range extending in the moving direction of the person (the distance becomes longer) is set as the social distance range. For example, the social distance range is an elliptical shape centered on the center point of the person, and the range behind the person is lengthened (widened) according to the moving speed. For example, the social distance determination unit 107 determines the risk of infection based on the determined social distance range. Further, the social distance determination unit 107 changes the determination criteria according to the movement of the person, determines the risk of infection by a long threshold value (threshold value longer than the left-right direction) in the front-back direction of the movement of the person, and determines the risk of infection of the person. The risk of infection may be determined by a short threshold in the left-right direction of movement.

The display unit 108 changes the display mode of the social distance range according to the movement direction or the movement speed of the person. That is, when displaying the social distance range of a person, the display unit 108 extends in the moving direction of the person (so that the front-back direction is wider than the left-right direction) as in the determination of the social distance range. , The elliptical area is displayed in the top view centered on the position of the person, and the rear range is displayed longer (wider) according to the moving speed. 15 to 17 show display examples in this embodiment. In the example of FIG. 15, in the bird's-eye view V1 and the top view V2, the social distance ranges D1 and D2 are displayed, for example, in an elliptical shape extending in the front-back direction of movement. In the example of FIG. 16, the length of the elliptical shape of the social distance ranges D1 and D2 in the long axis direction is changed according to the speed of the person. For example, when the moving speed of the person A1 is faster than that of another person, the social distance ranges D1 and D2 of the person A1 are displayed longer in the moving direction than the other person. Display so that the long axis of the ellipse becomes longer as the moving speed increases. Further, it may be displayed so that the rear side of the person is longer than the front side according to the moving speed. Further, as shown in FIG. 17, the social distance ranges D1 and D2 may be displayed in a circle, and the size of the circle may be changed according to the moving speed. For example, when the moving speed of the person A1 is faster than that of another person, the circles of the social distance ranges D1 and D2 of the person A1 are displayed larger than those of the other person. The circle is displayed so that it becomes larger as the moving speed increases. The social distance range D1 similar to that of the third embodiment may be displayed, and the area defined as the social distance range in the present embodiment may be displayed as an easily infected area.

As described above, in the present embodiment, the display mode of the social distance range is changed according to the moving direction and speed of the person as the display mode according to the risk of infection of the infectious disease. This makes it possible to display the social distance range according to the risk of infection. In addition, the risk of infection can be determined more appropriately by changing the criteria for determining the social distance according to the moving direction and speed of the person.

(Other embodiments)
Not limited to the above embodiment, in the determination of social distance (risk of infection), a pair of persons presumed to be parent and child may be excluded from the determination target. For example, if a tall person and a short person are close to each other for a certain period of time, it is presumed that the two people are parents and children. In this case, it is not necessary to display the alert because it is determined that the alert is unnecessary even if the distance between the persons is short.

In addition, the display mode of the alert may be changed depending on whether or not the person wears a mask or a face shield. When a person wears a mask or the like, the risk of droplet infection is low, so the display mode indicates that the risk is lower than when the person does not wear a mask or the like. For example, when it is detected that a person is wearing a mask or the like using face recognition technology, it is judged that an alert is unnecessary even if the distance between the persons is short, and the alert is not displayed. Alternatively, a mask or the like may be used to indicate that the risk is low (for example, neutral color display). Similarly, if there is an object that prevents splashing, such as a vinyl wall, between people, it may be displayed to indicate that the risk of infection is low.

The present disclosure is not limited to the above embodiment, and can be appropriately modified without departing from the spirit.

Each configuration in the above-described embodiment is composed of hardware and / or software, and may be composed of one hardware or software, or may be composed of a plurality of hardware or software. Each device and each function (processing) may be realized by a computer 20 having a processor 21 such as a CPU (Central Processing Unit) and a memory 22 which is a storage device, as shown in FIG. For example, a program for performing the method in the embodiment (for example, a collation method in the cyber physical person attribute collation device) is stored in the memory 22, and each function is realized by executing the program stored in the memory 22 on the processor 21. You may.

These programs can be stored and supplied to a computer using various types of non-transitory computer readable medium. Non-temporary computer-readable media include various types of tangible storage media. Examples of non-temporary computer-readable media include magnetic recording media (eg, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (eg, magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs. Includes CD-R / W, semiconductor memory (eg, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (random access memory)). The program may also be supplied to the computer by various types of transient computer readable medium. Examples of temporary computer-readable media include electrical, optical, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

Some or all of the above embodiments may also be described, but not limited to:
(Appendix 1)
A detector that detects the social distance range to prevent the transmission of infectious diseases of the person included in the acquired image,
A display unit that displays the detected social distance range in a display mode according to the risk of infection of the infectious disease, and a display unit.
A display device.
(Appendix 2)
The display unit changes any of the colors, shapes, and sizes of the social distance range as the display mode.
The display device according to Appendix 1.
(Appendix 3)
The display unit displays the social distance range so as to be emphasized as the risk of infection increases.
The display device according to Appendix 1 or 2.
(Appendix 4)
The display unit displays a region having a predetermined shape centered on the position of the person as the social distance range.
The display device according to any one of Supplementary note 1 to 3.
(Appendix 5)
The predetermined shape is any of a circular shape, a semicircular shape, a fan shape, and an elliptical shape when viewed from above.
The display device according to Appendix 4.
(Appendix 6)
The display unit changes the display mode of the social distance range according to the distance between a plurality of persons included in the image.
The display device according to any one of Supplementary note 1 to 5.
(Appendix 7)
The display unit of the first person and the second person depends on whether or not the distance between the first person and the second person included in the image is closer than a predetermined distance. Change the display mode of the social distance range,
The display device according to Appendix 6.
(Appendix 8)
The display unit has the first person and the second person depending on whether or not the social distance range of the first person and the social distance range of the second person included in the image overlap. Change the display mode of the social distance range of
The display device according to Appendix 6.
(Appendix 9)
The display unit has a social distance range between the first person and the second person according to the amount of overlap between the social distance range of the first person and the social distance range of the second person. Change the display mode of
The display device according to Appendix 8.
(Appendix 10)
The display unit changes the display mode of the social distance range of the first person depending on whether or not the second person is included in the social distance range of the first person included in the image. ,
The display device according to Appendix 6.
(Appendix 11)
The display unit changes the display mode of the social distance range according to the density of a plurality of people included in the image.
The display device according to any one of Supplementary note 1 to 10.
(Appendix 12)
The display unit changes the display mode of the social distance range according to the number of overlapping social distance ranges of a plurality of persons included in the image.
The display device according to Appendix 11.
(Appendix 13)
The display unit changes the display mode of the social distance range according to the number of other persons included in the social distance range of the person included in the image.
The display device according to Appendix 11.
(Appendix 14)
The display unit changes the display mode of the social distance range according to the orientation of the face of the person.
The display device according to any one of Supplementary note 1 to 13.
(Appendix 15)
The display unit displays the social distance range so that the front side is wider than the back side of the person's face.
The display device according to Appendix 14.
(Appendix 16)
The display unit changes the display mode of the social distance range according to the moving direction of the person.
The display device according to any one of Supplementary note 1 to 15.
(Appendix 17)
The display unit displays the social distance range so that the front and back are wider than the left and right with respect to the moving direction of the person.
The display device according to Appendix 16.
(Appendix 18)
The display unit changes the display mode of the social distance range according to the moving speed of the person.
The display device according to any one of Supplementary note 1 to 17.
(Appendix 19)
The display unit displays so that the social distance range increases as the moving speed of the person increases.
The display device according to Appendix 18.
(Appendix 20)
The display unit displays the social distance range in a bird's-eye view display area of a display screen in a three-dimensional format viewed from a predetermined viewpoint.
The display device according to any one of Supplementary note 1 to 19.
(Appendix 21)
The display unit displays the social distance range together with the acquired image in the bird's-eye view display area.
The display device according to Appendix 20.
(Appendix 22)
The display unit displays the social distance range in the top view display area of the display screen in a two-dimensional format viewed from the top surface.
The display device according to any one of Supplementary note 1 to 21.
(Appendix 23)
The display unit displays the social distance range between the bird's-eye view display area and the top view display area in chronological order.
The display device according to any one of Supplementary Provisions 20 to 22.
(Appendix 24)
A detector that detects the social distance range to prevent the transmission of infectious diseases of the person included in the acquired image,
As the detected social distance range, a display unit that displays an area of a predetermined distance range centered on a position higher than the feet of the person, and a display unit.
A display device.
(Appendix 25)
The height of the center is the height of the waist of the person,
The display device according to Appendix 24.
(Appendix 26)
The region of the predetermined distance range is a circular region when viewed from above.
The display device according to Appendix 24 or 25.
(Appendix 27)
Detects the social distance range to prevent the transmission of infectious diseases of the person contained in the acquired image,
The detected social distance range is displayed in a display mode according to the risk of infection of the infectious disease.
Display method.
(Appendix 28)
In the display, any one of the color, shape, and size of the social distance range is changed as the display mode.
The display method described in Appendix 27.
(Appendix 29)
Detects the social distance range to prevent the transmission of infectious diseases of the person contained in the acquired image,
As the detected social distance range, a region of a predetermined distance range centered on a position higher than the feet of the person is displayed.
Display method.
(Appendix 30)
The height of the center is the height of the waist of the person,
The display method described in Appendix 29.
(Appendix 31)
Detects the social distance range to prevent the transmission of infectious diseases of the person contained in the acquired image,
The detected social distance range is displayed in a display mode according to the risk of infection of the infectious disease.
A display program that lets a computer perform processing.
(Appendix 32)
In the display, any one of the color, shape, and size of the social distance range is changed as the display mode.
The display program described in Appendix 31.
(Appendix 33)
Detects the social distance range to prevent the transmission of infectious diseases of the person contained in the acquired image,
As the detected social distance range, a region of a predetermined distance range centered on a position higher than the feet of the person is displayed.
A display program that lets a computer perform processing.
(Appendix 34)
The height of the center is the height of the waist of the person,
The display program described in Appendix 33.

1 Social distance analysis system 10 Display device 11 Detection unit 12 Display unit 20 Computer 21 Processor 22 Memory 100 Social distance analysis device 101 Input unit 102 Acquisition unit 103 Camera calibration unit 104 Human joint detection unit 105 Coordinate calculation unit 106 Storage unit 107 Social distance determination unit 108 Display unit 109 Face direction detection unit 110 Movement detection unit 200 Camera

Claims (10)

A detector that detects the social distance range to prevent the transmission of infectious diseases of the person included in the acquired image,
A display unit that displays the detected social distance range in a display mode according to the risk of infection of the infectious disease, and a display unit.
A display device. The display unit changes any of the colors, shapes, and sizes of the social distance range as the display mode.
The display device according to claim 1. The display unit changes the display mode of the social distance range according to the distance between a plurality of persons included in the image.
The display device according to claim 1 or 2. The display unit changes the display mode of the social distance range according to the orientation of the face of the person.
The display device according to any one of claims 1 to 3. The display unit changes the display mode of the social distance range according to the movement direction or the movement speed of the person.
The display device according to any one of claims 1 to 4. A detector that detects the social distance range to prevent the transmission of infectious diseases of the person included in the acquired image,
As the detected social distance range, a display unit that displays an area of a predetermined distance range centered on a position higher than the feet of the person, and a display unit.
A display device. Detects the social distance range to prevent the transmission of infectious diseases of the person contained in the acquired image,
The detected social distance range is displayed in a display mode according to the risk of infection of the infectious disease.
Display method. Detects the social distance range to prevent the transmission of infectious diseases of the person contained in the acquired image,
As the detected social distance range, a region of a predetermined distance range centered on a position higher than the feet of the person is displayed.
Display method. Detects the social distance range to prevent the transmission of infectious diseases of the person contained in the acquired image,
The detected social distance range is displayed in a display mode according to the risk of infection of the infectious disease.
A display program that lets a computer perform processing. Detects the social distance range to prevent the transmission of infectious diseases of the person contained in the acquired image,
As the detected social distance range, a region of a predetermined distance range centered on a position higher than the feet of the person is displayed.
A display program that lets a computer perform processing.

Images