JP2010220077A - Display device, method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a display device, method and program, capable of efficiently monitoring an object to be monitored. <P>SOLUTION: The display device detects a predetermined object from respective pieces of video information imaged by a plurality of imaging units, and acquires positional information of the predetermined detected object, and image information of the object. The display device converts the acquired positional information to positional information in a display video being video information displayed by an internal display unit of each piece of the video information, and displays the acquired video information of the object along with the display video using the positional information obtained as the results of the conversion. The plurality of imaging units are used for imaging the same region from positions different from one another, and the display device acquires coordinate information in the region as positional information. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a display device, a display method, and a display program.

  A monitoring system that displays each of monitoring images obtained from a plurality of monitoring cameras on a monitor is known. A user who uses the monitoring system monitors a monitoring target (for example, a person) using a monitoring image (also referred to as a display image) displayed on the monitor. In addition, when there is a blind spot in the display image, the user displays another monitoring image corresponding to the blind spot and monitors the blind spot.

  Here, conventionally, a plurality of techniques for displaying a plurality of monitoring images on a single monitor are known. In a plurality of technologies, a plurality of monitoring images are simultaneously displayed on one monitor by reducing the size on the monitor that displays each monitoring image. In addition, a switching technique for switching the display video to another monitoring image is used. In the switching technique, a plurality of monitoring images are sequentially displayed on one monitor by limiting the time for which one monitoring image is displayed. In addition, increasing techniques are used to increase the number of monitors. In the increase technology, a plurality of monitoring images are displayed on a plurality of monitors.

JP 7-322239 A (page 1-3, FIG. 2) Japanese Patent Laying-Open No. 2003-244683 (page 1-3, FIG. 1)

  By the way, the above-described conventional technique has a problem that it is difficult to efficiently monitor the monitoring target. For example, when there is a blind spot in the display image, another monitoring image corresponding to the blind spot has to be displayed.

  Further, even if the above-described plural techniques are used, the size on the monitor on which the individual monitoring images are displayed is reduced, and it is difficult to monitor the monitoring target. Further, even when the switching technique is used, the time for displaying one monitoring image is limited, and it is difficult to monitor the monitoring target. Moreover, even if the increase technology is used, it is difficult for the user to check a plurality of monitors, and it is difficult to monitor the monitoring target.

  Accordingly, the present invention has been made to solve the above-described problems of the prior art, and an object thereof is to provide a display device, a display method, and a display program capable of efficiently monitoring a monitoring target. To do.

  In order to solve the above-described problems and achieve the object, the display device includes a detection unit that detects a predetermined object from each of video information captured by a plurality of imaging units. The display device further includes an acquisition unit that acquires position information of the predetermined object detected by the detection unit and video information of the object. In addition, the display device includes a conversion unit that converts the position information acquired by the acquisition unit into position information in a display video that is video information displayed by the internal display unit of each of the video information. In addition, the display device includes a display control unit that displays the video information of the object acquired by the acquisition unit together with the display video using the position information obtained as a result of conversion by the conversion unit.

  It is possible to efficiently monitor the monitoring target.

FIG. 1 is a diagram for explaining the outline of the display device according to the first embodiment. FIG. 2 is a block diagram for explaining an example of the configuration of the display device according to the first embodiment. FIG. 3 is a diagram for explaining an example of the arrangement of the imaging units in the first embodiment. FIG. 4 is a diagram for explaining position measurement by the detection unit in the first embodiment. FIG. 5 is a diagram for explaining the image acquisition unit according to the first embodiment. FIG. 6 is a diagram for explaining an example of a mark display by the overlay unit in the first embodiment. FIG. 7 is a diagram for explaining an example of the moving direction display by the overlay unit in the first embodiment. FIG. 8 is a flowchart for explaining an example of the overall processing flow of the display apparatus according to the first embodiment. FIG. 9 is a flowchart for explaining an example of the flow of display video selection processing using identification information in the first embodiment. FIG. 10 is a block diagram for explaining an example of the configuration of the display device according to the second embodiment. FIG. 11 is a diagram for explaining an example of the arrangement of the imaging units in the second embodiment. FIG. 12 is a schematic diagram illustrating an example of a program for the display device according to the first embodiment. FIG. 13 is a diagram for explaining pixel number calculation processing by the overlay unit in the first embodiment.

  Hereinafter, embodiments of a display device, a display method, and a display program according to the present invention will be described in detail with reference to the accompanying drawings. In the following, an overview of the display device according to the first embodiment, a configuration of the display device, and a processing flow will be described in order, and then other embodiments will be described.

[Outline of display device]
First, the outline of the display device according to the first embodiment will be described with reference to FIG. FIG. 1 is a diagram for explaining the outline of the display device according to the first embodiment.

  As shown in (1) of FIG. 1, the display device according to the first embodiment includes a plurality of imaging units that capture monitoring images (also referred to as video information), and the plurality of imaging units are identically monitored from different positions. Image the area.

  In the example shown in (1) of FIG. 1, the imaging unit “1” and the imaging unit “2” capture video information about a monitoring area where the person 10 and the shelf 20 exist. Here, the imaging unit “1” captures video information including the person 10 captured in the center and the shelf 20, and the imaging unit “2” captures video information including only the shelf 20. The reason why the person 10 is not included in the video information captured by the imaging unit “2” is that the person 10 is located in the blind spot by the shelf 20 when viewed from the imaging unit “2”. The imaging unit “2” will be described as imaging the person 10 on the left side of the video information when the influence of the blind spot due to the shelf 20 is not considered. In addition, the display device according to the first embodiment includes a display unit that displays video information captured by the imaging unit.

  As illustrated in (2) of FIG. 1, the display device according to the first embodiment detects a predetermined object from each piece of video information captured by the imaging unit, for example, from the video information “1”. As a result, the person 10 is detected.

  Then, as shown in (3) and (4) of FIG. 1, the display device according to the first embodiment acquires the detected position information of the predetermined object and the video information of the object. For example, from the video information “1”, “center of video information” is acquired as the position information of the person 10 in the video information “1”, and the video information of the person 10 is acquired.

  As shown in (5) of FIG. 1, the display device according to the first embodiment converts the acquired position information into position information in a display image that is image information displayed by an inner display unit of each image information. And convert. For example, the display device converts “the center of the video information” that is the position of the person 10 in the video information “1” to “the left side of the video information” that is the position of the person 10 in the video information “2”. .

  As shown in FIG. 1 (6), the display device according to the first embodiment displays the acquired video information of the object together with the display video using the position information obtained as a result of the conversion. For example, the display device displays the acquired video information of the person 10 on the shelf 20 at a position on the left side of the video information “2”.

  For this reason, the display device according to the first embodiment can efficiently monitor the monitoring target.

[Configuration of display device]
Next, an example of the configuration of the display device 100 illustrated in FIG. 1 will be described with reference to FIG. FIG. 2 is a block diagram for explaining the configuration of the display device. As illustrated in FIG. 2, the display device 100 includes an imaging unit 101, a display unit 102, and a control unit 200.

  The imaging unit 101 is connected to the control unit 200. The imaging unit 101 captures video information, for example, a surveillance camera. Note that the video information corresponds to a still image or a moving image captured by the imaging unit 101, for example, a monitoring image or a monitoring video obtained from a monitoring camera. In addition, the imaging unit 101 sends the captured video information to the control unit 200.

  Here, as illustrated in FIG. 3, the plurality of imaging units 101 capture video information about the monitoring area from different positions. FIG. 3 is a diagram for explaining an example of the arrangement of the imaging units in the first embodiment. Hereinafter, each of the plurality of imaging units 101 is referred to as an imaging unit “1”, an imaging unit “2”, and the like.

  In the following description, unless otherwise specified, each of the plurality of imaging units 101 does not capture video information about the entire monitoring area, but captures video information about each partial area of the monitoring area. . Further, each partial area will be described as having an area overlapping with each other partial area.

  For example, as illustrated in FIG. 3, the imaging units “1” to “9” are described as being installed for each column in a monitoring area where shelves are arranged in rows. In addition, it is assumed that the partial area in which each imaging unit 101 captures video information includes a column in which the image capturing unit is installed and an adjacent column, and the adjacent column is a blind spot due to a shelf. explain.

  For example, the partial area where the imaging unit “4” captures video information is equal to the partial area where the imaging unit “3” captures video information and the partial area where the imaging unit “5” captures video information. Duplicate parts. In other words, the imaging unit “4” captures video information about the column in which the imaging unit “4” is installed, the column in which the imaging unit “3” is installed, and the column in which the imaging unit “5” is installed. To do. Further, for example, it is assumed that the partial area of the imaging unit “4” and the partial areas of the imaging units “7” to “9” do not overlap.

  The display unit 102 is connected to the control unit 200 and displays video information output from the control unit 200. For example, the display unit 102 corresponds to a monitor.

  The control unit 200 has an internal memory for storing a program that defines various display processing procedures and the like, and executes various display processes. The control unit 200 includes a detection unit 201, an image acquisition unit 202, a display video selection unit 203, and an overlay unit 204.

  When the detection unit 201 is connected to the imaging unit 101, the image acquisition unit 202, and the display video selection unit 203 and receives video information from the imaging unit 101, the detection unit 201 extracts a predetermined object from each of the video information captured by the plurality of imaging units 101. To detect. For example, the detection unit 201 detects a predetermined object from the video information by using a background difference method, an optical flow, or other known methods. Note that the detection unit 201 may use a combination of a plurality of background subtraction methods, optical flows, and other known methods.

  Note that the background subtraction method prepares video information in a state where a predetermined object is not captured in advance, acquires a background differential image that is a difference from the current video information captured by the imaging unit 101, This is a method for detecting an object. The optical flow is a method for detecting a predetermined moving object by calculating a velocity vector of each point on the video information.

  In the first embodiment, a case where the detection unit 201 detects the person 10 as a predetermined object will be described as an example. The predetermined object is not limited to the person 10 but is arbitrarily set by a user who manages the display device 100 (for example, all “moving objects”).

  In addition, the detection unit 201 acquires position information of a predetermined object from the video information from the imaging unit 101, and acquires, for example, coordinate information in the monitoring area as the position information. The detection unit 201 sends the acquired position information and video information for detecting a predetermined object to the image acquisition unit 202, and displays identification information for identifying video information for detecting the predetermined object as a display video selection unit. 203.

  An example of position information acquisition processing by the detection unit 201 will be described with reference to FIG. FIG. 4 is a diagram for explaining position measurement by the detection unit in the first embodiment. In the following, coordinate information in the monitoring area where each of the imaging units “1” to “9” is installed, the monitoring direction in which each of the imaging units “1” to “9” is directed, and the camera installation angle “α” Is assumed to be known. As shown in (2) of FIG. 4, the camera installation angle “α” is an angle formed by the “perpendicular to the floor of the monitoring area” and the “optical axis of the camera”. The monitoring direction is the direction of the imaging unit when the camera installation angle “α” is “90 degrees”. In addition, it is assumed that the predetermined object is in contact with the floor surface of the monitoring area.

  Here, as shown in (1) of FIG. 4, the detection unit 201 calculates the distance “L” and the distance “S”, and combines them with the coordinate information in the monitoring area where the imaging unit is installed. The position information of the object is acquired. That is, the detection unit 201 advances the distance “L” on the monitoring direction axis as shown in (L) of FIG. 4 using the coordinate information of the imaging unit as a base point, and monitors as shown in (S) of FIG. The coordinates of the point advanced by “S” on the axis orthogonal to the direction axis are acquired as position information of the predetermined object.

  The distance “L” is a distance on the monitoring direction axis indicated by “A” in (1) of FIG. 4, and indicates a distance from a coordinate where the imaging unit 101 is installed to a predetermined object. Further, the distance “S” is a distance from the monitoring direction axis indicated by “A” in (1) of FIG. 4 and is an axis perpendicular to the monitoring direction axis indicated by “B” in (1) of FIG. "Indicates the distance above. Hereinafter, first, the point of acquiring the distance “L” using “β” and “γ” shown in (3) of FIG. 4 will be described, and then the point of acquiring the distance “S” will be described. Note that the distance “L” and the distance “S” are calculated using “β” and “γ” shown in (3) of FIG. 4 because the information received by the detection unit 201 is an image from the imaging unit 101. Because it is information only.

  The point of acquiring the distance “L” will be described using (2) and (3) of FIG. 4 (2) is a diagram seen from the side, and (3) in FIG. 4 is a diagram showing an example of video information seen from the imaging unit. Here, among the parameters shown in (2) of FIG. 4, the camera installation angle “α” is known, and the height “H” is known from the coordinate information in the monitoring area where the imaging unit is installed. Become. That is, if the angle “θ” formed by the “perpendicular to the floor of the monitoring area” and the “line connecting the contact point between the predetermined object and the floor of the monitoring area and the imaging unit 101” is known, the distance “L” ”= H tan θ, and the detection unit 201 can calculate the distance“ L ”.

  In addition, as shown in (3) of FIG. 4, the detection unit 201 uses the video information received from the imaging unit 101 to “contact the predetermined object with the floor surface of the monitoring area” from the camera optical axis. The number of pixels up to “β” is acquired. For example, the detection unit 201 acquires the number of pixels on the video information, and acquires the angle “β ′” after using the acquired number of pixels “β” and the resolution of the video information. Here, by making the correspondence with the angle per pixel known, the detection unit 201 calculates the angle “β ′” from the number of pixels “β”. Also, as shown in (1) of FIG. 4, the angle “θ” = angle “α” −angle “β ′”, and the detection unit 201 calculates the angle “θ” and acquires the distance “L”. .

  The point of calculating the distance “S” will be described mainly using (3) and (4) of FIG. In addition, (4) of FIG. 4 is the figure seen from the upper surface. Here, among the parameters shown in (4) of FIG. 4, the distance “L” is known. That is, if the angle “γ ′” formed by the camera optical axis and the “straight line connecting the predetermined object and the imaging unit 101” is known, the distance “S” = Ltanγ ′, and the detection unit 201 sets the distance “S”. It can be calculated.

  Then, as shown in (3) of FIG. 4, the detection unit 201 uses the video information received from the imaging unit 101 to calculate the angle “γ ′” in the same manner as the method of calculating the angle “β ′”. And the distance “S” is calculated.

  Note that the detection unit 201 performs the same processing for each piece of video information received from the plurality of imaging units 101. That is, for example, the detection unit 201 detects the person 10 for each of the video information “1” to “9” received from each of the imaging units “1” to “9”, and acquires position information from the video information from which the person 10 is detected. get. If it demonstrates using the example shown in FIG. 3, the detection part 201 will detect five persons 10 and will acquire five positional information.

  The image acquisition unit 202 is connected to the detection unit 201 and the overlay unit 204. The image acquisition unit 202 receives position information and video information from the detection unit 201 and acquires video information of a predetermined object detected by the detection unit 201. For example, the image acquisition unit 202 acquires, from the video information received from the detection unit 201, video information of a portion different from the background as video information of a predetermined object.

  Further, the image acquisition unit 202 acquires video information of an object for each predetermined object detected by the detection unit 201. If it demonstrates using the example shown in FIG. 3, since the person 10 was detected by the detection part 201, the image acquisition part 202 will acquire five image | video information of an object.

  Further, the image acquisition unit 202 associates the position information with the video information of the predetermined object, and sends it to the overlay unit 204, for example, sends five correspondences between the position information and the image information of the object.

  As described above, the image acquisition unit 202 and the detection unit 201 operate in cooperation to acquire position information of a predetermined object detected by the detection unit 201 and video information of the object.

  The display video selection unit 203 is connected to the detection unit 201 and the overlay unit 204, and sends the display video to the overlay unit 204. Here, the display video selection unit 203 receives, for example, an instruction from which image display unit 101 displays video information from the user, and sends the video information specified by the received instruction to the overlay unit 204. The display video selection unit 203 selects video information using the identification information received from the detection unit 201, and sends the selected video information to the overlay unit 204. The selection process using the identification information will be described later with reference to FIG.

  Here, the overlay unit 204 converts the position information acquired by the detection unit 201 into position information in the display video. For example, the overlay unit 204 performs a process reverse to the position information acquisition process performed by the detection unit 201, so that the number of pixels “β” and the number of pixels “γ” in the display video are obtained from the position information acquired by the detection unit 201. And convert to

  That is, the overlay unit 204 calculates “L” and “S” shown in FIG. 4 for the camera to be displayed from the coordinate information in the monitoring area where the imaging unit 101 is installed and the position information of the object. Then, the overlay unit 204 uses the calculated “L” and “S” to convert into “β” and “γ” shown in FIG. For example, when there are two people 10, two sets of “L” and “S” are calculated, and for each, the number of pixels “β” and the number of pixels “γ” for the camera to be displayed are calculated. Convert to In the example shown in (1) of FIG. 5, the overlay unit 204 converts the number of pixels “β1” and the number of pixels “γ1”, and the number of pixels “β2” and the number of pixels “γ2”.

  For example, as shown in FIG. 13, the distance “L” between the object obtained from the photographed camera 1 and the camera is obtained by the above method. Here, in the camera 2 to be displayed, the distance “P” from the camera 1 is known, and the distance corresponding to “L” of the camera 1 is represented by “PL”, and “β ″ in the figure. Can be calculated. As a result, the number of pixels “β” in the camera to be displayed is known, and this is used for overlay display. Also, “S” can be obtained by the same method, and the number of pixels “γ” can be calculated. In the example shown in FIG. 13, attention is paid only to the distance between the two cameras, but the same calculation can be performed even when the position where each camera is arranged is represented by planar map coordinates. FIG. 13 is a diagram for explaining pixel number calculation processing by the overlay unit in the first embodiment.

  Note that the overlay unit 204 performs a conversion process on each piece of position information included in the association received from the image acquisition unit 202. On the other hand, the overlay unit 204 does not always acquire position information in the display video as a result of the conversion process. This is because the partial area in which the imaging unit 101 captures video information does not necessarily overlap with the entire partial area in which the other imaging unit 101 captures video information.

  This will be specifically described with reference to the example shown in FIG. Note that video information from the imaging unit “4” is a display video. That is, it is set as a range in which video information about a column in which the imaging units “3” to “5” are installed is captured. Here, the position information of the person “D” and the person “E” detected from the video information of the imaging unit “7” and the imaging unit “8” is within the range where the imaging unit “4” captures the video information. In other words, the number of pixels “β” and the number of pixels “γ” obtained as a result of the conversion are not positions in the display image. Further, the position information of the person “A” and the person “C” detected from the video information of the imaging unit “3” and the imaging unit “5” is within the range in which the imaging unit “4” captures the video information. The number of pixels “β” and the number of pixels “γ” obtained as a result of the conversion are positions in the display image. Also, the person “B” illustrated in FIG. 3 is a person detected from the video information from the imaging unit “4”. As a result, the overlay unit 204 receives five associations from the image acquisition unit 202 and obtains position information about the person “A” and the person “C” in the display video as a result of the conversion.

  As shown in (2) of FIG. 5, the overlay unit 204 uses the position information obtained as a result of the conversion to display the video information of the object acquired by the image acquisition unit 202 together with the display video. 102. In order to display on the display unit 102, the overlay unit 204 sends video information that combines the video information of the object acquired by the image acquisition unit 202 to the display unit 102, and the display unit 102 displays the overlay unit 204. Indicates that the video information received from is displayed.

  In the example illustrated in (2) of FIG. 5, the overlay unit 204 includes the number of pixels “β1” and the number of pixels “γ1”, which are position information obtained as a result of the conversion, and the number of pixels “β2” and the number of pixels “γ2”. The video information of the object associated with each piece of the position information is displayed at the position “”.

  The overlay unit 204 also displays the display image and the object image information together by providing transparency to the object image information. For example, the overlay unit 204 thins the video information of the predetermined object and displays it on the display video so that the video information of the predetermined object and the display video at the same position can be discriminated together. 102.

  As shown in FIG. 6, the overlay unit 204 may display video information serving as a mark indicating the video information of the object together with the video information of the object displayed together with the display video. FIG. 6 is a diagram for explaining an example of the mark display by the overlay unit in the first embodiment. In the example illustrated in FIG. 6, the overlay unit 204 displays the mark 30 together with the video information of the object. For example, when the number of people 10 is large, it is possible to efficiently monitor the people 10 by using a mark.

  As shown in FIG. 7, the overlay unit 204 may display video information indicating the moving direction of the object together with the video information of the object to be displayed together with the display video. FIG. 7 is a diagram for explaining an example of the moving direction display by the overlay unit in the first embodiment. For example, the overlay unit 204 displays a moving direction display arrow 40 that is an arrow from an object before a predetermined time elapses to an object after a predetermined time elapses as video information indicating the moving direction.

  Further, the overlay unit 204 may display video information indicating the line of sight of the person 10 and the orientation of the face together with the video information of the person 10 displayed together with the display video. For example, the overlay unit 204 recognizes a face from the video information of the object acquired by the image acquisition unit 202, and displays the direction in which the face is facing as an arrow. The face recognition process by the overlay unit 204 may use a known method.

[Processing by display device]
Next, an example of the overall processing flow of the display device 100 according to the first embodiment will be described with reference to FIG. FIG. 8 is a flowchart for explaining an example of the overall processing flow of the display apparatus according to the first embodiment.

  As shown in FIG. 8, in the display device 100, when the detection unit 201 detects a predetermined object (Yes in step S <b> 101), that is, when a predetermined object is detected from each of video information captured by a plurality of imaging units 101. The position information of the predetermined object is acquired (step S102). For example, the detection unit 201 acquires coordinate information in the monitoring area as the position information.

  Then, the image acquisition unit 202 acquires video information of the object (step S103). That is, the image acquisition unit 202 acquires video information of a predetermined object detected by the detection unit 201. Then, the overlay unit 204 converts the position information acquired by the detection unit 201 into position information in the display video (step S104). For example, the overlay unit 204 performs processing reverse to the position information acquisition processing performed by the detection unit 201, thereby converting into position information in the display video.

  The overlay unit 204 displays the video information of the object acquired by the image acquisition unit 202 on the display unit 102 together with the display video using the position information obtained as a result of the conversion (step S105). For example, the overlay unit 204 thins the video information of the predetermined object and displays it on the display video so that the video information of the predetermined object and the display video at the same position can be discriminated together. 102.

[Example of display image determination processing]
Next, a display video selection process using identification information in the first embodiment will be described with reference to FIG. FIG. 9 is a flowchart for explaining an example of a flow of display video selection processing using identification information in the first embodiment.

  That is, as shown in FIG. 9, the display video selection unit 203 uses the information regarding the range in which each of the imaging units 101 captures video information and the identification information for identifying the video information in which the person is detected, to display the display video. Select. Note that the information regarding the range in which each of the imaging units 101 captures the video information includes information regarding the range where each of the imaging units 101 overlaps with the range in which the other imaging unit 101 captures the video information. For example, to explain using an example in FIG. 3, each of the image capturing units 101 corresponds to information that captures video information of a column in which the image capturing unit is installed and columns adjacent to both.

  Here, as shown in FIG. 9, when the detection unit 201 detects a predetermined object (Yes in step S201), the display video selection unit 203 identifies video information in which the predetermined object is detected, for example. When the identification information to be received is received from the detection unit 201, display video selection processing is performed.

  In other words, the display video selection unit 203 first determines whether or not the person 10 is detected continuously in only two rows using the received identification information (step S202). For example, the display video selection unit 203 determines whether identification information for identifying the imaging unit 101 that is continuous in only two columns is received as the identification information. If it is determined that the person 10 is detected continuously only in two columns (Yes in step S202), the display image selection unit 203 selects one of the columns in which only two columns are detected in which the person 10 is detected. Is selected (step S203). This is because, regardless of which of the two consecutive columns is selected, the person 10 detected in the other column can be displayed by overlaying on the video information for the selected column.

  For example, the display video selection unit 203 receives identification information from the detection unit 201 as needed, and identifies the continuous mode of the column in which the person 10 is detected each time using the identification information. Specifically, referring to the example shown in FIG. 3, when the imaging units 101 are installed in the order of numbers, they are identified using the continuity of the numbers. For example, the person 10 is not detected in the imaging unit “6”, the person 10 is detected in the imaging unit “7” and the imaging unit “8”, and the person 10 is not detected in the imaging unit “9”. The unit 203 identifies that a person has been detected in two consecutive rows, and selects video information about either the imaging unit “7” or the imaging unit “8”.

  On the other hand, if the display video selection unit 203 determines that the person 10 is not detected continuously for only two columns (No at Step S202), it determines whether the person 10 is detected for only three columns (Step S204). Here, if it is determined that the person 10 is detected continuously in only three columns (Yes in step S204), the display image selecting unit 203 displays the image information about the middle column of the three consecutive columns in which the person 10 is detected. Is selected (step S205). That is, for example, the person 10 is not detected by the imaging unit “2”, the person 10 is detected in three consecutive rows by the imaging unit “3” to the imaging unit “5”, and the person 10 is detected by the imaging unit “6”. If not detected, the display video selection unit 203 selects video information for the imaging unit “4”. This is because by selecting the middle of three consecutive columns, the person 10 detected in the columns at both ends can be displayed by overlaying on the video information for the selected middle column.

  On the other hand, if the display video selection unit 203 determines that the person 10 is not detected continuously for only three rows (No at Step S204), it determines that the person 10 is detected for four or more rows continuously (Step S206). Then, the display video selection unit 203 displays the video information about the second column from the end, and the video information about each of the columns skipped from the second column from the end, among the four or more consecutive columns. Select (step S207). For example, when the person 10 is detected by the imaging unit “1” to the imaging unit “5”, the display video selecting unit 203 displays the video information captured by the imaging unit “2” and the imaging unit “5”. Each of the video information picked up by the image pickup unit “1” and the image pickup unit “4” is selected.

  Note that the display video selection unit 203 selects the column skipped by one when there is no column skipped by two from the second column from the end. As a result, it is possible to display reliably even after the column skipped from the end by one column.

  As described above, by selecting the display video using the identification information, it is possible to efficiently display all the video information of the detected person 10, for example, reducing the number of video information to be displayed. Is possible.

[Effect of Example 1]
As described above, according to the first embodiment, the display device 100 detects a predetermined object from each of the video information captured by the plurality of imaging units 101, and detects the position information of the detected predetermined object and the object information. Get video information. Then, the display device 100 converts the acquired position information into position information in the display image, and displays the acquired object image information together with the display image using the position information obtained as a result of the conversion. . As a result, it is possible to efficiently monitor the monitoring target.

  For example, even if there is a blind spot in the display image, the person 10 in the blind spot can be displayed together with the display image, and the monitoring target can be efficiently monitored. That is, for example, when the person 10 is detected at a position corresponding to the blind spot in the display video by video information different from the display video, the detected image of the person 10 is combined with the position of the blind spot in the display video. It can be displayed, and the monitoring efficiency can be improved.

  As a result, according to the first embodiment, the display device 100 can reduce the switching frequency of the video information by the user. That is, information about a predetermined object obtained from other video information can be displayed together with the display video, and the frequency of switching to video information in which the predetermined object is imaged can be reduced.

  As a result, according to the first embodiment, the display device 100 can reduce the number of monitors that display video information. That is, information about a predetermined object obtained from other video information can be displayed together with the display video, and the video about the predetermined object can be displayed without displaying all the video information obtained by imaging the predetermined object. Information can be displayed. As a result, the number of video information to be displayed is reduced, and the number of monitors that display video information can be reduced.

  Also, according to the first embodiment, in the display device 100, a plurality of imaging units 101 capture the same region from different positions. Further, since the display device 100 acquires the coordinate information in the region as the position information, it is possible to accurately convert the position information into the position information in the display video.

  In addition, according to the first embodiment, the display device 100 displays video information serving as a mark indicating the video information of the object, so that the user can easily determine the position of the video information of the object displayed together with the display video. It is possible to grasp.

  Further, according to the first embodiment, the display device 100 displays video information indicating the moving direction of the object, so that the user can easily grasp the moving direction of the video information of the object displayed together with the display video. Is possible.

  Further, according to the first embodiment, the display device 100 displays video information indicating the direction of the line of sight and the face of the person 10, so that the line of sight and the direction of the face of the person 10 displayed together with the display video are used. Can be easily grasped by a person.

  Although the embodiments of the present invention have been described so far, the present invention may be implemented in other embodiments besides the above-described first embodiment. Therefore, other embodiments will be described below.

[Recording]
For example, as illustrated in FIG. 10, the display device 100 may further include a storage unit 300 that stores video information captured by each of the imaging units 101. FIG. 10 is a block diagram for explaining an example of the configuration of the display device according to the second embodiment.

  In the example illustrated in FIG. 10, the storage unit 300 is connected to the detection unit 201. In addition, in the display device 100, the detection unit 201 (also referred to as “storage unit”) stores each piece of video information captured by the plurality of imaging units 101 in the storage unit 300 together with the acquired position information. Also good. As a result, when the video information is stored (recorded) in the storage unit 300, the position information can be recorded together.

[Imaging unit installation position]
For example, in the first embodiment, the plurality of imaging units 101 capture video information about each of the partial areas of the monitoring area. For example, imaging is performed in the monitoring area where shelves are arranged in rows. Each of the parts “1” to “9” has been described as being installed in each row. However, the present invention is not limited to this.

  For example, as illustrated in FIG. 11, the image capturing unit 101 may be further installed so that a small number of image capturing units 101 capture image information of the entire monitoring area. For example, as illustrated in “A to D” in FIG. 11, the imaging units 101 may be installed at the four corners of the monitoring area, or the imaging unit 101 may be installed as an omnidirectional camera on the ceiling. FIG. 11 is a diagram for explaining an example of the arrangement of the imaging units in the second embodiment. Also, the display video selection unit 203 may select video information from the imaging unit 101 installed at the four corners of the monitoring area or video information from the imaging unit 101 installed as an omnidirectional camera installed on the ceiling as a display video. Good.

  In addition, the display device 100 acquires the position information of the predetermined object and the image information of the object using the image information from the imaging unit 101 installed for each column, and is installed as an omnidirectional camera or at the four corners. You may display together with the video information about the imaging part 101. FIG.

  Further, for example, the display device 100 divides the screen of the display unit 102 into five, displays video information from the imaging unit 101 installed at the four corners of the monitoring area in four areas, and displays the remaining one The video information in which the person 10 is detected is displayed in the area. That is, all the monitoring areas are displayed in the four areas for displaying the video information about the imaging unit 101 installed in the corner, and all the people 10 in the monitoring area are displayed by overlaying as necessary. In addition, the display unit 102 may display the video information in which the person 10 is detected in the remaining one region so that the user can monitor the detected person 10 in detail. The display device 100 may be divided into an arbitrary number when dividing the screen, and a plurality of monitors may be used in combination.

[Image information of object]
For example, in the first embodiment, a case has been described in which a part of video information is used as video information of an object to be displayed together with a display video. However, the present invention is not limited to this, and CG (Computer) (Graphics). For example, in the display device 100, when the image acquisition unit 202 acquires the video information of the object, the image acquisition unit 202 fits the size of the CG image to the same size as the video information of the object, and the CG fitted as the video information of the object. It may be sent to the overlay unit 204.

[Monitoring area]
For example, in the first embodiment, the case where the imaging unit 101 captures video information for one monitoring area has been described, but the present invention is not limited to this. For example, the plurality of imaging units 101 may capture video information from different positions for each of the plurality of monitoring areas.

[Combination of Examples]
Also, for example, in the first embodiment, a method of displaying the video of the person 10 detected in the other monitoring video together with the display video at a position in the display video corresponding to the position where the person 10 is detected. In addition to the above, the case where various methods are implemented together has been described. Specifically, (Method 1) Method using position information as coordinate information, (Method 2) Method of displaying landmarks, (Method 3) Method of displaying moving direction, and (Method 4) Displaying face orientation The Example which combines and implements the method to perform was demonstrated. However, the present invention is not limited to this, and only a part of (Method 1) to (Method 4) may be performed together, or none of them may be performed together.

[System configuration]
Further, for example, all or a part of the processes described as being automatically performed among the processes described in the present embodiment can be manually performed. For example, a predetermined object is manually detected. May be. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters (for example, FIG. 1 to FIG. 12) shown in the above documents and drawings are arbitrary unless otherwise specified. Can be changed.

  Each component of each illustrated device is functionally conceptual and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. For example, referring to FIG. 2, the display device 100 and the display unit 102 may be separate devices. In addition, among the units in the control unit 200, the detection unit 201 and the image acquisition unit 202 that acquire position information about a predetermined object and video information of the object may be separate devices.

[Computer]
The various processes described in the above embodiments can be realized by executing a program prepared in advance on a computer such as a personal computer or a workstation. In the following, an example of a computer that executes a display program having the same function as that of the above-described embodiment will be described with reference to FIG. FIG. 12 is a diagram for explaining the program of the display device according to the first embodiment.

  As shown in FIG. 12, a computer 3000 includes an operation unit 3001, a microphone 3002, a speaker 3003, an imaging unit 3004, a display 3005, a communication unit 3006, a CPU (Central Processing Unit) 3010, a ROM (Read Only Memory) 3011, a RAM ( Random Access Memory) 3013 is connected by a bus 3009 or the like.

  The ROM 3011 stores a control program that performs the same functions as those of the detection unit 201, the image acquisition unit 202, the display video selection unit 203, and the overlay unit 204 described in the first embodiment, that is, as illustrated in FIG. In addition, a detection program 3011a, an image acquisition program 3011b, a display video selection program 3011c, and an overlay program 3011d are stored in advance. Note that these programs 3011a to 3011d may be integrated or separated as appropriate, similarly to each component of the display device 100 shown in FIG.

  Then, the CPU 3010 reads out these programs 3011a to 3011d from the ROM 3011 and executes them, and as shown in FIG. It functions as a process 3010c and an overlay process 3010d. Each process 3010a to 3010d corresponds to the detection unit 201, the image acquisition unit 202, the display video selection unit 203, and the overlay unit 204 illustrated in FIG.

  The CPU 3010 executes a display program using the video information data 3013a and the position information data 3013b stored in the RAM 3013.

[Others]
The display device described in the present embodiment can be realized by executing a program prepared in advance on a computer such as a personal computer or a workstation. This program can be distributed via a network such as the Internet. The program can also be executed by being recorded on a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, an MO, and a DVD and being read from the recording medium by the computer.

  Regarding the embodiment including the above-described Examples 1 and 2, the following additional notes are further disclosed.

(Additional remark 1) The detection part which detects a predetermined | prescribed object from each image | video information imaged by several imaging part,
An acquisition unit that acquires position information of the predetermined object detected by the detection unit and video information of the object;
A conversion unit that converts the position information acquired by the acquisition unit into position information in a display video that is video information displayed by an internal display unit of each of the video information;
A display control unit that displays the video information of the object acquired by the acquisition unit together with the display video using the position information obtained as a result of the conversion by the conversion unit. apparatus.

(Supplementary Note 2) The plurality of imaging units are configured to image the same region from different positions,
The display device according to appendix 1, wherein the acquisition unit acquires coordinate information in the region as the position information.

(Additional remark 3) The said display control part displays the video information used as the mark which shows the video information of the said object with the video information of the said object displayed together with the said display image, It adds to 1 or 2 characterized by the above-mentioned. The display device described.

(Additional remark 4) The said display control part displays the video information which shows the moving direction of the said object with the video information of the said object displayed with the said display image, Any one of Additional remarks 1-3 characterized by the above-mentioned. The display device described in one.

(Supplementary Note 5) The predetermined object is a person,
Any one of Supplementary notes 1 to 4, wherein the display control unit displays video information indicating the line of sight and / or face of the person together with the video information of the person displayed together with the display video. The display device according to one.

(Supplementary note 6) The supplementary notes 1 to 5, further comprising a storage unit that stores the video information captured by the plurality of imaging units in the storage unit together with the positional information acquired by the acquisition unit. The display device according to any one of the above.

(Additional remark 7) The detection process which detects a predetermined | prescribed object from each image | video information imaged by several imaging part,
An acquisition step of acquiring position information of the predetermined object detected by the detection step and video information of the object;
A conversion step of converting the position information acquired by the acquisition step into position information in a display video that is video information displayed by an internal display unit of each of the video information;
A display control step of displaying the video information of the object acquired by the acquisition step together with the display video using the position information obtained as a result of the conversion by the conversion step. Method.

(Appendix 8) The computer
A detection step of detecting a predetermined object from each of video information imaged by a plurality of imaging units;
An acquisition step of acquiring position information of the predetermined object detected by the detection step and video information of the object;
A conversion step of converting the position information acquired in the acquisition step into position information in a display image that is image information displayed by an inner display unit of each of the image information;
A display control step of displaying the video information of the object acquired in the acquisition step together with the display video using the position information obtained as a result of the conversion in the conversion step. program.

10 people 20 shelves 100 display device 101 imaging unit 102 display unit 200 control unit 201 detection unit 202 image acquisition unit 203 display video selection unit 204 overlay unit 300 storage unit

Claims (7)

A detection unit for detecting a predetermined object from each of video information captured by a plurality of imaging units;
An acquisition unit that acquires position information of the predetermined object detected by the detection unit and video information of the object;
A conversion unit that converts the position information acquired by the acquisition unit into position information in a display video that is video information displayed by an internal display unit of each of the video information;
A display control unit that displays the video information of the object acquired by the acquisition unit together with the display video using the position information obtained as a result of the conversion by the conversion unit. apparatus. The plurality of imaging units capture the same region from different positions,
The display device according to claim 1, wherein the acquisition unit acquires coordinate information in the region as the position information.   3. The display according to claim 1, wherein the display control unit displays video information serving as a mark indicating video information of the object together with video information of the object to be displayed together with the display video. apparatus.   The said display control part displays the video information which shows the moving direction of the said object with the video information of the said object displayed together with the said display video, The any one of Claims 1-3 characterized by the above-mentioned. Display device. The predetermined object is a person,
The display control unit displays video information indicating the line of sight and / or face of the person together with the video information of the person to be displayed together with the display video. A display device according to any one of the above. A detection step of detecting a predetermined object from each of video information imaged by a plurality of imaging units;
An acquisition step of acquiring position information of the predetermined object detected by the detection step and video information of the object;
A conversion step of converting the position information acquired by the acquisition step into position information in a display video that is video information displayed by an internal display unit of each of the video information;
A display control step of displaying the video information of the object acquired by the acquisition step together with the display video using the position information obtained as a result of the conversion by the conversion step. Method. Computer
A detection step of detecting a predetermined object from each of video information imaged by a plurality of imaging units;
An acquisition step of acquiring position information of the predetermined object detected by the detection step and video information of the object;
A conversion step of converting the position information acquired in the acquisition step into position information in a display image that is image information displayed by an inner display unit of each of the image information;
A display control step of displaying the video information of the object acquired by the acquisition step together with the display video, using the position information obtained as a result of the conversion by the conversion step. program.

Images