JP2010123069A - Sensing data retrieval device and retrieval image creation method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a sensing data retrieval device quickly and properly selecting a desired reproduction point by use of a flow line. <P>SOLUTION: The retrieval device 110 has, in addition to a flow line creation part 111 and a reproduction time setting part 115 controlling reproduction of sensing data corresponding to a flow line section and a flow line part designated by a user; an auxiliary figure creation part 112 creating an auxiliary figure related to the flow line not appearing in the flow line based on a position measurement result; and a flow line retrieval image creation part 113 creating a retrieval image including the flow line and the auxiliary figure, and displaying the retrieval image on a display part 160. The user can select the reproduction point in reference to the auxiliary figure related to the flow line as well as the flow line, so that the user can quickly and properly select the desired reproduction point by use of the flow line. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a sensing data retrieval apparatus and a retrieval image creation method, and more particularly to a technique for retrieving sensing data such as a moving image while viewing a flow line.

  Conventionally, for example, Patent Literature 1 discloses a flow line that measures a position of a monitoring target person using a wireless signal from a wireless tag attached to the monitoring target person and indicates a movement history of the monitoring target person based on the measurement history. And a technique for displaying the flow line is disclosed. Japanese Patent Application Laid-Open No. 2005-228561 discloses a technique for reproducing a suspicious person's video when a person to be monitored is photographed by a monitoring camera and it is determined that the person to be monitored is a suspicious person.

  In Patent Document 2, for example, by mounting a GPS device and a video camera on a vehicle, a travel locus (traffic line) of the vehicle and an image at the time of travel are recorded, and on the travel locus specified by the user. A video search device that searches for and plays back video corresponding to a section is disclosed.

In Patent Document 3, a movement trajectory (flow line) and a direction indicating element are displayed on a display screen, and the coordinates at the designated coordinates are designated by designating coordinates on the movement trajectory and directions at the coordinates. An image display device that reproduces and displays an image in a specified direction is disclosed.
Japanese Patent Laid-Open No. 2004-5511 JP 2001-290820 A JP 2001-215941 A

  By the way, although the technique which reproduces a suspicious person's animation is indicated by patent documents 1, a suspicious person's animation is not necessarily the animation which a user wants to watch. Therefore, it is inconvenient in that the user cannot freely select the moving image that he wants to see.

  In addition, the technique disclosed in Patent Document 2 and Patent Document 3 for retrieving a reproduction image using a flow line is convenient because the user can visually designate a reproduction point while viewing the flow line.

  However, when the user designates a playback point on the flow line, the user's playback point selection criterion is only the coordinates of the moving object (monitoring target). When the user selects a desired playback image, It is considered insufficient. In other words, if the playback image at the playback point selected on the flow line is not the playback image that he / she wanted to see, the user needs to newly select a playback point on the flow line, so that the desired playback can be quickly performed. It is difficult to obtain an image. In other words, there is no problem if the playback image you want to view can be determined from the flow line alone, but if you cannot determine the playback image you want to view from the flow line alone, you can reselect the playback point many times while actually checking the playback image. Is required.

  For example, when a playback point is selected from its own flow line as in a navigation system, a desired playback point can be selected relatively quickly from the flow line. If there is a monitoring target, and there are a plurality of flow lines or a flow line different from the own flow line, it is difficult to quickly select a desired reproduction point only from the flow line.

  The present invention has been made in consideration of such points, and when selecting a playback point using a flow line, a sensing data search device capable of selecting a desired playback point more quickly and accurately than in the past. And a search image creation method.

  One aspect of the sensing data search device of the present invention is a flow line creation means for creating a flow line that is a movement trajectory of a monitoring target based on a positioning result for the monitoring target, and a flow line location designated by the user Alternatively, reproduction control means for controlling the reproduction section by the reproduction means so that sensing data corresponding to the flow line section is reproduced, and auxiliary figure creation means for creating an auxiliary figure related to the flow line based on the positioning result. And a search image creating means for creating a search image including the flow line and the auxiliary graphic and displaying the search image on a display means.

  One aspect of the search image creation method of the present embodiment includes a step of creating a flow line that is a movement trajectory of a monitoring target based on a positioning result, and an auxiliary figure related to the flow line based on the positioning result. Creating a search image including the flow line and the auxiliary figure.

  According to the present invention, the user can select a reproduction point while referring to not only the flow line but also the auxiliary graphic related to the flow line, so that a desired reproduction point can be selected quickly and accurately.

  One feature of the present invention is that a search image in which an auxiliary figure related to a flow line is added and displayed in addition to a flow line that is a movement locus of a monitoring target. As a result, a user who searches for a reproduction image can select a reproduction point with reference to not only a flow line but also an auxiliary figure related to the flow line, so that a desired reproduction point can be selected quickly and accurately.

  Further, the inventors diligently studied what kind of figure should be created as the auxiliary figure so that the user can select a desired reproduction point quickly and accurately. At that time, the inventors focused on the fact that there is a characteristic part on the flow line that does not appear on the flow line (or that it is difficult for the user to determine from the flow line alone). And the inventors discovered the preferable arrangement | positioning and content of an auxiliary figure by earnestly examining what such a characteristic location is. One feature of the present invention is the arrangement and content of auxiliary graphics to be displayed in addition to the flow lines.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(Embodiment 1)
[Constitution]
FIG. 1 shows a configuration of a sensing data search system according to Embodiment 1 of the present invention.

  The sensing data search system 100 is roughly divided into a search device 110, a sensing data DB (DB means a database) 120, a positioning result DB 130, an accident information DB 140, a sensing data reproducing unit 150, and a display unit 160. And having.

  Sensing data for reproduction is stored in the sensing data DB 120. The sensing data for reproduction is, for example, a moving image acquired by a surveillance camera. Also, the sensing data for reproduction may be biological data (heart rate, body temperature, etc.), environmental data such as temperature / humidity.

  The positioning result DB 130 stores the positioning result data to be monitored. The positioning result data is acquired based on, for example, a wireless signal from a wireless tag attached to the monitoring target, and is, for example, a list of monitoring target user ID, time, and position. That is, the position of each time is stored for each monitoring target. The positioning result data is data that is synchronized with the sensing data. Note that the positioning result data is not limited to data acquired based on a wireless signal from the wireless tag. The positioning result data may be, for example, data acquired by a GPS device of a navigation system or data obtained by processing a captured image and acquiring position information of a monitoring target.

  The accident information DB 140 includes accident information (for example, time, accident content, user ID, user ID, sensor acquired from the sensing data DB 120 and sensor acquired from the positioning result DB 130). Position) is stored.

  The search device 110 creates a search image for the user to select a playback point by the sensing data playback unit 150 and causes the display unit 160 to display the search image. The search device 110 controls the reproduction operation of the sensing data reproduction unit 150 based on the reproduction point selected and designated by the user.

  The configuration of the search device 110 will be described in detail.

  The search device 110 inputs the positioning data of the positioning result DB to the flow line search image creation unit 113. The flow line creation unit 111 creates a flow line indicating the movement trajectory of the monitoring target based on the positioning data. When there are a plurality of monitoring targets, the flow line creation unit 111 creates a plurality of flow lines. The flow line creation unit 111 sends the created flow line data S1 to the flow line search image creation unit 113, the selection status creation unit 114, and the reproduction time setting unit 115. The flow line data S1 is data composed of vector data and time metadata. The vector data is used to display a flow line. The time metadata is used by the user to start / start sensing data. Used to reverse the coordinates from the mouse position when selecting the end position.

  The auxiliary figure creating unit 112 creates an auxiliary figure related to the flow line. In the case of the present embodiment, the auxiliary figure creating unit 112 creates auxiliary figure information S2 using the positioning data in the positioning result DB 130 and the accident information in the accident information DB 140, and uses this as the flow line search image creating unit 113. And sent to the selection status creation unit 114. The auxiliary graphic is a graphic that indicates a characteristic part that does not appear on the flow line (or that it is difficult for the user to determine from the flow line alone).

  Further, in the present invention, various ideas are added to the auxiliary figure, and details thereof will be described in detail in the second embodiment and later.

  The flow line search image creation unit 113 creates the search image information S5 by combining the flow line data S1, the auxiliary graphic information S2, and the input situation information S4 created by the selection situation creation unit 114. This is sent to the display unit 160.

  The user input reception unit 116 receives user input information input by the user using a keyboard, a mouse, or the like, creates user selection information S3 indicating the current coordinates of the mouse and click status from the user input information, and selects this. It is sent to the situation creation unit 114 and the reproduction time setting unit 115.

  The selection status creation unit 114 creates input status information S4 using the user selection information S3, the flow line data S1, and the auxiliary graphic information S2, and sends this to the flow line search image creation unit 113. The input status information S4 includes information for displaying a mouse cursor at a position selected by the user. The input status information S4 includes information for controlling the flow line selected by the user, the color of the auxiliary graphic, and the like. Further, the input status information S4 may include information indicating whether or not the auxiliary graphic is displayed, in which position the auxiliary graphic is displayed.

  The reproduction time setting unit 115 obtains a reproduction time based on the user selection information S3 and the flow line data S1, and sends a reproduction control signal S6 including the reproduction time to the sensing data reproduction unit 150. Specifically, the reproduction time setting unit 115 uses the coordinates on the flow line specified by the user included in the user selection information S3, the coordinate data of the flow line included in the flow line data S1, and the time metadata. Thus, time data corresponding to the designated flow line coordinates is obtained as the reproduction time.

  The sensing data reproduction unit 150 reproduces the sensing data of the reproduction section corresponding to the reproduction time, and sends the reproduction image information S7 to the display unit 160.

  The display unit 160 displays the search image created by the flow line search image creation unit 113 and the playback image played back by the sensing data playback unit 150. Note that the display unit 160 may display the search image and the reproduction image on the same display, or may display them on different displays.

[Display image]
FIG. 2 shows an example of a display image displayed on the display unit 160 of the sensing data search system 100. 2A is an example of a search image created by the flow line search image creation unit 113, and FIGS. 2B and 2C are examples of a reproduction image reproduced by the sensing data reproduction unit 150 (in this example, by a camera). Captured moving image).

  As is clear from FIG. 2A, the search image displays a flow line L1 that is the movement locus of the monitoring target, auxiliary figures F1 and F2 related to the flow line, and a mouse cursor. Incidentally, these are displayed on the map which shows the plane arrangement | positioning of the structure of the vicinity of the flow line L1. In the configuration of FIG. 1, the map showing the planar arrangement is not mentioned, but a database storing map data is provided, and map data in the vicinity thereof is displayed together with the flow line based on the coordinates of the flow line L1. This can be easily realized.

  As described above, the auxiliary figures F1 and F2 are figures that show a characteristic part that does not appear on the flow line L1 (or that it is difficult for the user to determine from the flow line L1 alone). A staying location indicating that the event has occurred, a location where an accident has occurred, and the like are conceivable. Details of the auxiliary figure will be described later.

  When the user moves the mouse cursor over the auxiliary figure F1 indicating the characteristic location and clicks the mouse at that position, as shown in FIG. 2B, sensing data corresponding to the flow line at the position of the auxiliary figure F1 (FIG. 2A, FIG. In the case of 2B, moving image data) is reproduced and displayed. Further, when the user drags the mouse cursor in a predetermined section on the flow line, as shown in FIG. 2C, the moving image to be monitored in the dragged section is played and displayed.

  In addition, when one point on the flow line is clicked, the corresponding sensing data is reproduced and displayed from the location corresponding to the click, or when two points on the flow line are clicked, the corresponding sensing data is the first Clicking the auxiliary graphic F1 that starts playback from the location corresponding to the click and ends playback at the location corresponding to the next click, or clicks the auxiliary figure F1 indicating the feature location, only the location corresponding to the feature location F1 is played back There is a specification method that is displayed.

  As described above, by using the sensing data search system 100 according to the present embodiment, the user can search and reproduce a desired moving image quickly and accurately while looking at the flow line and the auxiliary figure.

[Operation]
Next, the operation of the sensing data search system 100 will be described with reference to FIGS. Note that the arrows in the sequence diagrams of FIGS. 3 to 5 indicate the data movement direction. Actually, a data acquisition request or the like may be sent in the direction opposite to the arrow before the data is moved.

  FIG. 3 shows a processing sequence of search image creation processing. FIG. 3 shows a processing sequence in a case where a flow line and an auxiliary figure are displayed in a state where there is no input from the user for easy understanding.

  First, when the positioning result is input from the positioning result DB 130, the flow line creation unit 111 creates the flow line data S1 from the positioning result in step ST11. As described above, the flow line data S1 is composed of, for example, vector data and time metadata attached to each coordinate on the vector.

  Next, when the auxiliary figure creating unit 112 inputs the positioning result from the positioning result DB 130 and the accident information from the accident information DB 140, the auxiliary figure creating unit 112 creates auxiliary figure information S2 based on these in step ST12. Specifically, the auxiliary figure creation unit 112 does not show the stay of the monitoring target, the round trip, the unnatural movement, etc. on the flow line based on the positioning result (or the user does not see the flow line alone). It is detected that it is a characteristic part (which is difficult to judge), and an auxiliary figure indicating this characteristic part is created. In the case of FIG. 2A, the auxiliary figure is a black circle indicating a characteristic location. In the case of the present embodiment, the auxiliary figure creating unit 112 creates a figure indicating the place where the accident occurred as an auxiliary figure based on the accident information stored in the accident information DB 140.

  Next, in step ST13, the selection status creating unit 114 creates input status information S4 indicating the input status, and waits for an input from the user. Here, the input status information S4 indicates, for example, the status of the mouse cursor. In FIG. 3, the input status information for displaying the mouse cursor stopped at a specific position is the initial state where there is no input from the user. S4 is created.

  Next, in step ST14, the flow line search image creation unit 113 creates the search image information S5 by combining the flow line data S1, the search image information S2, and the input status information S4. Next, the display part 160 displays the search image based on search image information S5 by step ST15.

  FIG. 4 shows a processing sequence of search image creation processing when the user moves the mouse.

  First, user input information (the user input information here is information before selecting a playback start point, for example, information indicating that the mouse cursor is moved) is input to the user input receiving unit 116, and user input is performed. User selection information S3 corresponding to the user input information is output from the receiving unit 116. Then, in step ST21, the selection status creating unit 114 creates input status information S4 that matches the user selection, and outputs this.

  Next, when the input state information S4, the auxiliary graphic information S2, and the flow line data S1 are input, the flow line search image creating unit 113 updates the image information according to the input state information S4 in step ST22. The update of the image information means reflection of the input situation information S4 on the search image and change of auxiliary figures and flow lines according to the input situation information S4. Reflecting the input status information S4 means moving the mouse cursor, deforming an icon, displaying an auxiliary figure, and the like on the search image. The change of the auxiliary graphic and the flow line means that the flow line is emphasized or the color of the auxiliary graphic is changed according to the position of the mouse cursor.

  Next, the display unit 160 displays the updated search image in step ST23.

  The process of FIG. 4 is repeated (looped) whenever there is a user input.

  FIG. 5 shows a processing sequence of search image creation processing when the user performs a selection operation of the playback start / end point.

  First, user input information (here, the user input information is information indicating that a playback start point has been selected, for example, information indicating that a mouse has been clicked) is input to the user input receiving unit 116, and the user input receiving unit 116 receives user input information. User selection information S3 corresponding to the input information is output. Since the user has selected the playback start point this time, the user selection information S3 is also sent to the playback time setting unit 115.

  In step ST31, the selection status creation unit 114 creates input status information S4 that matches the user input, and sends this to the flow line search image creation unit 113.

  In step ST32, the reproduction time setting unit 115 obtains time metadata of the flow line data S1 located (corresponding) to the coordinates of the mouse cursor included in the user selection information S3. If time metadata is not attached to the flow line data S1, the time corresponding to the coordinates of the clicked mouse cursor may be obtained with reference to the positioning result in the positioning DB.

  Next, the reproduction time setting unit 115 sets the obtained time as the reproduction start / end time in step ST33, and creates the reproduction control signal S6 when both the start time and the end time are set in subsequent step ST34. . The reproduction control signal S6 is sent to the sensing data reproduction unit 150. The sensing data reproduction unit 150 acquires data from the sensing data DB 120 according to the reproduction control signal S6, obtains a reproduction image, and sends the reproduction image data S7 to the display unit 160.

  In step ST35, the flow line search image creating unit 113 emphasizes the selected flow line position or changes the color of the selected auxiliary figure in accordance with the input selection information S4 on the search image. The search image information is updated by performing a process such as the above.

  Next, in step ST36, the display unit 160 displays the reproduction image and the updated search image.

  Note that the sensing data playback unit 150 may output playback data of the playback image in a format that can be used by another application.

[effect]
As described above, according to the present embodiment, the flow line creation unit 111 that creates the flow line that is the movement locus of the monitoring target based on the positioning result of the monitoring target, and the movement specified by the user. A reproduction time setting unit 115 that controls a reproduction section by the sensing data reproduction unit 150 so that sensing data corresponding to the line location is reproduced, and an auxiliary graphic creation unit that creates an auxiliary graphic related to the flow line based on the positioning result 112 and a flow line search image creation unit 113 that creates a search image including a flow line and an auxiliary graphic and displays the search image on the display unit 160, so that the user can Since playback points can be selected with reference to auxiliary figures related to the flow line, when playing points are selected using flow lines, the desired playback can be performed more quickly and accurately than in the past. It is able to select the Into.

[Application example]
The sensing data search system 100 described in the present embodiment is applicable to a monitoring system that monitors the movement of a person or an object in a company or factory, a vehicle navigation system, or the like. When the configuration of the present embodiment is applied to a monitoring system, an image captured by the monitoring camera may be stored in the sensing data DB 120. When applied to a navigation system, the sensing data DB 120 stores an image of the surroundings of the host vehicle captured by a camera provided in the host vehicle, and the positioning result DB 130 is acquired by the GPS device of the host vehicle. The positioning result data may be stored.

  In addition, the sensing data search system 100 can be applied to a system for examining changes in the body when performing sports, for example. In this case, the sensing data DB 120 may store biometric data (heart rate, body temperature, etc.) acquired by a measuring instrument attached to a person performing sports, ambient temperature / humidity, and the like. Since the auxiliary figure shows the flow line where the person who is doing the sport moves characteristically, the user can see the biometric data and the surrounding environment when the person who is doing the sport makes a characteristic movement. It can be played quickly. Here, the biometric data and the surrounding environment when a person who is doing a sport makes a characteristic movement can be used as a reference for improving sports ability, for example. By using the configuration of the present embodiment, such reference data can be quickly searched.

(Embodiment 2)
In this embodiment, as an auxiliary figure related to a flow line, a monitoring target is correlated with a movement trajectory that matches a motion pattern registered in advance as stay, stop, reciprocation, high-speed movement, or unnatural movement, or a plurality of monitoring targets. It presents that the figure which shows the part which moved high is displayed. A movement track that matches the motion pattern registered in advance as a stay, stop, reciprocation, high-speed movement, or unnatural movement, or a location that has a highly correlated movement among multiple monitoring objects It is a part that cannot be recognized only from or difficult to recognize. In this embodiment, in consideration of this, the monitoring target is staying, stopping, reciprocating, high-speed movement, a movement trajectory that matches an operation pattern registered in advance as an unnatural movement, or a plurality of monitoring targets have high correlation. Auxiliary figures showing the places that have moved are displayed. As a result, the user can select a movement trajectory that matches a motion pattern registered in advance as a stay, stop, reciprocation, high-speed movement, or unnatural movement, or a place where a plurality of monitoring objects have a highly correlated movement. It can be easily recognized from the auxiliary figure.

  By the way, if the monitoring target is a staying, stopping, reciprocating, high-speed movement, a movement trajectory that matches the motion pattern registered in advance as an unnatural movement, or a location where a movement with high correlation among multiple monitoring targets is detected, the monitoring target It can be said that it is a suspicious part (that is, a monitoring important part). Therefore, by displaying the auxiliary graphic indicating these locations together with the flow line, the user can quickly display a reproduced image when the monitoring target moves suspiciously.

  Further, in the present embodiment, it is presented that an auxiliary graphic indicating the staying state of the monitoring target is displayed. As a result, the user can specify the playback point more quickly and in detail while viewing the auxiliary figure than when only the flow line is used.

  FIG. 6 shows an example of the auxiliary figure of the present embodiment. In the example of FIG. 6, the auxiliary graphic indicating the period during which the monitoring target stays is arranged as the auxiliary graphic in a line with the accumulated flow line location or superimposed on the accumulated flow line location. Here, the retention indicates a state where the monitoring target stays for a predetermined period or more in a narrow section.

  FIG. 6A shows an example of a search image when an auxiliary flow line is displayed as an auxiliary figure. In the auxiliary flow line L2, the horizontal coordinate corresponds to the coordinate on the flow line L1, and the vertical coordinate indicates the passage of time. If the user looks at the auxiliary conducting wire L2, the user can recognize that the monitoring target stays in the place P1 on the flow line L1 from the time t2 to the time t4. In other words, if there is no auxiliary flow line L2, the user cannot even recognize that the place P1 on the flow line L1 is a staying place, but by displaying the auxiliary flow line L2, the user can see the stagnation point P1 and the stagnation time t2. t4 can be recognized at a glance. If the time point on the auxiliary flow line L2 is associated with the time point of the moving image, the moving image can be reproduced from the time point specified by the user on the auxiliary flow line L2. For example, if the user clicks on a time t3 on the auxiliary flow line L2, a moving image from the time t3 can be reproduced. In addition, if the user clicks time point t3 and time point t5 on auxiliary flow line L2, the moving image from time point t3 to time point t5 can be reproduced.

  FIG. 6B shows an example of a search image when a black circle indicating a staying location is displayed on the flow line L1 as a supplementary graphic and a graphic M1 indicating a staying time is displayed. In the example of FIG. 6B, when the user moves the mouse cursor to the position of the black circle in the figure and clicks the mouse, a figure M1 indicating the staying time appears, and when the state where the mouse cursor is clicked continues, the staying time is set. The color in the figure M1 shown changes in the direction of the arrow, the point corresponding to the point in time when the click is stopped is set as the moving image reproduction start point, and the moving image is reproduced. When the mouse is clicked at the position of the black circle in the figure, a figure M1 indicating the residence time appears, and a reproduction start point may be set by selecting a desired reproduction time point in the figure M1.

  FIG. 6C shows an example in which a line L3 indicating a staying time is connected to a staying place on the flow line L1 as an auxiliary figure. The line L3 indicating the residence time is preferably a color different from the flow line L1 or a different line type. The length of the line L3 indicating the residence time is preferably set to a length proportional to the residence time.

  FIG. 6D shows an example in which a circle M2 having a larger radius as the dwell time is displayed as an auxiliary figure at a dwell location on the flow line. For example, if the vicinity of the center of the circle M2 is clicked, the moving image from the time point t2 is reproduced, and if the vicinity of the outer periphery of the circle M2 is clicked, the moving image from the time point t4 is reproduced.

  For example, if the accident occurrence point is displayed on the auxiliary flow line L2 in FIG. 6A, the user can recognize the time when the accident occurred from the auxiliary figure, so that the video before and after the occurrence of the accident can be easily reproduced. Will be able to.

  FIG. 7 in which the same reference numerals are assigned to the parts corresponding to those in FIG. 1 shows the configuration of the sensing data search system of the present embodiment. In the following description, a part of the sensing data search system 200 that is different from the sensing data search system 100 of FIG. 1 will be described.

  The sensing data search system 200 includes a moving image DB 201 and a moving image reproducing unit 202. The search device 210 has a staying figure creating unit 211 that creates a staying figure as an auxiliary figure.

  Based on the positioning result data from the positioning result DB 130, the staying figure creating unit 211 detects a staying state and a staying time as shown in FIG. The staying figure information S10 indicating that is created. The staying figure creating unit 211 sends the staying figure information S10 to the staying section selecting unit 212, the flow line search image creating unit 213, and the reproduction position setting unit 214.

  The stay section selection unit 212 uses the user selection information S3, the flow line data S1, and the staying graphic information S10 to create input status information S11 and sends this to the flow line search image creation unit 213. The input status information S11 includes information for displaying a mouse cursor at a position selected by the user. The input status information S11 includes information for controlling the flow line selected by the user, the color of the staying figure, and the like. Further, the input status information S11 may include information indicating whether or not to display a staying figure, and at which position to display the staying figure.

  The reproduction position setting unit 214 obtains a reproduction time based on the user selection information S3, the flow line data S1, and the staying figure information S10, and sends a reproduction control signal S6 including this reproduction time to the moving image reproduction unit 202. . Specifically, the reproduction position setting unit 214 includes coordinates on the flow line or staying graphic specified by the user included in the user selection information S3, coordinate data and time included in the flow line data S1 or staying graphic information S10. The time data corresponding to the specified flow line coordinates or staying figure is obtained as the reproduction time.

  Instead of the staying figure creation unit 211, a movement locus that matches a motion pattern registered in advance as a stop, reciprocation, high-speed movement, or unnatural movement, or a location that has a highly correlated movement among a plurality of monitoring targets is obtained. By providing an auxiliary figure creation unit that creates auxiliary figures indicating these locations, a movement trajectory that matches a motion pattern registered in advance as a stop, reciprocation, high-speed movement, unnatural movement of a monitoring target, or a plurality of Locations that are highly correlated in the monitoring target can be displayed as auxiliary figures.

(Embodiment 3)
In the present embodiment, as an auxiliary figure related to a flow line, it is presented that a graphic including a thumbnail image is arranged at a location where a monitoring target on the flow line has a characteristic movement. As a result, the user can more accurately determine the point to be reproduced based on the thumbnail image as compared with the case where a graphic such as a black circle is simply displayed on the flow line in which the monitoring target has a characteristic movement. Therefore, a desired reproduction point can be selected more quickly and accurately.

  FIG. 8 shows an example of a search image according to the present embodiment. Thumbnail images J1, J2, and J3 are arranged and displayed at locations where the monitoring target on the flow line L1 has a characteristic movement. Then, for example, when the user moves the mouse cursor over the thumbnail image J2 and clicks the mouse at that position, a moving image of a predetermined section before and after the thumbnail image J2 is reproduced.

  FIG. 9 in which the same reference numerals are assigned to corresponding parts as in FIG. 1 shows the configuration of the sensing data search system of the present embodiment. In the following description, portions where the sensing data search system 300 is different from the sensing data search system 100 of FIG. 1 will be described.

  The sensing data search system 300 includes a moving image DB 301 and a moving image playback unit 302. The search device 310 includes a thumbnail creation unit 311 that creates a graphic including a thumbnail image as an auxiliary graphic.

  The thumbnail creation unit 311 inputs the positioning result data from the positioning result DB 130 and the moving image data from the moving image DB 301. Based on the positioning result data from the positioning result DB 130, the thumbnail creating unit 311 detects a location where the monitoring target as described above has a characteristic movement. The thumbnail creation unit 311 acquires an image of the detected location from the moving image DB 301 and uses this as a thumbnail image. The thumbnail creation unit 311 sends thumbnail image information S20 including a thumbnail image display location and a thumbnail image to the selection status creation unit 312, the flow line search image creation unit 313, and the playback position setting unit 314.

  The selection status creation unit 312 creates input status information S21 using the user selection information S3, the flow line data S1, and the thumbnail image information S20, and sends this to the flow line search image creation unit 313. The input status information S21 includes information for displaying a mouse cursor at a position selected by the user. Further, the input status information S21 includes information for controlling such as changing the color of the flow line selected by the user or adding a frame to the thumbnail image selected by the user.

  The reproduction position setting unit 314 obtains a reproduction time based on the user selection information S3, the flow line data S1, and the thumbnail image information S20, and sends a reproduction control signal S6 including the reproduction time to the moving image reproduction unit 302. . Specifically, the playback position setting unit 314 includes the coordinates of the flow line or thumbnail specified by the user included in the user selection information S3, and the coordinate data and time meta data included in the flow line data S1 or the thumbnail image information S20. Using the data, time data corresponding to the specified flow line coordinates or thumbnail image is obtained as the reproduction time.

(Embodiment 4)
In the present embodiment, as an auxiliary figure related to a flow line, it is presented that a graphic showing a camera that captures a moving image of each flow line section is displayed. Accordingly, the user can select which camera should play back the moving image captured by using the auxiliary figure, and thus when the moving images acquired by the plurality of cameras are stored in the moving image DB. In addition, a desired reproduction point and a desired camera image can be selected quickly and accurately.

  FIG. 10A shows an example of a search image according to the present embodiment. FIG. 10B shows an example of a moving image to be played back.

  As shown in FIG. 10A, along the flow line L1, flow lines L1-1, L1-2, and L1-3 indicating the shooting ranges of the cameras K1, K2, and K3 are displayed as auxiliary figures. In the example of FIG. 10A, the flow line L1-1 indicating the shooting range by the camera K1 and the camera K1 is displayed in red, the flow line L1-2 indicating the shooting range by the camera K2 and the camera K2 is displayed in green, and the camera K3. The flow line L1-3 indicating the shooting range by the camera K3 is displayed in blue.

  For example, when the user designates a reproduction start point and a reproduction end point on the flow line L1, as shown in FIG. 10B, the moving images acquired by all the cameras K1, K2, and K3 are reproduced in the designated reproduction section. Is done.

  On the other hand, for example, when the user designates a reproduction start point and a reproduction end point on the flow line L1-1, a moving image that is acquired by the camera K1 and is designated is reproduced.

  For example, in the section H1 in the figure, a moving image acquired by the camera K1 and a moving image acquired by the camera K2 can be reproduced. Of these moving images, the user views a moving image obtained by the camera K2. If desired, the playback point may be specified on the flow line L1-2.

  Further, as shown in FIG. 10A, by displaying the arrangement positions of the cameras K1, K2, and K3 as auxiliary figures, the user can determine from which direction the moving image stored in the moving image DB is captured. So that you can select the video you want more quickly.

  FIG. 11 in which the same reference numerals are assigned to corresponding parts as in FIG. 1 shows the configuration of the sensing data search system of the present embodiment. In the following description, the sensing data search system 400 will be described with respect to parts different from the sensing data search system 100 of FIG.

  The sensing data search system 400 includes a moving image DB 401 and a moving image playback unit 402. The moving image DB 401 stores a plurality of moving images acquired by a plurality of cameras. The search device 410 includes a camera information creation unit 411 that creates a figure indicating a camera that has captured a moving image of each flow line section as an auxiliary figure.

  The camera information creation unit 411 inputs positioning result data from the positioning result DB 130. Also, the camera information creation unit 411 inputs camera identification information and camera imaging range information attached to the moving image data from the moving image DB 401. The camera information creation unit 411 creates flow lines L1-1, L1-2, and L1-3 as shown in FIG. 10A from these pieces of information, and uses them as camera information S30. The image is sent to the search image creation unit 413 and the playback position setting unit 414.

  Note that the camera information creation unit 411 performs image processing on moving image data and associates a person's detection position and time with positioning result data, thereby causing flow lines L1-1, L1-2, and L1-3 as illustrated in FIG. 10A. May be created.

  The camera selection unit 412 creates input status information S31 using the user selection information S3, the flow line data S1, and the camera information S30, and sends this to the flow line search image creation unit 413. The input status information S31 includes information for displaying a mouse cursor at a position selected by the user. The input status information S31 includes information for controlling the flow line selected by the user or the color of the camera.

  Based on the user selection information S3, the flow line data S1, and the camera information S30, the playback position setting unit 414 determines which camera plays the moving image and the playback time, and determines the camera type and playback. A reproduction control signal S6 including the time is sent to the moving image reproduction unit 402.

(Embodiment 5)
In the present embodiment, as an auxiliary figure related to a flow line, the flow line of another monitoring target indicating a movement trajectory in which the other monitoring target has moved in the same time zone as the flow line section designated on the flow line of the certain monitoring target. Present to display. As a result, the user can recognize how the other monitoring target was moving during the period in which the certain monitoring target moved in a certain section by looking at the auxiliary graphic, and other monitoring displayed as an auxiliary graphic. It is also possible to select a playback video using the target flow line.

  FIG. 12 shows an example of a search image according to this embodiment. First, as shown in FIG. 12A, a flow line L1 to be monitored is displayed. When the user drags the mouse cursor in a predetermined section of the flow line L1, as shown in FIG. 12B, other monitored flow lines L10 and L11 in the same time zone as the dragged section are displayed. When a playback point is designated on the flow lines L10 and L11, a moving image corresponding to the designated flow lines L10 and L11 is reproduced.

  FIG. 13 in which the same reference numerals are assigned to the parts corresponding to those in FIG. 1 shows the configuration of the sensing data search system of the present embodiment. In the following description, a part in which the sensing data search system 500 is different from the sensing data search system 100 in FIG. 1 will be described.

  The sensing data search system 500 includes a moving image DB 501 and a moving image playback unit 502. The search device 510 extracts, as an auxiliary figure, a flow line of another monitoring target indicating a movement trajectory in which the other monitoring target has moved in the same time zone as the flow line section specified on the flow line of a certain monitoring target. A flow line extraction unit 511 is included.

  Note that the corresponding flow line extraction unit 511 may extract only other flow lines to be monitored that exist in the vicinity of the flow line L1 to be monitored.

  The corresponding flow line extraction unit 511 creates a flow line based on the input status information S41 output from the selection status creation unit 512 and the positioning result data. Specifically, since the input status information S41 includes information on the flow line section specified by the user, the corresponding flow line extraction unit 511 sets the period corresponding to the specified flow line section to the positioning result DB 130. If there is a flow line of another monitoring target in the determined period, the flow line of another monitoring target corresponding to the determined period is extracted (created). The corresponding flow line extraction unit 511 sends the extracted flow line data S40 to the selection status creation unit 512, the flow line search image creation unit 513, and the reproduction position setting unit 514.

  The selection status creation unit 512 creates input status information S41 using the user selection information S3 and the flow line data S1 and S40, and sends this to the flow line search image creation unit 513. The input status information S41 includes information for displaying a mouse cursor at a position selected by the user. The input status information S41 includes information for controlling the color of the flow line selected by the user.

  The reproduction position setting unit 514 obtains a reproduction time based on the user selection information S3 and the flow line data S1 and S40, and sends a reproduction control signal S6 including the reproduction time to the moving image reproduction unit 502. In addition, the playback position setting unit 514 selects one of the camera images suitable for the flow line depending on which flow line to be monitored is selected by the user from a plurality of moving images acquired by different cameras. You may select and play.

(Embodiment 6)
In the present embodiment, it is presented that a graphic showing the current playback point is displayed as an auxiliary graphic related to the flow line. As a result, the user can easily recognize which part on the current flow line is being reproduced.

  14 and 15 show examples of search images according to the present embodiment. In the example of FIG. 14, an example is shown in which a line L <b> 20 that extends as the playback point advances along the flow line L <b> 1 is displayed as an auxiliary figure. In addition, in the example of FIG. 14, an example in which a reproduced image (sensing data) J10 up to the current reproduction point is intermittently displayed on the flow line L1 as an auxiliary figure is also shown. FIG. 15 shows an example in which a reproduction image (sensing data) J20 corresponding to the current reproduction point is arranged as an auxiliary figure at a position corresponding to the reproduction point on the flow line L1.

  FIG. 16, in which parts corresponding to those in FIG. 1 are assigned the same reference numerals, shows the configuration of the sensing data search system of this embodiment. In the following description, a part in which the sensing data search system 600 is different from the sensing data search system 100 in FIG. 1 will be described.

  The sensing data search system 600 includes a moving image DB 601 and a moving image playback unit 602. The search device 610 includes a reproduction position creation unit 611 that creates a graphic indicating the current reproduction point as an auxiliary graphic.

  The playback position creation unit 611 inputs positioning result data from the positioning result DB 130 and inputs time information of the current playback location from the moving image playback unit 602. Then, the playback position creation unit 611 creates a figure serving as a mark indicating the current playback point, as shown in FIG. 14 or FIG. 15, as the auxiliary figure, and uses this as the playback position information S50 as a moving image search image creation unit. 113 and the selection status creation unit 114.

  In the present embodiment, the search image obtained by the flow line search image creation unit 113 and the playback video obtained by the video playback unit 602 are simultaneously displayed in different display areas of the display unit 160. ing.

(Embodiment 7)
In the present embodiment, as an auxiliary graphic relating to the flow line, it is presented that a graphic capable of specifying the monitoring target corresponding to the flow line designated by the user is formed in the reproduced image. As a result, the user can easily recognize which monitoring target is the designated flow line on the reproduced image.

  FIG. 17A shows an example of a search image according to the present embodiment. FIG. 17B shows an example of a reproduced image of the present embodiment.

  As shown in FIG. 17A, the flow lines L1-a, L1-b, and L1-c of the monitoring targets a, b, and c are displayed in the search image. In addition, auxiliary graphics F1 and F2 indicating the characteristic locations are displayed on the flow line L1-a. Of course, the auxiliary graphic indicating the characteristic location may also be displayed on the flow lines L1-b and L1-c. Then, for example, when the user clicks on the feature location F2 on L1-a, a moving image corresponding to the feature location F2 is reproduced. At this time, as shown in FIG. 17B, a frame W-a surrounding the monitoring target corresponding to the flow line L1-a is displayed as an auxiliary figure. Thereby, the user can recognize the monitoring target a corresponding to the designated flow line L1-a.

  FIG. 18, in which the same reference numerals are assigned to corresponding parts as in FIG. 1, shows the configuration of the sensing data search system of this embodiment. In the following description, a part of the sensing data search system 700 that is different from the sensing data search system 100 of FIG. 1 will be described.

  The sensing data search system 700 includes a moving image DB 701 and a moving image playback unit 702. The moving image DB 701 stores a moving image, a time, a user ID for identifying a monitoring target, and position information. That is, data such as (moving image, (time, (user ID, position information))) is stored in the moving image DB 701. As a specific example, for example, the following data is stored.

(test.mpg, (t1, (UID1, (10,20)), (UID2, (40,20))),
(t2, (UID1, (10,25)), (UID2, (40,25))),
(t3, (UID1, (10,30)), (UID2, (40,30)), (UID3, (5,5)))
This data means that at time t1, the monitoring target UID1 is located at the in-screen coordinates (10, 20) and UID2 is located at (40, 20). Also, at time t3, it means that the monitoring target UID3 appears at the in-screen coordinates (5, 5).

  The playback time setting unit 711 outputs the playback control signal S6 including the monitoring target user ID corresponding to the selected flow line in addition to the playback start / end time.

  The video playback unit 702 acquires the video of the playback section specified by the playback time setting unit 711 from the video DB 701. In addition, the moving image reproduction unit 702 acquires metadata of position information within the screen of the designated user ID from the moving image DB 701. In addition, the moving image reproduction unit 702 creates frame graphic data (the size of the person is a fixed size) based on the acquired position information, and superimposes the frame Wa (FIG. 17B) on the reproduced moving image. Specifically, when test time t1, end time t3, and user ID: UID1 are specified as test.mpg, coordinates (10, 20) at time t1 and coordinates (10 at time t2) in the playback video. 25), a frame is displayed at the coordinates (10, 30) at time t3. As described above, in the present embodiment, the moving image reproducing unit 702 forms the frame Wa that is an auxiliary figure.

  Although FIG. 18 shows a configuration in which a frame for specifying a monitoring target corresponding to a designated flow line is created using metadata, the frame can be created in real time. FIG. 19 shows a configuration for creating a frame in real time.

  In FIG. 19, in which the same reference numerals are assigned to corresponding parts as in FIG. 18, the sensing data search system 800 includes a moving image DB 801, a moving image playback unit 802, a person DB 803, and a person tracking unit 804.

  The playback time setting unit 711 outputs the playback control signal S6 including the monitoring target user ID corresponding to the selected flow line in addition to the playback start / end time.

  The video playback unit 802 transmits the playback start / end times and the monitoring target user ID corresponding to the selected flow line to the person tracking unit 804, and the person tracking unit 804 sends the user ID in the screen of the corresponding user ID. Get the coordinates. The video playback unit 802 creates frame graphic data based on the in-screen coordinates acquired from the person tracking unit 804, and superimposes the frame Wa (FIG. 17B) on the playback video acquired from the video DB 801.

  The person tracking unit 804 acquires a person image corresponding to the user ID input from the moving image playback unit 802 from the person DB 803, and performs a pattern matching process using the moving image acquired from the moving image DB 801 and the person image acquired from the person DB 803. Thus, the coordinates of the person corresponding to the user ID in the moving image are obtained, and the coordinates are sent to the moving image reproducing unit 802.

  The present invention has an effect of selecting a desired playback point quickly and accurately when selecting a playback point using a flow line. For example, a desired video can be selected from a large number of videos as in a surveillance system. Therefore, it is suitable for use in reproduction.

The block diagram which shows the structure of the sensing data search system which concerns on Embodiment 1 of this invention. 2A is a diagram illustrating an example of a search image according to the first embodiment, FIG. 2B is a diagram illustrating an example of a reproduction image corresponding to a characteristic location, and FIG. 2C is an example of a reproduction image corresponding to a location designated by a drag. Figure Sequence diagram for explaining search image creation processing when there is no input from the user Sequence diagram for explaining search image creation processing when the user moves the mouse Sequence diagram for explaining search image creation processing when user selects playback start / end point 6A to 6D are diagrams showing examples of auxiliary figures in the second embodiment. Block diagram showing the configuration of the sensing data search system of the second embodiment FIG. 10 is a diagram illustrating an example of a search image in the third embodiment Block diagram showing a configuration of a sensing data search system according to the third embodiment FIG. 10A is a diagram showing an example of a search image in the fourth embodiment, and FIG. 10B is a diagram for explaining a reproduced image in the fourth embodiment. Block diagram showing a configuration of a sensing data search system according to the fourth embodiment FIG. 12A is a diagram showing an example of a search image when the user designates a flow line section, and FIG. 12B is a diagram showing an example of a search image in which another flow line corresponding to the designated flow line section is displayed. Block diagram showing a configuration of a sensing data search system according to the fifth embodiment FIG. 10 is a diagram illustrating an example of a search image in the sixth embodiment FIG. 10 is a diagram illustrating an example of a search image in the sixth embodiment Block diagram showing a configuration of a sensing data search system according to the sixth embodiment FIG. 17A is a diagram illustrating an example of a search image according to the seventh embodiment, and FIG. 17B is a diagram illustrating an example of a reproduction image according to the seventh embodiment. Block diagram showing a configuration of a sensing data search system according to the seventh embodiment Block diagram showing a configuration of a sensing data search system according to the seventh embodiment

Explanation of symbols

100, 200, 300, 400, 500, 600, 700, 800 Sensing data search system 110, 210, 310, 410, 510, 610, 710 Search device 111 Flow line creation unit 112 Auxiliary figure creation unit 113, 213, 313, 413, 513 Flow line search image creation unit 114, 312, 512 Selection status creation unit 115, 711 Playback time setting unit 116 User input reception unit 120 Sensing data DB
130 Positioning result DB
140 Accident Information DB
150 Sensing data reproduction unit 160 Display unit 201, 301, 401, 501, 601, 701, 801 Movie DB
202, 302, 402, 502, 602, 702, 802 Movie playback unit 211 Stay figure creation unit 212 Stay section selection unit 214 314 414 514 Playback position setting unit 311 Thumbnail creation unit 411 Camera information creation unit 412 Camera selection unit 511 Corresponding flow line extraction unit 611 Playback position creation unit 803 Person DB
804 Person tracking unit S1, S40 Flow line data S2 Auxiliary graphic information S3 User selection information S4, S11, S21, S31, S41 Input situation information S5 Search image information S6 Playback control signal S7 Playback image information S10 Retention image information S20 Thumbnail image information S30 Camera information S50 Playback position information L1, L10, L11 Flow line F1, F2 Auxiliary figure J1-J3 Thumbnail image

Claims (16)

Based on the positioning result for the monitoring target, a flow line creating means for creating a flow line that is a movement locus of the monitoring target;
Reproduction control means for controlling the reproduction section by the reproduction means so that sensing data corresponding to the flow line location or the flow line section designated by the user is reproduced;
An auxiliary graphic creating means for creating an auxiliary graphic related to the flow line based on the positioning result;
A search image creating means for creating a search image including the flow line and the auxiliary figure, and displaying the search image on a display means;
Sensing data retrieval apparatus comprising: The auxiliary graphic is a graphic showing a location where the monitoring target has moved characteristically,
The search image is an image in which a graphic showing the location of the characteristic movement is arranged on the flow line.
The sensing data search device according to claim 1. The characteristic movement is dwell, stop, reciprocation, high-speed movement, movement trajectory that matches an operation pattern registered in advance as an unnatural movement, or a movement highly correlated with a plurality of monitoring targets.
The sensing data search device according to claim 2. The auxiliary graphic is a graphic showing a period during which the monitoring target stays or stops,
The search image is an image in which the auxiliary graphic is arranged side by side or overlapping the flow line where the monitoring target stays or stops.
The sensing data search device according to claim 1. The auxiliary graphic is a graphic showing the location where the accident occurred,
The sensing data search device according to claim 1. The auxiliary graphic is a graphic showing a monitoring important point,
The sensing data search device according to claim 1. The sensing data is a moving image of the monitoring target,
The auxiliary graphic is a graphic including the thumbnail image to be monitored,
The search screen creation means creates an image in which the thumbnail image is arranged at a location where the monitoring target has a characteristic movement on the flow line as the search image.
The sensing data search device according to claim 1. The sensing data is a plurality of moving images captured by a plurality of cameras,
The auxiliary figure is a figure showing a camera that captures a moving image of each flow line section,
The search image is an image in which the flow line and the auxiliary figure are arranged side by side or overlapped.
The sensing data search device according to claim 1. The flow line creating means creates a plurality of flow lines based on positioning results for a plurality of monitoring targets,
The auxiliary figure creating means obtains a period corresponding to a designated section of the flow line designated by the user using a flow line of one of the plurality of monitoring objects, and the auxiliary figure as the auxiliary figure Create a flow line that is the movement trajectory of other monitored objects in
The search screen creation means creates an image including the flow line of the one monitoring target and the flow line of the other monitoring target as the search image.
The sensing data search device according to claim 1. The sensing data is a moving image of the monitoring target,
The auxiliary figure creating means receives information of a playback point currently being played back from the reproducing means, creates a figure indicating the playback point as the auxiliary figure,
The search screen creation means creates an image including the flow line and a graphic showing the playback point as the search image.
The sensing data search device according to claim 1. The graphic indicating the playback point is a graphic including a playback image corresponding to the current playback point, arranged at a position corresponding to the playback point on the flow line.
The sensing data search device according to claim 10. The sensing data is a moving image in which the monitoring target is imaged,
The auxiliary graphic is a graphic including intermittent reproduction images,
The search image is an image in which playback images up to the current playback point are intermittently arranged on the flow line.
The sensing data search device according to claim 10. The sensing data is a moving image in which the monitoring target is imaged,
The reproduction unit forms a reproduction image capable of specifying a monitoring target corresponding to a flow line designated by a user;
The sensing data search device according to claim 1. The auxiliary graphic is temporally associated with the sensing data,
The reproduction control means reproduces sensing data for a period specified by the user using the auxiliary graphic.
The sensing data search device according to any one of claims 1 to 13. Based on the positioning result, creating a flow line that is a movement trajectory of the monitoring target;
Creating an auxiliary figure related to the flow line based on the positioning result;
Creating a search image including the flow line and the auxiliary figure;
Search image creation method including The flow line and the auxiliary figure are associated with reproduced sensing data.
The search image creation method according to claim 15.

Images