JP7132860B2 - VIDEO INFORMATION MANAGEMENT SYSTEM AND VIDEO INFORMATION MANAGEMENT METHOD - Google Patents

Description

The present invention relates to a video information management system and a video information management method, and is suitable for application to, for example, a video information management system and a video information management method for managing video information.

In Japan, the number of sewage pipeline facilities exceeding their service life will increase rapidly in the future. Inspection work is expected to increase in order to formulate construction plans for this renewal and rehabilitation. In the past, people had to enter the sewage pipes to inspect them, but as the amount of work increases, it is expected that there will be an increase in the number of methods in which robots such as vehicle-type robots, ship-type robots, and drones are placed inside the pipes to photograph the inside. be. As a result, the number of video files captured by robots in sewage pipes is expected to increase significantly.

If a commercially available camera is used to take a picture of the inspection point, a file name including a sequential number such as "CAM00123.mp4" is often automatically assigned when the video file is recorded. From the file name alone, it is not possible to know the contents of the video, such as which sewage pipe was inspected, or what construction work was involved in the inspection.

Rewriting the name of the video file each time an inspection is performed is time-consuming, and in addition to that, is it necessary to add only the construction name to the file name, add the date and time, or include other information? There is a high possibility that the format will be different each time depending on the worker, such as whether or not it is possible, and a new problem will arise that makes management difficult.

Here, it is conceivable to make it possible to understand the contents of the inspection by projecting a construction blackboard on which unique information related to the inspection work, such as the name of the sewage pipe to be inspected, the name of the construction work, and the date and time of the inspection, is shown in a part of the video. However, the content is unknown until the video file is opened and viewed. Since video files generally have a large capacity, playback does not start immediately, and it takes time to see the part where the blackboard for construction is shown. If it turns out that the image file is different from the desired image file, it is necessary to repeat the same operation, and it is not possible to efficiently confirm the inner surface image corresponding to the sewage pipe to be confirmed.

In this regard, a technique has been disclosed for efficiently providing information individually needed by an individual from a huge amount of information (see Patent Document 1).

JP 2006-031379 A

However, the technique described in Patent Literature 1 passively captures the position information, and for example, when there is only an image inside the sewage pipe, the technique cannot be applied because there is no position information. Therefore, with the technique described in Patent Document 1, position information corresponding to the image (related information) may not be obtained, and there is a possibility that the inner surface image corresponding to the sewage pipe to be confirmed cannot be efficiently presented.

SUMMARY OF THE INVENTION The present invention has been made in consideration of the above points, and is intended to propose a video information management system that allows easy access to video information of a subject to be checked.

In order to solve such a problem, in the present invention, an input unit for inputting shooting target information related to a shooting target, a shooting target to be shot is selected from the shooting target information input by the input unit, and the selected shooting target is selected. a selection unit that selects identification information that can uniquely identify a target from the imaging target information of the imaging target; and a display that displays the identification information that uniquely identifies the imaging target selected by the selection unit on a display device. an extraction unit for extracting the identification information displayed on the display device from the image information captured by the image capturing device; a linking unit that specifies target information, generates and outputs linking information linking the specified shooting target information or part of the shooting target information with which the shooting target can be identified, and the video information; I set it up.

According to the above configuration, since the linking information in which the shooting target and the video information are linked is output, for example, the user can accurately grasp the video information of the desired shooting target from the video information captured in large quantities. , the contents can be checked in a short time.

According to the present invention, video information can be appropriately managed.

It is a figure showing an example of composition concerning a picture information management system by a 1st embodiment. It is a figure which shows an example of the inspection plan list|wrist by 1st Embodiment. It is a figure which shows an example of the imaging|photography aspect of inspection plan characteristic information by 1st Embodiment. It is a figure which shows an example of the inspection plan characteristic information by 1st Embodiment. FIG. 5 is a diagram showing an example of a mode when a plurality of pieces of image information are captured by the image linking device according to the first embodiment; 4 is a diagram showing an example of the data structure of video information according to the first embodiment; FIG. It is a figure which shows an example of the reference table by 1st Embodiment. FIG. 4 is a diagram showing an example of processing in the video information management system according to the first embodiment; FIG. FIG. 7 is a diagram showing an example of processing related to searching for detection marks according to the first embodiment; It is a figure which shows an example of the inspection plan characteristic information by 1st Embodiment. It is a figure which shows an example of the inspection plan characteristic information by 1st Embodiment. It is a figure which shows an example of the inspection plan characteristic information by 1st Embodiment. FIG. 10 is a diagram showing an example of the configuration of a video information management system according to a second embodiment; FIG. FIG. 11 is a diagram showing an example of a configuration of a video information management system according to a third embodiment; FIG.

One embodiment of the present invention will be described in detail below with reference to the drawings. The present invention relates to a technique for managing a plurality of pieces of image information, and for example, to a technique for capturing a predetermined work (inspection, investigation, etc.) and efficiently confirming a desired image after the work. There is no particular limitation as long as there is shooting target information including information that allows the shooting target to be uniquely identified before shooting the video, and a plurality of pieces of video information obtained as a result of shooting. Below, inspection of a sewage pipe is mentioned as an example and demonstrated as a predetermined operation|work.

(1) First Embodiment In FIG. 1, 100 generally indicates a video information management system according to the first embodiment.

The video information management system 100 includes a video linking device 110 and a video playback device 120, and manages video information (for example, video files and video data) captured by the imaging device 130. FIG.

The image linking device 110 is a tablet terminal, a notebook computer, a personal computer, a server device, or the like, and includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), a HDD (Hard Disk), which are not shown. Drive), communication unit, etc. Note that the video linking device 110 may include a display device, or may be communicably connected to the display device.

The functions of the video linking device 110 (the input unit 111, the selection unit 112, the display unit 113, the extraction unit 114, the linking unit 115, etc.) are, for example, that the CPU reads a program stored in the ROM into the RAM and executes it (software ), hardware such as a dedicated circuit, or a combination of software and hardware. Also, part of the functions of the video linking device 110 may be implemented by another computer (computer) that can communicate with the video linking device 110 .

The input unit 111 reads the inspection plan list 141 . The selection unit 112 selects an inspection plan to be inspected from the read inspection plan list 141, and inspection information (an example of inspection target information) corresponding to the selected inspection plan (inspection). information 116) is provided to the display unit 113. FIG. The display unit 113 displays on-site inspection plan feature information 142 including unique identification information corresponding to the selected inspection plan (inspection to be performed on-site from the inspection plan list 141). The extraction unit 114 extracts the identification information from the image included in the video information 143 captured on site by the imaging device 130 . The linking unit 115 compares the extracted identification information (for example, the individual video feature information 117 described later) with the inspection plan list 141 (which may be the inspection plan feature information 142), and the selected inspection plan and the image information 143 (for example, a reference table 144 to be described later) is generated and output.

The video playback device 120 is a tablet terminal, a notebook computer, a personal computer, a server device, or the like, and includes a CPU, RAM, ROM, HDD, communication section, and the like (not shown). Note that the video playback device 120 may include a display device, or may be communicably connected to the display device.

The functions of the video playback device 120 (designation unit 121, playback unit 122, etc.) may be realized, for example, by the CPU reading a program stored in the ROM into the RAM and executing it (software), or by using a dedicated circuit. or by a combination of software and hardware. Also, part of the functions of the video playback device 120 may be implemented by another computer that can communicate with the video playback device 120 .

The designation unit 121 receives designation of an inspection plan 145 that the user wants to confirm, and acquires video information 143 (extracted inspection video 123) of the designated inspection plan 145 based on the reference table 144 . The reproducing unit 122 reproduces (displays) the extracted inspection video 123 as the inspection video 146 .

The photographing device 130 is a device capable of photographing an object to be inspected, and is a robot (vehicle type robot, ship type robot, drone, etc.), a digital camera, a video camera, or the like. As for the imaging device 130, a commercially available product can be appropriately adopted.

As shown in FIG. 1, the functions of the video linking device 110 and the functions of the video playback device 120 may be provided in different computers, but the present invention is not limited to this. For example, the functions of the video linking device 110 and the functions of the video playback device 120 may be provided in the same computer.

FIG. 2 is a diagram showing an example of the inspection plan list 141. As shown in FIG. The inspection plan list 141 includes a serial number 201 by which an inspection (inspection target) can be identified, a construction name 202 indicating the name of the construction work to be inspected, a scheduled date and time 203 indicating the scheduled date and time of the inspection, and an inspection schedule. Inspection information in which a scheduled position 204 indicating a scheduled position to be inspected is associated with display coordinates 205 on a screen map of an inspection target is stored. An inspection plan list 141 including inspection information for multiple inspections is prepared in advance prior to inspection.

If the inspection information includes identification information that can uniquely identify the inspection target in the inspection plan list 141, some of the data items 201 to 205 described above may not be included. However, information different from the data items 201 to 205 described above may be included.

FIG. 3 is a diagram showing an example of a shooting mode of the inspection plan characteristic information 142. As shown in FIG. In FIG. 3 , a tablet terminal is used as the image linking device 110 , inspection plan characteristic information 142 is displayed, and an example of an image (image 300 ) captured by the imaging device 130 is shown. Placing the image linking device 110 at the inspection site makes it possible to easily include the inspection plan characteristic information 142 in the image 300 of the image information 143 .

Here, it is possible to capture the location information and the like in the video information 143 by reflecting the blackboard for construction in the video, but since the blackboard for construction is written by hand at the site, it is shortened depending on the person who writes it. It includes writing errors, etc., and there is a possibility that the content of the description does not match the inspection plan. Furthermore, when recognizing characters on a construction blackboard by image processing, there is a possibility that reading errors may occur due to the variety of writing habits and character styles.

In this regard, since the inspection plan characteristic information 142 includes identification information that can uniquely identify the inspection target extracted from the inspection information, the above-described situation can be avoided according to the identification information. .

FIG. 4 is a diagram showing an example of the inspection plan characteristic information 142. As shown in FIG. The inspection plan feature information 142 includes a one-dimensional barcode 410 that can uniquely identify an inspection target in the inspection plan list 141, a detection mark 420 for facilitating detection of the one-dimensional barcode, and an inspection target in the inspection plan list 141. It includes information 430 (construction date and time, construction name, construction site, construction content, construction company, combination of these, etc.) that can uniquely identify the object.

The one-dimensional barcode 410 includes identification information (for example, individual video feature information 117). The information 430 that can uniquely identify the inspection object also includes identification information (for example, the individual video feature information 117).

Detection marks 420 are provided at both ends of the one-dimensional barcode 410 . The detection mark 420 has a striped pattern similar to the one-dimensional barcode 410, but is not limited to this form. Forms (patterns, colors, or shapes) that are not or rarely found in nature can be appropriately adopted. In the following description, an example in which a striped pattern of a predetermined color is used as the detection mark 420 will be described.

As described above, the inspection plan characteristic information 142 includes the one-dimensional barcode 410 and the information 430 that can uniquely identify the inspection target. Therefore, from the computer's point of view, when linking the inspection object and the image information 143, the one-dimensional barcode 410 makes it possible to easily grasp the inspection. For the user, when reproducing the video information 143, the information 430 that allows the inspection object to be uniquely identified allows the user to easily comprehend the inspection. Note that the inspection plan feature information 142 is not limited to a configuration that includes both the one-dimensional barcode 410 and the information 430 that can uniquely identify the inspection target, and includes either one of them. good too.

FIG. 5 is a diagram showing an example (folder 500) when a plurality of pieces of image information 143 are taken into the image linking device 110. As shown in FIG.

As shown in FIG. 2, when multiple inspections are included in the inspection plan list 141, video information 143 corresponding to each inspection is generated after the inspection. When using a commercially available camera to photograph an inspection object, in many cases, a file name including a sequential number is automatically assigned, such as "CAM00123.mp4". As shown in FIG. 5, it is not possible to know which sewage pipe was inspected in the video or what kind of construction the inspection was performed from only the file name. Therefore, the inside image corresponding to the sewage pipe to be checked cannot be reproduced immediately.

FIG. 6 is a diagram showing an example of the data structure of the video information 143. As shown in FIG. A horizontally long bar line 601 shown in FIG. 6 indicates the shooting time or recording amount of the video information 143 . In the case of an inspection of a sewage pipe, as shown in a balloon 602, most of the image is of the inside of the sewage pipe, but as shown in a balloon 603, the inspection plan characteristic information 142 is included in the beginning part. The inspection plan characteristic information 142 does not need to be at the beginning, but by determining that the image is captured at the beginning or within a certain period of time from the beginning, it is possible to further reduce the computational complexity of image processing.

FIG. 7 is a diagram showing an example of the reference table 144. As shown in FIG. The reference table 144 has a data structure in which columns are added to the inspection plan list 141 . More specifically, the reference table 144 includes inspection information, a photographing time 701 indicating the time when the inspection target was photographed, and a GPS (Global Positioning System) measurement value indicating the position at which the inspection target was photographed. 702 and information of a video information name 703 indicating the name of the video information are stored in association with each other.

Reference table 144 is not limited to the data structure described above. For example, the shooting time 701 may not be provided, and the GPS measurement value 702 may not be provided.

FIG. 8 is a diagram showing an example of processing in the video information management system 100. As shown in FIG. FIG. 8 exemplifies locations and temporal flow assumed when the video information management system 100 is actually used. (1-1) to (3-4) shown in FIG. 8 correspond to the processing relating to the video linking device 110, and (4-1) to (4-3) correspond to the processing relating to the video reproducing device 120. FIG.

In step S<b>801 , the input unit 111 inputs the prepared inspection plan list 141 . The input unit 111 provides the input inspection plan list 141 to the selection unit 112 and the linking unit 115 .

In step S802, the selection unit 112 selects inspection information (inspection plan) to be inspected from the inspection plan list 141 according to the operator's operation at the site. At the inspection site, the operator implements the inspection plans included in the inspection plan list 141 one by one.

The selection unit 112 provides the selected inspection information (selected inspection plan information 116) to the display unit 113. FIG. The selective inspection plan information 116 includes identification information (unique information that does not overlap). The selected inspection plan information 116 is information included in the inspection plan list 141 . For example, the selected inspection plan information 116 may be either serial number 201, construction name 202, scheduled date and time 203, scheduled position 204, display coordinates 205 on the screen map, or some combination thereof. .

In step S803, the display unit 113 displays the selected inspection plan information 116 as the inspection plan feature information 142 on the screen at the inspection site.

In step S804, the image capturing device 130 captures an image of the site to be inspected. Here, the photographing device 130 is operated and photographed so that the inspection plan characteristic information 142 is reflected in the same video information 143 as the inspection target. The captured video is recorded as video information 143 . At this time, the photographing device 130 automatically adds the name of the video information 143 using a serial number or the like.

As described above, the display unit 113 is a device such as a notebook computer or a tablet terminal that can display a screen at the inspection site in order to display the inspection plan feature information 142 in the video information 143. is desirable. Alternatively, it is desirable to have a computer (computer) with a screen in the mobile vehicle used for inspection.

In step S805, the processes of steps S802 to S804 are performed for the number of inspection plans included in the inspection plan list 141. FIG. Note that the video information 143 is captured by the video linking device 110 at a predetermined timing (for example, periodically in real time according to an operator's operation) and stored in a predetermined folder.

In step S806, the extraction unit 114 automatically reads all the video information 143 in the folder one by one.

In step S807, the extraction unit 114 extracts (recognizes) the individual video feature information 117 included in the video of the read video information 143 by image processing. The individual video feature information 117 includes identification information. The individual image feature information 117 may be the same as the selected inspection plan information 116, may be the same as the inspection plan feature information 142, or may be a part of the inspection plan feature information 142 (for example, the inspection target is identified). possible part). The image processing performed here may be automatic recognition of characters, numbers, symbols, or the like, or may be a technique such as bar code recognition that is less likely to cause errors during reading, or may be a combination of these. The extraction unit 114 provides the extracted individual video feature information 117 to the linking unit 115 . Note that the linking unit 115 is provided with an inspection plan list 141 in addition to the individual video feature information 117 . In addition, examples of extracting the individual video feature information 117 will be described later with reference to FIGS. 9 to 12. FIG.

In step S<b>808 , the linking unit 115 searches whether the inspection plan list 141 includes the individual video feature information 117 . When the inspection plan list 141 contains the individual image feature information 117, the name of the opened image information 143 and the row (inspection information) of the inspection plan list 141 are matched, and the inspection plan list 141 is displayed. By adding (associating) the name of the video information 143 to the row, the reference table 144 is automatically generated.

In addition, regarding which information among the information in the inspection plan list 141 is used to generate the identification information, predefined regulation information is stored, and the linking unit 115 uses the stored regulation information. may be used to determine whether or not the inspection plan list 141 includes the individual video feature information 117 . The selection unit 112 refers to the inspection plan list 141 to select a unique combination of information, stores (holds) selection information indicating the selected combination, and the linking unit 115 uses the stored selection information. It may be determined whether or not the inspection plan list 141 includes the individual video feature information 117 .

It should be noted that the video of the same inspection plan list 141 may be shot multiple times, such as the first time and the second time. In that case, a plurality of video information names may be recorded in the video information name 703 of the reference table 144, or a column may be added to the right of the video information name 703 of the reference table 144 to record the video information name. Multiple records may be made.

In step S809, the processes of steps S807 and S808 are performed on each video information 143 read in step S806. Note that the linking unit 115 transmits the generated reference table 144 to the specifying unit 121 at a predetermined timing (for example, at a predetermined time in response to an operator's operation when all the video information 143 has been processed). do.

In step S810, the designating unit 121 receives the inspection plan 145 whose video is to be confirmed. The inspection plan 145 whose image is to be checked is, for example, the construction name 202 in the second column and the scheduled date and time 203 in the third column, taking the reference table 144 shown in FIG. 7 as an example. According to the reference table 144, it is possible to determine the name of the image information 143 (extracted inspection image 123) corresponding to the inspection plan 145 whose image is to be confirmed.

In step S811, the specifying unit 121 uses the reference table 144 to specify the image information 143 (extracted inspection image 123) corresponding to the inspection plan 145 whose image is to be confirmed, and stores the image information 143 (extracted inspection image 123) in the image linking device 110 (inside the folder). The extracted inspection image 123 is acquired from the image information 143 stored in the image information 143 and output to the reproduction unit 122 . In addition, if a plurality of video information names are recorded in the same row of the reference table 144, a selection screen may be displayed for selecting which one to reproduce.

In the example of the reference table 144 shown in FIG. 7, various columns other than the construction name 202 and the scheduled date and time 203 are shown as examples. The inspection plan 145 for which video confirmation is desired may be given as information in any column. That is, the serial number 201 in the first column, the planned position 204 (latitude and longitude) in the fourth column, the shooting time 701 in the sixth column, the GPS measurement value 702 in the seventh column, and the like may be used as the inspection plan 145 .

Maps are one of the things that are easier to imagine than these character and numerical information. For example, if a map of sewage pipes is displayed on the screen and the person in charge can select the inspection plan 145 to check the video by specifying (for example, clicking) on the sewage pipe, the work will be reduced for the person in charge. Very likely. The image of the coordinates of this clicked point is the display coordinates 205 on the screen map in the fifth column of the reference table 144 shown in FIG. For simplicity, only one x-coordinate and one y-coordinate (that is, one clickable point) are shown here, but the present invention is not limited to this. For example, the reference table 144 may have a plurality of x-coordinate and y-coordinate information in order to select anywhere on the line of the sewage pipe on the map, or to select even if the point to be clicked is slightly deviated. Alternatively, the reference table 144 may have information (for example, an object identifier) that can identify a sewage pipe (object to be inspected) on the map.

In step S812, the reproducing unit 122 displays (reproduces) the extracted inspection video 123 as the inspection video 146 on the display device. In this way, once the reference table 144 is constructed, after that, when the information of the inspection plan 145 whose image is to be confirmed is given, the desired inspection image 146 can be viewed, and the work efficiency can be greatly improved. becomes.

Next, extraction of the individual video feature information 117 will be described.

In the video information management system 100 , the extractor 114 detects the detection mark 420 and extracts the individual video feature information 117 using the detected detection mark 420 .

Here, detection marks 420 can be used to accurately read the one-dimensional barcode 410 . In the image processing, the detection mark 420 is first searched, and if the detection mark 420 is found, the image between them is processed as the one-dimensional barcode 410 to extract the individual video feature information 117 . Using the detection mark 420 in this manner is useful for reducing the amount of calculation of image processing required for reading the one-dimensional barcode 410 and improving accuracy.

Also, the detection mark 420 preferably satisfies the following requirements.

(Requirement A) Accuracy In the image captured by the image capturing device 130, in addition to the image linking device 110 and the image inside the sewage pipe, various background images such as buildings, vehicles, trees, grass, the ground, manhole walls, etc. However, the detection mark 420 should be found correctly among them.

(Requirement B) Robustness When photographing the image linking device 110 with the photographing device 130, there is a high possibility that the effects of relative distance, relative tilt, and environmental light will differ. be able to read the position of

(Requirement C) High Speed Since the number of video frames is large, such as 30 or 60 frames per second, the calculation can be executed at high speed and the calculation time required for image processing is short.

An example of a technique for satisfying (requirements A) to (requirements C) will be described below with reference to FIG.

FIG. 9 is a diagram showing an example of processing related to searching for the detection mark 420. As shown in FIG.

Here, a plurality of vertical lines 1011 to 1019 are used as detection marks 420 as shown in FIG. These vertical lines 1011 to 1019 have the same thickness, but different colors from left to right: blue, green, red, green, blue, green, red, green, and blue. This color arrangement may be arbitrary, but there are various patterns that are not included in images such as buildings, vehicles, trees, grass, the ground, and manhole walls on the ground (which do not exist on the earth, or have a very high probability of existing). must be a pattern of predetermined color combinations that are low in color).

First, in steps S901 and S902, the extraction unit 114 performs quaternarization processing to reduce the influence of the color of ambient light.

In step S<b>901 , the extraction unit 114 sets a predetermined threshold value, and sets the color of pixels with luminance equal to or lower than the predetermined threshold value to black.

In step S902, the extracting unit 114 compares pixels with brightness equal to or higher than a predetermined threshold value with three pieces of color information of RGB, and the color information of the color with the largest value has the maximum value (usually 255). Let the color information of the color be the minimum value (0).

By performing the above-described processing, the colors of pixels in each image (frame) become four colors of black, red, blue, and green. By processing in this way, the influence of ambient light can be reduced.

Next, the extraction unit 114 performs color pattern matching (step S903). First, the extraction unit 114 horizontally scans an image (frame) from left to right to read color information. When pixels of the same color are consecutive, the extraction unit 114 stores the number of consecutive pixels (the number of consecutive pixels). Further, the extraction unit 114 determines that the detection mark 420 is found when the color information is blue, green, red, green, blue, green, red, green, and blue consecutively from the left and the number of consecutive pixels is the same. I reckon. Note that the number of continuous pixels may not be exactly the same, but may have some difference (may be substantially the same).

With this processing, even if the detection mark 420 in the captured image is small because the distance from the video linking device 110 (electronic blackboard) is long (see FIG. 11), the relative positions of the video linking device 110 and the photographing device 130 are tilted. 12), the position of the detection mark 420 can be detected.

In color pattern matching, the extraction unit 114 scans the image (frame) from the top to the bottom of the screen. However, it is not limited to this process. For example, a condition that the detection mark 420 is in a predetermined area (eg, upper half) of the frame may be employed. In this case, the scanning range may be a predetermined area (for example, the upper half) of the screen, and the time required for image processing can be halved. In addition, since the detection mark 420 has a height in the vertical direction, it is possible to scan every other line, every two lines, or even every few lines without scanning all the lines. In that case, the time required for image processing can be further shortened.

By taking such processing, the above (requirement A), (requirement B), and (requirement C) can be satisfied.

In the above case, as a prerequisite, it is assumed that the image linking device 110 is at a distance of several tens of centimeters or more when the photographing device 130 captures the image. In the case of the photographing device 130, a method of displaying a two-dimensional barcode on a part of the image linking device 110 and reading it by image processing may be used.

FIG. 10 is a diagram showing an example of photographed inspection plan feature information 142 (inspection plan feature information 142 in video information 143). FIG. 10 shows an example of normal shooting from the front of the image linking device 110 .

Scan 1001 shows that detection mark 420 is detected, while scan 1002 shows that detection mark 420 is not detected. More specifically, scan 1001 has 3 pixels for blue vertical line 1011, 3 pixels for green vertical line 1012, 3 pixels for red vertical line 1013, 3 pixels for green vertical line 1014, and 3 pixels for blue vertical line. There are two combinations (rows) of 3 pixels for 1015, 3 pixels for green vertical line 1016, 3 pixels for red vertical line 1017, 3 pixels for green vertical line 1018, and 3 pixels for blue vertical line 1019. is determined and the detection mark 420 is detected. Scan 1002 determines that there is no alignment as described above, and detection mark 420 is not detected.

FIG. 11 is a diagram showing an example of the photographed inspection plan feature information 142. As shown in FIG. FIG. 11 shows an example when the image is taken from a place far from the image linking device 110 .

In scan 1101, detection mark 420 is detected. In scan 1101, vertical blue line 1111 is 1 pixel, vertical green line 1112 is 1 pixel, vertical red line 1113 is 1 pixel, vertical green line 1114 is 1 pixel, vertical blue line 1115 is 1 pixel, and green vertical line 1115 is 1 pixel. 1116 for 1 pixel, 1 pixel for red vertical line 1117, 1 pixel for green vertical line 1118, and 1 pixel for blue vertical line 1119. A mark 420 is detected.

FIG. 12 is a diagram showing an example of the photographed inspection plan feature information 142. As shown in FIG. FIG. 12 shows an example in which an image is shot while being tilted with respect to the image linking device 110 .

In scan 1201, detection mark 420 is detected. In scan 1201, vertical blue line 1211 is 4 pixels, vertical green line 1212 is 4 pixels, vertical red line 1213 is 4 pixels, vertical green line 1214 is 4 pixels, vertical blue line 1215 is 4 pixels, and green vertical line 1215 is 4 pixels. 4 pixels for the vertical line 1216 of , 4 pixels for the vertical line 1217 for red, 4 pixels for the vertical line 1218 for green, and 4 pixels for the vertical line 1219 for blue. detected.

As described above, according to the present embodiment, by specifying a record in the reference table 144, the user can easily reproduce the desired video image information 143 from the captured video information 143 in a short time. .

(2) Second Embodiment FIG. 13 is a diagram showing an example of the configuration of a video information management system 1300 according to a second embodiment. The present embodiment differs from the first embodiment mainly in that the name of the video information 143 is changed and the video playback device 120 is not provided. In this embodiment, a configuration different from that of the first embodiment is mainly described.

A video information management system 1300 is configured including a video linking device 1310 . The video linking device 1310 is basically the same as the video linking device 110 and includes an input unit 111 , a selection unit 112 , a display unit 113 , an extraction unit 114 and a linking unit 115 . The linking unit 115 may include the name changing unit 1301 , or the name changing unit 1301 may be provided separately from the linking unit 115 . In addition, the functions of the video linking device 1310 (input unit 111 to linking unit 115, name changing unit 1301) may be realized by the same computer, or input unit 111 to linking unit 115 and name changing unit 1301 may be implemented on different computers.

In the present embodiment, first, processing similar to that described in the first embodiment is performed, and the reference table 144 is obtained as a result. The name changing unit 1301 changes the names of the pieces of video information 143 based on the reference table 144 .

As described in the first embodiment, there is also a method of obtaining the corresponding extracted inspection image 123 and viewing the inspection image 146 by giving the inspection plan 145 whose image is to be confirmed. In some cases, the content may be viewed by designating (for example, double-clicking) the video information 143 within. For example, it is conceivable that the video information 143 in the folder 500 as shown in FIG. In such a case, if the name of the image information 143 is something like "CAM00123.mp4", it is difficult to efficiently select the inspection image 146 to be viewed.

Therefore, the name changing unit 1301 changes the names of the plurality of pieces of video information 143 based on the reference table 144 . For example, in the case of construction with "serial number: 0000" shown in the first line of FIG. Change to a name constructed from unique information contained in the reference table 144 . The name is information indicating the inspection target, and may be the same as the identification information or may be different from the identification information. As for how to decide the names after the change, it is possible to define the order in which the information contained in the reference table 144 is arranged. You can change the name to something more relevant and meaningful. As a result, for example, even if the computer does not have the functions of the image linking devices 110 and 1310, the contents can be understood just by looking at the name of the image information 143, so that the desired image can be efficiently selected and the contents can be viewed. be possible.

The name of the video information 143 can be changed manually, but by using this technology, the name can be assigned in a form that is completely consistent with the inspection plan list 141 in a short period of time. It is possible to reduce management problems such as duplication and loss of files due to data loss, and to reduce time loss.

As described above, according to the present embodiment, the user can view the name of the video information 143 and double-click the desired video information 143 to be shot to reproduce the video information 143 .

(3) Third Embodiment FIG. 14 is a diagram showing an example of the configuration of a video information management system 1400 according to the third embodiment. This embodiment mainly differs from the first embodiment in that the name of the video information 143 is changed. In this embodiment, a configuration different from the first embodiment and the second embodiment will be mainly described.

The video information management system 1400 includes a video linking device 1310 and a video playback device 120 . The video linking device 1310 includes an input unit 111 , a selection unit 112 , a display unit 113 , an imaging device 130 , an extraction unit 114 and a linking unit 115 . The linking unit 115 may include the name changing unit 1301 , or the name changing unit 1301 may be provided separately from the linking unit 115 . The video playback device 120 includes a designation unit 121 and a playback unit 122 . The functions of the video linking device 1310 and the functions of the video playback device 120 may be implemented by the same computer, or the functions of the video linking device 1310 and the functions of the video playback device 120 may be implemented by different computers.

This embodiment is basically the same as the first embodiment, but differs in that a name changing unit 1301 is included in the image linking device 110 . By adding this configuration, it is possible to realize a video information management system 1400 that incorporates the merits of the second embodiment. As a result, the data items (serial number 201, construction name 202, scheduled date and time 203, scheduled position 204, display coordinates 205 on the screen map, photographing time 701, GPS measurement value 702, etc.) as shown in FIG. Since it is possible to select an inspection plan and display the inspection image 146, and to give it a meaningful name such as "0000AA trunk line 1 (180101).mp4", for example, the functions of the image linking devices 110, 1300, and 1400 can be Even if the computer does not have it, the contents can be understood just by looking at the name of the image information 143, so that it is possible to efficiently select the desired image and view the contents.

Also, in order to obtain the same advantages as in the first embodiment even with a computer having the functions of the image linking devices 110, 1300, and 1400, the image information name 703 at the right end of the reference table 144 shown in FIG. It is also possible to generate a reference table 1401 in which the video information name after the change is overwritten. may be added.

As described above, according to the present embodiment, the effects of the first embodiment and the effects of the second embodiment can be obtained.

(4) Other Embodiments In the above-described embodiments, the case where the present invention is applied to a video information management system has been described, but the present invention is not limited to this, and various other systems, It can be widely applied to devices, methods and programs.

In the above-described embodiment, the detection mark 420 is used to accurately read the one-dimensional barcode 410, but the present invention is not limited to this. may be used to identify the image containing the inspection plan feature information 142, or may be used for both purposes.

In the above-described embodiment, the case where the inspection plan characteristic information 142 is photographed at the beginning or within a certain period of time from the beginning was described, but the present invention is not limited to this, and the inspection plan characteristic information 142 is photographed for a certain period of time. You may make it In this case, the detection mark 420 can be detected every several frames, and the time required for image processing can be shortened.

In the above-described embodiment, the specifying unit 121 acquires the extracted inspection image 123 from the image information 143 stored in the image linking device 110 (inside the folder), but the present invention is limited to this. First, the image information 143 may be loaded into the image reproduction device 120 at an appropriate timing, and the designation unit 121 may acquire the extracted inspection image 123 from the image information 143 stored in the image reproduction device 120 .

Also, in the above-described embodiments, the configuration of each table is an example, and one table may be divided into two or more tables, or all or part of the two or more tables may be one table. may

Further, in the above-described embodiment, for convenience of explanation, XX tables and XX files have been used to describe various types of data, but the data structure is not limited and may be expressed as XX information or the like.

In the above description, information such as programs, tables, and files that implement each function is stored in a memory, hard disk, SSD (Solid State Drive), or other storage device, or recorded on an IC card, SD card, DVD, or the like. You can put it on the medium.

In the above, as an example of the image information management system, inspection of sewage pipes was explained as an object, but the object is not limited to sewage pipes, and works plan list (for example, Inspection plan list, investigation plan list, etc.) and work (inspection, investigation) that uses images can be applied. Even if it is not a work, the present invention can be applied to any object that has a plan list of objects to be photographed in advance and can be photographed with a screen that displays the information of the plan list. It is possible.

Moreover, the present invention has, for example, the following features.

For example, a video information management system (video information management system 100, video information management system 1300, video information management system 1400, etc.), wherein the video information management system stores shooting target information related to shooting targets (inspection, investigation, etc.) (e.g., inspection information, inspection plan list 141) is input from the input unit (e.g., input unit 111), and from the imaging target information input by the input unit, the imaging target to be imaged is selected, and the selected imaging target is selected. Identification information that can uniquely identify the object (for example, unique identification information corresponding to the inspection to be performed on site in the inspection plan list 141, selected inspection plan information 116, individual image feature information 117, inspection plan feature information 142, etc.) a selection unit (e.g., selection unit 112) for selecting from the imaging target information of the imaging target, and identification information capable of uniquely identifying the imaging target selected by the selection unit is displayed on a display device (e.g., display device) and the identification information displayed on the display device is extracted from the image information (for example, the image information 143) captured by the image capturing device (for example, the image capturing device 130). an extracting unit (for example, an extracting unit 114), and the photographing object information of the photographing object identified by the identification information extracted from the video information by the extracting unit, and the specified photographing object information or the photographing object can be identified. Linking information that links part of the shooting target information with the video information (for example, the video information name created based on the information of the reference table 144 and the inspection plan list 141, the display on the drawing map and a linking unit (eg, linking unit 115) that generates and outputs information in which the coordinates 205 and the image information 143 are associated with each other.

According to the above configuration, since the linking information in which the shooting target and the video information are linked is output, for example, the user can accurately grasp the video information of the desired shooting target from the video information captured in large quantities. , the contents can be checked in a short time. As a result, the amount of work can be reduced, work efficiency can be improved, and more time can be allocated to other work.

Further, for example, the linking unit generates the linking information (for example, the reference table 144) by adding information (for example, a video information name 703) that can specify the video information to the shooting target information. It is characterized by

According to the above configuration, for example, the user can specify the video information of the desired shooting target from the information that can specify the video information added to the shooting target information.

Also, for example, the linking unit (for example, the name changing unit 1301) changes the name of the video information to information indicating the shooting target (for example, a name composed of unique information included in the reference table 144). By doing so, the linking information is generated.

According to the above configuration, for example, the user can reproduce the video information of the desired shooting target by double-clicking the name of the video information in the folder.

Further, for example, the associating unit generates the associating information by associating information of a graphical user interface (for example, display coordinates 205 on a drawing map) capable of designating the imaging target with the image information. A designation unit (for example, a designation unit 121) that designates a shooting target via a graphical user interface, and a reproduction unit (for example, a reproduction unit 122) that reproduces video information associated with the photography target designated by the designation unit. ), and

According to the above configuration, for example, the user can reproduce video information of a desired imaging target by double-clicking a sewage pipe on the map displayed as a graphical user interface.

Further, for example, the display unit displays pattern information (for example, detection mark 420) indicating a pattern used for detecting the identification information together with the identification information.

According to the above configuration, for example, since the image of the video information captured by the imaging device includes the pattern information together with the identification information, the extraction unit can detect the pattern information from the image by uniform processing. It is possible to extract individual identification information for each photographed object from the image.

In addition, for example, the pattern is a combination of predetermined colors that do not exist on the earth or have a very low probability of existing on the earth (for example, blue, green, red, green, blue, green, red, green, blue ), and the extraction unit extracts the identification information from an image including the predetermined color combination.

According to the above configuration, for example, even if the background is included, the extraction unit can accurately detect the pattern, so that the identification information can be extracted with high accuracy.

Further, for example, the extracting unit determines that the identification information is included when determining that the number of pixels in which each color is continuous in the predetermined combination of colors is the same or substantially the same in the image of the video information. characterized in that

According to the above configuration, for example, the extracting unit can detect the distance between the display device and the imaging device at the time of imaging, even when the display device is not photographed from the front, and when both are combined. However, patterns can be detected.

Further, for example, the identification information includes a barcode (for example, one-dimensional barcode 410) that can uniquely identify the object to be imaged, and/or any arbitrary combination of letters, numbers, and symbols that can uniquely identify the object to be imaged. (for example, information 430 that can uniquely identify the inspection object) is included.

According to the above configuration, for example, appropriate identification information can be adopted according to the video information management system.

Moreover, the above-described configurations may be appropriately changed, rearranged, combined, or omitted within the scope of the present invention.

According to the configuration described above, it is possible to appropriately manage video information.

100: video information management system, 110: video linking device, 120: video reproducing device.

Claims (9)

an input unit for inputting shooting target information related to the shooting target;
a selection unit that selects a shooting target to be shot from the shooting target information input by the input unit, and selects identification information that can uniquely identify the selected shooting target from the shooting target information of the shooting target;
a display unit for displaying, on a display device, identification information capable of uniquely identifying an imaging target selected by the selection unit;
an extracting unit for extracting the identification information displayed on the display device from video information captured by an imaging device;
specifying the shooting target information of the shooting target identified by the identification information extracted from the video information by the extraction unit, and specifying the specified shooting target information or part of the shooting target information capable of identifying the shooting target; a linking unit that generates and outputs linking information linked with video information;
A video information management system comprising: The linking unit generates the linking information by adding information that can specify the video information to the shooting target information.
The video information management system according to claim 1, characterized by: The linking unit generates the linking information by changing the name of the video information to information indicating the shooting target.
The video information management system according to claim 1, characterized by: The associating unit generates the associating information by associating information of a graphical user interface capable of designating the imaging target with the image information,
a designation unit that designates an imaging target via the graphical user interface;
a reproducing unit that reproduces video information associated with the shooting target specified by the specifying unit;
comprising
The video information management system according to claim 1, characterized by: The display unit displays pattern information indicating a pattern used for detecting the identification information together with the identification information.
The video information management system according to claim 1, characterized by: The pattern is a pattern of a predetermined color combination that does not exist on earth or has a very low probability of existing on earth;
The extraction unit extracts the identification information from an image including the predetermined color combination.
6. The video information management system according to claim 5, characterized in that: If the extracting unit determines that the number of pixels in which each color is continuous in the predetermined color combination is the same or substantially the same in the image of the video information, it determines that the identification information is included.
7. The video information management system according to claim 6, characterized by: The identification information includes a barcode that can uniquely identify the object to be photographed, and/or any combination of letters, numbers, and symbols that can uniquely identify the object to be photographed.
The video information management system according to claim 1, characterized by: a first step in which the input unit inputs imaging target information relating to the imaging target;
A selection unit selects a shooting target to be shot from the shooting target information input by the input unit, and selects identification information capable of uniquely identifying the selected shooting target from the shooting target information of the shooting target. 2 steps;
a third step in which the display unit displays identification information that can uniquely identify the imaging target selected by the selection unit on the display device;
a fourth step in which the extraction unit extracts the identification information displayed on the display device from video information captured by an imaging device;
A linking unit specifies shooting target information of a shooting target identified by the identification information extracted from the video information by the extracting unit, and the specified shooting target information or a part of the shooting target information with which the shooting target can be identified. a fifth step of generating and outputting linked information linking the information of and the video information;
A video information management method comprising:

Images