WO2014091736A1

WO2014091736A1 - Display device for panoramically expanded image

Info

Publication number: WO2014091736A1
Application number: PCT/JP2013/007209
Authority: WO
Inventors: 偉志渡邊; 藤井　博文; 藤松　健; 森村　淳
Original assignee: パナソニック株式会社
Priority date: 2012-12-10
Filing date: 2013-12-06
Publication date: 2014-06-19

Abstract

The present invention is provided with: an omnidirectional-camera-image input unit (10) that inputs an image from an omnidirectional camera; a path information input unit (11) that obtains the path of a subject captured in the image from the omnidirectional camera; a camera central-position input unit (12) that inputs the central position of the omnidirectional camera; an expansion reference position determining unit (13) that, from the inputted camera central position and path information, sets an expansion reference position for panoramically expanding the image from the camera; an omnidirectional-camera-image panoramic expansion unit (14) that panoramically expands the image from the camera by cutting the image on the basis of the set expansion reference position; and a display unit (15) that displays the panoramically expanded image.

Description

Panorama development image display device

The present invention relates to a panoramic unfolded image display device that unfolds and displays an image captured by an omnidirectional camera.

As the above-described panoramic development image display device, for example, the one described in Patent Document 1 is known. The panoramic developed image imaging device described in Patent Document 1 forms a circular image by a fisheye lens on a light-receiving surface on which a fisheye lens and a plurality of light-receiving elements are arranged, and generates captured image data having a circular image. Imaging means, memory for storing the position or size of the inner and outer circle boundaries that define the range of the ring shape used to generate a 360-degree panoramic unfolded image in the captured image, and the above based on the user's operation User update means for updating the position or size of at least one of the inner circle boundary and the outer circle boundary stored in the memory, and the inner information stored in the memory in the captured image generated by the imaging means Panorama expanded image generating means for generating a 360 ° panoramic expanded image based on an image within a ring-shaped range sandwiched between a circular boundary and the outer circular boundary. , Comprising a.

FIG. 16 is a diagram illustrating an example of a camera image 100 taken by an omnidirectional camera and a panoramic development image 101 obtained by panoramic development of the camera image 100.

Japanese Unexamined Patent Publication No. 2008-028606

However, when panoramic development is performed on a camera image taken with an omnidirectional camera, depending on the installation location of the omnidirectional camera and a reference position for panoramic development from the camera image (hereinafter referred to as “development reference position”) There is a problem that it is difficult to easily understand the situation of the place where the omnidirectional camera is installed (that is, it is difficult to link the panoramic image and the real world).

This problem will be explained with an example. FIG. 17 shows a camera image 100 ((a) in the figure) obtained by photographing the inside of the convenience store with an omnidirectional camera, and a panoramic developed image 101 ((b) in the figure) obtained by panoramic developing the camera image 100. FIG. 4 is a diagram showing an arrangement of actual articles in a convenience store ((c) in the figure). The panoramic development image 101 is a panoramic development of the camera image 100 based on the development reference line 110 shown in FIG. In addition, the flow line 120-1 in the panorama development image 101 corresponds to the passage 130-1 in the convenience store, the flow line 120-2 corresponds to the passage 130-2 in the convenience store, and the flow line 120-3 It corresponds to the passage 130-3 in the convenience store. With the fixed development reference line 110, the passage may be divided (the flow line 120-2 is divided) in the panoramic development image 101 as in the passage 130-2 in FIG. 17, and an omnidirectional camera is installed. It becomes difficult to understand the situation of the place.

The present invention has been made in view of such circumstances, and provides a panoramic unfolded image display device that can obtain a panoramic unfolded image from which a camera installation location can be easily understood from a camera image taken by an omnidirectional camera. The purpose is to do.

The panoramic unfolded image display device of the present invention includes an image input means for inputting an omnidirectional image, a path acquisition means for obtaining a path of an object photographed in the omnidirectional image, and the omnidirectional image from a predetermined center position. Panorama development means for developing a panorama by cutting along a cutting line in the circumferential direction; and display means for displaying the panorama developed image, wherein the intersection of the passage with the passage is a predetermined value Set to be less than the value.

According to the above configuration, the cutting line (development reference line) for panoramic development of the omnidirectional image is set so that the intersection with the passage is smaller than a predetermined value. The passage is not divided, and a panoramic development image that allows easy understanding of the situation of the camera installation location can be obtained.

In the above configuration, the cutting line is set so that the passage perpendicular to the passage is larger than a predetermined value.

In the above configuration, the omnidirectional image is an image taken by an omnidirectional camera, and the predetermined center position is a position in the image taken immediately below the omnidirectional camera.

In the above configuration, the information on the passage is generated from the flow line by obtaining the flow line of the object in the omnidirectional image.

In the above configuration, the vehicle has a congestion degree calculating means for calculating a congestion degree for each passage, and the cutting line is set so as to cross a passage having the congestion degree less than a predetermined value.

According to the above configuration, the degree of congestion for each passage is analyzed, and the cutting line is set so that the passage with a high degree of congestion is prioritized, so that a busy passage can be clearly seen.

In the above configuration, the congestion degree calculating means calculates the congestion degree for each time zone, and the cutting line is set for each time zone according to the congestion degree for each time zone.

According to the above configuration, since the cutting line is changed for each time zone depending on the degree of congestion for each passage, it is possible to clearly see the congested passage for each time zone.

The panoramic developed image output device of the present invention includes an image input means for inputting an omnidirectional image, a path acquisition means for obtaining a path of an object photographed in the omnidirectional image, and the omnidirectional image from a predetermined center position. Panorama developing means for outputting the panorama by cutting along a cutting line in the circumferential direction, and the cutting line is set so that the intersection with the passage is less than a predetermined value.

The panoramic developed image display method of the present invention includes an image input step of inputting an omnidirectional image, a passage acquisition step of obtaining a passage of an object photographed in the omnidirectional image, and the omnidirectional image from a predetermined center position. A panoramic development step for panoramic development by cutting along a cutting line in the circumferential direction; and a display step for displaying the panoramic development image. The cutting line has a predetermined intersection with the passage. Set to be less than the value.

According to the present invention, it is possible to obtain a panoramic development image that can easily understand the situation of the camera installation location from a camera image taken by an omnidirectional camera.

1 is a block diagram showing a schematic configuration of a panoramic developed image display apparatus according to Embodiment 1 of the present invention. The figure which shows an example of a camera image when an omnidirectional camera is on a passage (A)-(c) The figure which shows an example of a camera image when an omnidirectional camera is not on a passage The flowchart which shows operation | movement of the panoramic expansion image display apparatus of FIG. The figure which shows the expansion | deployment reference line determined by the expansion | deployment reference position determination part of the panoramic expansion | deployment image display apparatus of FIG. The figure which shows the expansion | deployment reference line determined by the expansion | deployment reference position determination part of the panoramic expansion | deployment image display apparatus of FIG. (A), (b) The figure which shows the comparative example with the case where a development reference position is set automatically, and the case where a development reference position is fixed FIG. 3 is a block diagram showing a schematic configuration of a panoramic developed image display apparatus according to Embodiment 2 of the present invention. (A), (b) The figure which shows an example of the passage information estimated by the person flow line information of N frames accumulate | stored in the human flow line information log | history storage part of the panoramic expansion image display apparatus of FIG. 8, and a passage position estimation part The flowchart which shows operation | movement of the panoramic expansion image display apparatus of FIG. FIG. 3 is a block diagram showing a schematic configuration of a panoramic developed image display apparatus according to Embodiment 3 of the present invention. The flowchart which shows operation | movement of the panoramic expansion image display apparatus of FIG. FIG. 5 is a block diagram showing a schematic configuration of a panoramic developed image display apparatus according to Embodiment 4 of the present invention. The flowchart which shows operation | movement of the panoramic expansion image display apparatus of FIG. The congestion degree average table for each passage stored in the congestion degree information history storage unit of the panoramic developed image display device of FIG. 13 and 10: 00-11: 0 and 12: 00-13: 00 in the congestion degree average table. The figure which shows an example of the channel | path on the camera image in each The figure which shows an example of the original image which the omnidirectional camera image | photographed, and the panorama expansion | deployment image which carried out the panorama expansion | deployment of this original image (A)-(c) The figure which shows arrangement | positioning of the original image obtained by image | photographing the inside of a convenience store with an omnidirectional camera, the panorama expansion | deployment image which expand | deployed the original image, and the actual goods in a convenience store The block diagram which shows schematic structure of Embodiment 5 of this invention. The block diagram which shows schematic structure of Embodiment 6 of this invention.

Hereinafter, preferred embodiments for carrying out the present invention will be described in detail with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram showing a schematic configuration of a panoramic developed image display apparatus according to Embodiment 1 of the present invention. In the figure, a panoramic developed image display device 1 according to the present embodiment includes an omnidirectional camera image input unit (image input unit) 10, a passage information input unit (passage acquisition unit) 11, and a camera center position input unit 12. A development reference position determination unit 13, an omnidirectional camera image panorama development unit (panorama development unit) 14, and a display unit (display unit) 15.

The omnidirectional camera image input unit 10 inputs an image of an omnidirectional camera (not shown) (hereinafter referred to as “camera image”). The passage information input unit 11 inputs information about a passage in the building (hereinafter referred to as “passage information”) from the camera image. For example, if the building is a convenience store and an omnidirectional camera is installed in the convenience store, the passage information in the convenience store is input. In the present embodiment, the passage information is manually input by the user himself / herself. The camera center position input unit 12 inputs a camera center position in a camera image obtained by photographing a omnidirectional camera. The user inputs the camera center position manually. The development reference position determination unit 13 sets a development reference position for panoramic development of the camera image from the camera center position input by the camera center position input unit 12 and the path information input by the path information input unit 11. In this case, the deployment reference position is set in a direction orthogonal to the passage if the omnidirectional camera is on the passage and is not an intersection of the passages, and if the omnidirectional camera is not on the passage, the development reference position is all around the camera position. An azimuth scan is performed and the line of intersection with the passage is set to be minimum and perpendicular.

FIG. 2 is a diagram illustrating an example of the camera image 100 when the omnidirectional camera is on the passage 130. As shown in the figure, when the camera position 150 is on the passage 130, the development reference line 110 indicating the development reference position is set in a direction orthogonal to the passage 130 at a location other than the intersection 160 of the passage. FIGS. 3A to 3C are views showing an example of the camera image 100 when the omnidirectional camera is not on the passage 130. FIG. As shown in the figure, when the camera position 150 is not on the passage 130, the development reference line 110 indicating the development reference position is set so that the line of intersection with the passage 130 is the smallest and perpendicular.

Returning to FIG. 1, the omnidirectional camera image panorama developing unit 14 develops a panorama by cutting the camera image based on the development reference position set by the development reference position determining unit 13. That is, the camera image is panoramicly developed by cutting along the development reference line in the circumferential direction from the center position centered on the camera position. The display unit 15 displays the image panorama developed by the omnidirectional camera image panorama developing unit 14.

Next, the operation of the panoramic developed image display device 1 according to the present embodiment will be described.
FIG. 4 is a flowchart showing the operation of the panoramic developed image display device 1 according to the present embodiment. In the figure, the omnidirectional camera image input unit 10 first inputs a camera image (step S1). Next, the camera center position input unit 12 inputs the camera center position designated by the user for the camera image (step S2). Simultaneously with the camera center position input, the passage information input unit 11 inputs passage information from an image of one frame (step S3). After the camera center position and passage information are input, the development reference position determination unit 13 determines whether the camera center position exists on the passage (step S4), and determines that the camera center position exists on the passage. (In other words, if “Yes” is determined), it is determined whether or not the camera center position exists at the passage intersection (step S5).

The deployment reference position determination unit 13 sets the deployment reference position in a direction orthogonal to the passage when it is determined that the camera center position does not exist at the passage intersection (that is, when “No” is determined) (step S6). FIG. 5 is a diagram showing a development reference line 110 on the development reference position when the camera center position is on a passage other than the passage intersection. When the development reference position is set, the omnidirectional camera image panorama development unit 14 develops a panoramic image of 360 degrees based on the development reference position and displays it on the display unit 15 (step S7). After the panorama development image for one frame of the camera image is displayed, the omnidirectional camera image panorama development unit 14 determines whether or not the camera image has been captured (step S8), and the camera image capture has not been completed. (That is, when “No” is determined), the camera image of the next frame is captured (step S9), and the camera image is panorama developed and displayed on the display unit 15. The omnidirectional camera image panorama developing unit 14 ends this process when it is determined that the capture of the camera image has been completed (that is, when it is determined “Yes”).

On the other hand, the development reference position determination unit 13 determines in step S4 that the camera center position does not exist on the passage (that is, determines “No”) or in step S5. When it is determined that the camera center position exists at the intersection (that is, when “Yes” is determined), a straight line (that is, a development reference line) is set while shifting the angle from the camera center position toward the camera image end by α. (Step S10). The development reference position determination unit 13 shifts the angle α from the camera center position toward the camera image end, thereby minimizing the number of intersection points between the development reference line and the passage and maximizing the number of intersection points at right angles. Is set (step S11). FIG. 6 is a diagram showing the development reference line 110 when the angle is shifted by α when the camera center position is not on the passage. Returning to FIG. 4, after setting the development reference position, the process proceeds to step S7. Thereafter, the above-described steps S7 to S9 are performed.

FIGS. 7A and 7B are diagrams showing a comparative example between the case where the development reference position is automatically set (the present invention) and the case where the development reference position is fixed (conventional). a) shows the camera image 100 and panorama development image 101 when the development reference position is fixed, and (b) shows the camera image 100 and panorama development image 101 when the development reference position is automatically set. As can be seen from the figure, the passage is divided when the development reference position is fixed, but the passage is not divided when the development reference position is automatically set.

As described above, the panoramic developed image display device 1 according to the present embodiment sets the development reference line for panoramic development of the camera image so that the intersection with the passage is smaller than a predetermined value, so that the passage is divided. Therefore, it is possible to obtain a panoramic developed image that can easily understand the situation of the camera installation location.

(Embodiment 2)
FIG. 8 is a block diagram showing a schematic configuration of a panoramic developed image display apparatus according to Embodiment 2 of the present invention. In the figure, the same reference numerals are given to the portions common to FIG. 1 described above. The panoramic unfolded image display device 2 according to the present embodiment has a function of estimating a passage position by analyzing a flow line (person's movement trajectory) using a person tracking technique, and an omnidirectional camera image input unit 10. A camera center position input unit 12, an omnidirectional camera image panorama development unit 14, a display unit 15, a person flow line information acquisition unit 16, a person flow line information history storage unit 17, and a passage position estimation unit 18. And a development reference position determining unit 19.

The person flow line information acquisition unit 16 acquires the person flow line information by obtaining the flow line of the person in the camera image. The person flow line information acquisition unit 16 acquires and outputs the person flow line information for each frame. The person flow line information history storage unit 17 accumulates the person flow line information acquired for each frame by the person flow line information acquisition unit 16. The passage position estimation unit 18 estimates the passage position from the person flow line information for each frame accumulated in the person flow line information history storage unit 17.

FIGS. 9A and 9B are diagrams illustrating an example of N-frame human flow information accumulated in the human flow information history storage unit 17 and passage information estimated from the N-flow human flow information. Yes (N: natural number), (a) in the figure is a camera image in which the flow line 120 is accumulated for N frames, and (b) is an estimation result of passage information.

Returning to FIG. 8, the development reference position determination unit 19 determines a development reference position for panoramic development of the camera image from the camera center position input by the camera center position input unit 12 and the passage position estimated by the passage position estimation unit 18. Set. The omnidirectional camera image panorama developing unit 14 develops a panorama by cutting the camera image based on the development reference position set by the development reference position determining unit 19. The display unit 15 displays the image panorama developed by the omnidirectional camera image panorama developing unit 14.

Next, the operation of the panoramic developed image display device 2 according to the present embodiment will be described.
FIG. 10 is a flowchart showing the operation of the panoramic developed image display device 2 according to the present embodiment. In the figure, the omnidirectional camera image input unit 10 first inputs a camera image (step S20). Next, the camera center position input unit 12 inputs the camera center position designated by the user for the camera image (step S21). Simultaneously with the input of the camera center position, the human flow line information acquisition unit 16 acquires human flow line information from one frame image (step S22). Next, the person flow line information history storage unit 17 accumulates the person flow line information acquired by the person flow line information acquisition unit 16 (step S23). When the human flow line information of N frames is accumulated in the human flow line information history storage unit 17, the passage position estimation unit 18 integrates the human flow line information of the past N frames (step S24), and then the integrated person The passage information is estimated based on the flow line information (step S25).

After the passage information is estimated by the passage position estimation unit 18, the development reference position determination unit 19 sets a development reference position for panoramic development of the camera image (step S26). Since this process (Process A) is the same as Steps S4 to S6 and Steps S10 and S11 of FIG. 4 described above, description thereof is omitted. After the development reference position is set, the omnidirectional camera image panorama development unit 14 develops the camera image and displays it on the display unit 15 (step S27). Note that the one-frame image used by the omnidirectional camera image panorama developing unit 14 is the same as the one-frame image first captured by the human flow line information acquiring unit 16. After the panorama development image for one frame of the camera image is displayed, the omnidirectional camera image panorama development unit 14 determines whether or not the camera image has been captured (step S28), and the camera image capture has not been completed. (That is, when “No” is determined), the camera image of the next frame is captured (step S29), and the camera image is panorama developed and displayed on the display unit 15. The omnidirectional camera image panorama developing unit 14 ends this process when it is determined that the capture of the camera image has been completed (that is, when it is determined “Yes”). In addition, when updating person flow line information for every frame, while acquiring the camera image of the next frame, the process from step S22 is repeated.

As described above, the panoramic developed image display device 2 according to the present embodiment acquires the person flow line information in the camera image and estimates the passage information from the acquired person flow line information. As a result, the operability can be improved.

(Embodiment 3)
FIG. 11 is a block diagram showing a schematic configuration of a panoramic developed image display apparatus according to Embodiment 3 of the present invention. In the figure, the same reference numerals are assigned to the portions common to those in FIGS. 1 and 8 described above. The panorama development image display device 3 according to the present embodiment analyzes the passage congestion degree from the estimated passage position in addition to the functions of the panorama development image display device 2 described above, and gives priority to a passage with a high degree of congestion. The omnidirectional camera image input unit 10, the camera center position input unit 12, the omnidirectional camera image panorama development unit 14, the display unit 15, and the human motion. A line information acquisition unit 16, a person flow line information history storage unit 17, a passage position estimation unit 18, a passage congestion degree calculation unit (congestion degree calculation means) 20, and a development reference position determination unit 21 are provided.

The degree of congestion is, for example, the ratio of the number of people who have actually passed to the maximum number of people passing in a unit time for each passage, expressed as a percentage. By calculating the degree of congestion for each passage, it is possible to set and display a development reference position so that a passage with a high degree of congestion is prioritized and is not interrupted. Note that the degree of congestion is described in detail in the prior art documents (for example, JP 2009-110152 A and JP 2009-110054 A), and thus description thereof is omitted in this specification.

Next, the operation of the panoramic developed image display device 3 according to the present embodiment will be described.
FIG. 12 is a flowchart showing the operation of the panoramic developed image display device 3 according to the present embodiment. In the figure, the omnidirectional camera image input unit 10 first inputs a camera image (step S30). Next, the camera center position input unit 12 inputs the camera center position designated by the user for the camera image (step S31). Simultaneously with the input of the camera center position, the human flow line information acquisition unit 16 acquires human flow line information from one frame image (step S32). The person flow line information history storage unit 17 accumulates the person flow line information acquired by the person flow line information acquisition unit 16 (step S33). When N frames of person flow line information are accumulated in the person flow line information history storage unit 17, the passage position estimation unit 18 integrates the person N flow line information of the past N frames (step S34), and further integrates the person. The passage information is estimated based on the flow line information (step S35).

After the passage information is estimated by the passage position estimation unit 18, the passage congestion degree calculation unit 20 calculates the degree of congestion for each passage (step S36). After the congestion degree is calculated for each passage, the development reference position determination unit 21 determines whether or not the camera center position exists on the passage (step S37), and determines that the camera center position exists on the passage ( That is, if “Yes” is determined), it is determined whether or not the camera center position exists at the passage intersection (step S38).

When the development reference position determination unit 21 determines that the camera center position does not exist at the passage intersection (that is, when it is determined “No”), the development reference position determination unit 21 sets the development reference position in a direction orthogonal to the passage (step S39). When the development reference position is set, the omnidirectional camera image panorama development unit 14 develops the camera image on the basis of the development reference position set by the development reference position determination unit 21 and displays it on the display unit 15 (step S1). S40). After the panorama development image for one frame of the camera image is displayed, the omnidirectional camera image panorama development unit 14 determines whether or not the camera image has been captured (step S41), and the camera image capture has not been completed. (That is, when “No” is determined), the camera image of the next frame is captured (step S42), and the process returns to step S32. The omnidirectional camera image panorama developing unit 14 ends this process when it is determined that the capture of the camera image has been completed (that is, when it is determined “Yes”).

On the other hand, the development reference position determining unit 21 determines in step S37 that the camera center position does not exist on the passage (that is, determines “No”) or in step S38. When it is determined that the camera center position exists at the intersection (that is, when “Yes” is determined), a straight line (that is, a development reference line) is set while shifting the angle from the camera center position toward the image edge by α ( Step S43). The development reference position determination unit 21 shifts the angle from the camera center position toward the image edge by α, thereby minimizing the number of intersections between the straight line and the passage, and maximizing the number of intersections at right angles, and the degree of congestion of the intersecting passage. The development reference position is set so that the average is minimized (step S44). After setting the development reference position, the process proceeds to step S40. Thereafter, the processes of steps S40 to S42 described above are performed.

As described above, the panoramic developed image display device 3 according to the present embodiment analyzes the degree of congestion for each passage and sets the development reference position so that the passage with high degree of congestion is prioritized and is not interrupted. You can clearly see the passage that is.

(Embodiment 4)
FIG. 13 is a block diagram showing a schematic configuration of a panoramic developed image display apparatus according to Embodiment 4 of the present invention. In the figure, the same reference numerals are given to the portions common to each of FIGS. 1, 8, and 11 described above. The panoramic developed image display device 4 according to the present embodiment has a function of changing the developed reference position for each time zone depending on the degree of congestion in addition to the functions of the panoramic developed image display device 3 described above. Camera image input unit 10, camera center position input unit 12, omnidirectional camera image panorama development unit 14, display unit 15, person flow line information acquisition unit 16, person flow line information history storage unit 17, passage The position estimation unit 18, the passage congestion degree calculation unit 20, the development reference position determination unit 21, and the congestion degree information history storage unit 22 are provided.

Next, the operation of the panoramic developed image display device 4 according to the present embodiment will be described.
FIG. 14 is a flowchart showing the operation of the panoramic developed image display device 4 according to the present embodiment. In the figure, first, the omnidirectional camera image input unit 10 inputs a camera image (step S50). Next, the camera center position input unit 12 inputs the camera center position designated by the user for the camera image (step S51). Simultaneously with the camera center position input, the human flow line information acquisition unit 16 acquires human flow line information from an image of one frame (step S52). Then, the person flow line information history storage unit 17 accumulates the person flow line information acquired by the person flow line information acquisition unit 16 (step S53). When N frames of person flow line information are accumulated in the person flow line information history storage unit 17, the passage position estimation unit 18 integrates the person N flow line information of the past N frames (step S54), and further, the integrated person The passage information is estimated based on the flow line information (step S55).

After the passage information is estimated, the passage congestion degree calculation unit 20 calculates the degree of congestion for each passage (step S56), and further acquires time information (step S57). Then, the congestion degree average table for each passage stored in the congestion degree information history storage unit 22 is updated (step S58). FIG. 15 is a diagram illustrating an example of the congestion degree average table for each passage and the paths on the camera image at 10: 00-11: 0 and 12: 00-13: 00 in the congestion degree average table. The passage 130 has a thickness corresponding to the degree of congestion. The thicker, the higher the congestion level.

Returning to FIG. 14, after the congestion degree average table for each passage is updated, the development reference position determination unit 21 determines whether or not the development position is updated (step S <b> 59), and when it is determined that the development position is updated (that is, If “Yes” is determined), the congestion degree average table stored in the congestion degree information history storage unit 22 is referred to and the congestion degree is set for each passage (step S60). Next, a development reference position is set (step S61). This process (Process B) is the same as Steps S37 to S39 and Steps S43 and S44 of FIG. After the development reference position is set, the omnidirectional camera image panorama development unit 14 develops the camera image and displays it on the display unit 15 (step S62). Note that the one-frame image used by the omnidirectional camera image panorama developing unit 14 is the same as the one-frame image first captured by the human flow line information acquiring unit 16. After the panorama development image for one frame of the camera image is displayed, the omnidirectional camera image panorama development unit 14 determines whether or not the camera image has been captured (step S63), and the camera image capture has not been completed. (That is, when “No” is determined), the camera image of the next frame is captured (step S64), and the process returns to step S52. The omnidirectional camera image panorama developing unit 14 ends this process when it is determined that the capture of the camera image has been completed (that is, when it is determined “Yes”).

On the other hand, if it is determined in step S59 that the development reference position determination unit 21 is not updating the development position (ie, “No” is determined), the previous development reference position is held (step S65), and step S62 is performed. Move on to processing.

As described above, the panoramic developed image display device 4 according to the present embodiment changes the development reference position for each time zone depending on the degree of congestion for each passage, so that a congested passage for each time zone can be clearly seen. it can.

(Embodiment 5)
FIG. 18 is a block diagram showing a schematic configuration of the fifth embodiment of the present invention. In the figure, the fifth embodiment includes an omnidirectional camera 181, an input unit 182, a processing unit 183, and a display unit 184. The omnidirectional camera 181 may be a camera that inputs an omnidirectional image, the input unit 182 may be an input device such as a keyboard, the processing unit 183 may be a personal computer, and the display unit 184 may be a liquid crystal display.

Embodiment 5 is a more specific implementation means of Embodiment 1 of the present invention. In relation to the first embodiment, the omnidirectional camera image input unit 10 of the first embodiment is an omnidirectional camera 181, the camera center position input unit 12 and the passage information input unit 11 are input units 182, and a development reference. The position determining unit 13 and the omnidirectional camera image panorama developing unit 14 are mounted in the processing unit 183, and the display unit 15 corresponds to the display unit 184.

(Embodiment 6)
FIG. 19 is a block diagram showing a schematic configuration of the sixth embodiment of the present invention. In the sixth embodiment, among the functions of the second embodiment, the functions other than the display unit 15 are realized in the omnidirectional camera as they are except for some functions.
In FIG. 19, the same components as those in FIG. 8 showing the second embodiment are given the same numbers. 19, the omnidirectional camera 191 includes an omnidirectional camera image input unit 10, a camera center position holding unit 192, a person flow line information acquisition unit 16, a person flow line information history storage unit 17, a passage position estimation unit 18, and a deployment. A reference position determining unit 19 and an omnidirectional camera image panorama developing unit 14 are incorporated.

In the sixth embodiment, the camera center position described in the first embodiment is held in advance. Other functions are the same as those of the second embodiment, and the panoramic image developed by the omnidirectional camera image panorama developing unit 14 is output from the omnidirectional camera 191.

The output panoramic image may be displayed on a liquid crystal display or the like connected to the omnidirectional camera 191, or may be displayed on a liquid crystal display at a location away from the omnidirectional camera 191 via a network, or may be stored as an image. It may be stored in a functioning storage device.

Although the present invention has been described in detail and with reference to specific embodiments, it will be apparent to those skilled in the art that various changes and modifications can be made without departing from the spirit and scope of the invention.

This application is based on a Japanese patent application (Japanese Patent Application No. 2012-269256) filed on Dec. 10, 2012, the contents of which are incorporated herein by reference.

The present invention has an effect that it is possible to obtain a panoramic development image from which a camera installation location can be easily understood from a camera image photographed by an omnidirectional camera. Application to the device is possible.

1, 2, 3, 4 Panorama development image display device 10 Omnidirectional camera image input unit 11 Path information input unit 12 Camera center position input unit 13 Development reference position determination unit 14 Omnidirectional camera image panorama development unit 15 Display unit 16 Human motion Line information acquisition unit 17 Person flow line information history storage unit 18 Passage

position estimation unit

19, 21 Development reference position determination unit 20 Passage congestion degree calculation unit 22 Congestion degree information history storage unit 100 Camera image 101 Panorama development image 110 Development reference line 130 Passage 150 Camera position 160 Intersection 181 Omnidirectional camera 182 Input unit 183 Processing unit 184 Display unit 191 Omnidirectional camera 192 Camera center position holding unit

Claims

An image input means for inputting an omnidirectional image;
Passage acquisition means for obtaining a passage of an object photographed in the omnidirectional image;
Panoramic development means for developing a panorama by cutting the omnidirectional image along a cutting line in a circumferential direction from a predetermined center position;
Display means for displaying the panoramic image,
The panoramic developed image display device characterized in that the cutting line is set so that the intersection with the passage is less than a predetermined value.
2. The panoramic developed image display device according to claim 1, wherein the cutting line is set so that the passage perpendicular to the passage is larger than a predetermined value.
The omnidirectional image is an image taken by an omnidirectional camera, and the predetermined center position is a position in the image taken immediately below the omnidirectional camera. Item 3. The panoramic expanded image display device according to Item 2.
4. The panoramic unfolded image display device according to claim 1, wherein the information on the passage is generated from the flow line obtained from the flow line of the object in the omnidirectional image. 5. .
Congestion degree calculating means for calculating the degree of congestion for each passage,
5. The panoramic unfolded image display device according to claim 1, wherein the cutting line is set so as to cross a passage where the degree of congestion is less than a predetermined value.
The congestion degree calculating means calculates a congestion degree for each time zone,
6. The panoramic unfolded image display device according to claim 5, wherein the cutting line is set for each time period according to the degree of congestion for each time period.
An image input means for inputting an omnidirectional image;
Passage acquisition means for obtaining a passage of an object photographed in the omnidirectional image;
Panorama developing means for outputting a panoramic image by cutting the omnidirectional image along a cutting line in a circumferential direction from a predetermined center position;
The panoramic developed image output device, wherein the cutting line is set so that an intersection with the passage is smaller than a predetermined value.
An image input step for inputting an omnidirectional image;
A passage obtaining step for obtaining a passage of an object photographed in the omnidirectional image;
A panoramic development step for panoramic development by cutting the omnidirectional image along a cutting line in a circumferential direction from a predetermined center position;
A display step of displaying the panoramic image.
The panoramic unfolded image display method, wherein the cutting line is set so that the intersection with the passage is less than a predetermined value.