JP2010130190A - Video processing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reproduce only the video images of one desired certain direction from the video images of a plurality of directions, without having to use a special camera. <P>SOLUTION: The video processing system includes a camera 10 for photographing video images and outputting corresponding video signals 9; a motor 50 for changing the attitude of the camera 10 so as to switch a photographic direction to the plurality of directions; a frame storage part 21 for storing a plurality of frames 9 acquired in the plurality of directions and included in the video signals from the camera 10; a frame extracting part 23 for extracting the frame 11, corresponding to a prescribed direction from the plurality of stored frames 9; and a display device 30 capable of displaying the frame 11 extracted in the frame extracting part 23. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a video processing system that processes, for example, a video obtained by photographing the surroundings.

  For example, a technique is known in the art that rotates a monitoring camera to periodically photograph the surroundings, reproduces the captured image on a display device, and monitors the occurrence of an abnormality such as the presence or absence of an intruder. . When viewing images taken by rotating the camera in this way, it is good to see if there are any abnormalities in all directions, but when you are concerned about the image in one direction, you should see the image only in that direction. I can't. If you try to view a video in one direction from a video that rotates, images in other directions are sandwiched between each rotation, so you have to see images in other directions one by one. Time-series changes are difficult to capture.

Corresponding to this, a technique described in Patent Document 1 for selecting an arbitrary direction from videos in a plurality of directions and continuously displaying the videos in the directions is known. In this prior art, one omnidirectional camera provided with a plurality of video capturing means (lenses) is prepared, and images are taken in a plurality of directions at the same time. Only one unidirectional video image can be selected from the captured video images in a plurality of directions.
JP-A-8-289179

  In the above-described prior art, only a desired one-way image can be reproduced by selecting only one-way image from a plurality of directions. However, in order to photograph all directions simultaneously, a special camera provided with a plurality of lenses had to be prepared.

  The problem to be solved by the present invention includes the above-described problem as an example.

  In order to solve the above-mentioned problem, the invention described in claim 1 is configured to capture an image and output a corresponding image signal, and to switch a shooting direction to a plurality of directions (hereinafter referred to as a plurality of directions). Driving means for driving the image pickup means to change its posture; frame storage means for storing a plurality of frames acquired in the plurality of directions included in the video signal from the image pickup means; and the frame storage means Frame extracting means for extracting a frame corresponding to a predetermined direction from the plurality of frames stored in the frame, and display means capable of displaying the frame extracted by the frame extracting means.

  An example of the configuration of a vehicle 3 equipped with a video processing system 100 according to an embodiment of the present invention is shown in FIG.

  The vehicle 3 is, for example, an automobile, and can move forward and backward or bend left and right according to the operation of the driver. The vehicle 3 is equipped with a video processing system 100, and the video processing system 100 includes a camera 10, a motor 50, an image processing device 20, a display device 30, and an operation switch 40.

  The camera 10 corresponds to an imaging unit, and is a so-called digital camera using, for example, a CCD (Charge Coupled Device). The camera 10 is attached to the inside of the roof of the vehicle 3 via a motor 50 (corresponding to a driving means), and is installed so as to be able to rotate in a horizontal direction in a loosely inclined posture. The camera 10 has a function of rotating clockwise, for example, to photograph the inside and outside of the vehicle 3 and outputting a corresponding video signal 9. The intruder in the vehicle 3 may be photographed.

  FIG. 2 is a plan view illustrating the photographing behavior of the rotating camera 10. In this embodiment, as an example, as shown in the figure, the camera 10 is used to photograph the inside and outside of the vehicle 3 with four fields of view, front, right, rear, and left, and the field of view in each direction is at an angle of 90 °. A case will be described as an example.

  Returning to FIG. 1, the image processing apparatus 20 functions as a computer having a central processing unit, such as a CPU, a memory such as a ROM and a RAM, and an input / output interface, although not particularly shown. That is, the image processing apparatus 20 performs signal processing according to an image processing program stored in advance in the ROM while using the temporary storage function of the RAM. By executing this image processing program, predetermined image processing is performed on the video signal 9 to extract an arbitrary predetermined frame, and the extracted frame data 11 is output to the display device 30. Thereby, it has the function to select the frame of the video of the fixed direction which an operator desires from the video of a plurality of directions which imaged the situation of the inside and outside of the vehicle 3 and to reproduce on the display device 30 (details will be described later).

  The display device 30 corresponds to display means, and reproduces and displays an image based on the frame data 11 received from the image processing device 20. The display device 30 is a display such as a liquid crystal display device provided in the vehicle 3. The display device 30 can also serve as, for example, a display device for a navigation device that guides a route to be traveled or a display device for a television receiver.

  The operation switch 40 inputs an instruction signal 43 for instructing extraction of a frame in a predetermined direction generated by selection by the operation to the image processing apparatus 20.

  FIG. 3 is a block diagram illustrating an example of a functional configuration of the image processing apparatus 20. As shown in FIG. 3, the image processing apparatus 20 includes a frame storage unit 21, a frame extraction unit 23, and an image output unit 25.

  The image processing apparatus 20 receives a video signal 9 obtained by imaging the situation inside and outside the vehicle 3 in a plurality of directions by the rotating camera 10 described above. The video signal 9 includes multi-directional frame data constituting a multi-directional image, and shooting time data and shooting direction data are attached to each frame data. Note that these data may not be assigned to all the frames 9 and 11, but may be provided every predetermined number of frames.

  The frame storage unit 21 corresponds to a frame storage unit, and stores frame data in a plurality of directions included in the video signal 9 input to the image processing device 20.

  The frame extracting unit 23 corresponds to a frame extracting unit, and an instruction signal 43 for instructing extraction of a frame in a predetermined direction generated by selection by operating the operation switch 40 is input. When a direction to be extracted is specified by inputting an instruction signal 43 by releasing a hand from a rotation key 41 (to be described later) in the operation switch 40, the frame extraction unit 23 stores frame data in a plurality of directions stored in the frame storage unit 21. The frame data 11 in the designated direction is read out and extracted from the captured frame data in a plurality of directions, and the extracted frame data 11 is output to the image output unit 25.

  The image output unit 25 acquires the frame data 11 output from the frame extraction unit 23 and outputs it to the display device 30. Thereby, when the operator selects the direction, the video corresponding to the frame in the desired direction is reproduced on the display device 30. Note that, by inputting the instruction signal 43 from the operation switch 40, the frame extraction unit 23 may access the frame storage unit 21 to extract frame data in a specified direction from the frame storage unit 21.

FIG. 4 is a conceptual diagram illustrating a frame configuration of a video image captured by the camera 10.
In this example, the camera 10 starts shooting from the first rotation four times in the clockwise direction, and in each rotation from the first rotation to the fourth rotation, the front, right, rear, left shown in FIG. The four directions are taken. In addition, it is not limited to such four rounds per round, but once in units or in units of several degrees, and in accordance with the interior layout, "vehicle front""passenger seat direction""left rear seat side""vehiclerear""right rear It may be classified as "seat side" or "driver's seat direction". Further, an up-down direction section may be added to these planar sections. For example, 360 degree round imaging may be performed at an obliquely downward angle, and then 360 degree round imaging may be performed at an obliquely upward angle.

  Then, the frames constituting the four-directional video obtained by shooting as described above are the first frame to the 16th frame in the shooting time order, and are schematically expanded and shown with respect to the shooting direction and time. They are arranged in a manner similar to a helix. That is, the video signal 9 is stored in the frame storage unit 21 as frame data arranged in a spiral manner with respect to the shooting direction and time.

FIG. 5 shows an example of the switch panel surface of the operation switch 40.
As shown in FIG. 5, the operation switch 40 includes a cross key 43 having a rotation key 41 and a time key 42 for selecting a frame to be extracted. FIG. 6 shows how a frame is selected from the frame arrangement using the rotation key 41 and the time key 42.

  The rotation key 41 has a function of selecting a direction in which frame data is to be extracted from frame data in a plurality of directions stored in the frame storage unit 21.

  That is, when the “+” key at one end of the rotation key 41 is pressed with a finger, the frame is advanced in the forward direction from the upper left to the lower right along the spiral of the frame arrangement shown in FIG. Is reproduced on the display device 30. That is, in this case, frame extraction by the frame extraction unit 23 is not performed. When the “−” key at the other end of the rotation key 41 is pressed, the frame is moved back from the lower right to the upper left along the spiral of the frame arrangement shown in FIG. This is performed by the display device 30.

  In any case, only while the “+” and “−” keys of the rotation key 41 are pressed, the display device 30 displays an image along the corresponding time-series direction or the backward direction. When the image is displayed in the time series direction, the operation of pressing the rotation key 41 corresponds to the second operation means. When the rotation key 41 is stopped, the direction to which the frame at the time when the pressing is stopped, that is, the frame direction is selected. Therefore, the operation to stop pressing the rotation key 41 corresponds to the first operation means. The image displayed on the display device 30 stops while reproducing the image corresponding to the frame at the time when the rotation key 41 is stopped.

  On the other hand, the time key 42 has a function of selecting a normal or reverse time sequence for extracting video frame data in a selected specific direction. As described above, by stopping the pressing of the rotation key 41, a specific direction determined by stopping the pressing is selected.

  In this state, when the “+” key at one end of the time key 42 is pressed with a finger, the frame extraction unit 23 linearly skews the spiral of the frame arrangement downward from the top as shown in FIG. A plurality of frames in the specific direction are sequentially extracted in the time series direction so as to be longitudinally cut. The “−” key at the other end of the time key 42 corresponds to a reverse order instruction means, and when this “−” key is pressed, the spiral of the frame arrangement is vertically cut in a skewer direction in a specific direction upward from the bottom. As described above, the plurality of frames in the specific direction are sequentially extracted in the direction going back in time series. In either case, only while the time key 42 is being pressed, the corresponding frame in a specific direction is extracted by the frame extraction unit 23 and an image is displayed on the display device 30. When the pressing of the time key 42 is stopped, the image displayed on the display device 30 is stopped while reproducing the image corresponding to the frame at the time when the pressing of the time key 42 is stopped.

  Note that when the rotation key 41 or the time key 42 is pressed as described above, an image is not displayed on the display device 30, but one of these keys 41 or 42 (or a separately provided cursor moving key may be used. ) May be moved manually to a desired specific frame in the frame arrangement shown in FIG. In this case, when you release your finger from the key (or press the play button provided after you release it), the direction to which the current frame belongs, that is, the frame direction, is selected. A plurality of frames in the specific direction may be sequentially extracted in the time-series direction so as to be vertically cut into skewers in a downward direction from the bottom. Accordingly, the corresponding frame in the specific direction is extracted by the frame extraction unit 23, and the images are displayed on the display device 30 in order of time (or going back in time).

  Further, the rotation key 41 is used as a cursor moving key in the left / right direction (or a separate key for moving the cursor left / right may be provided), and the first to fourth frames in the uppermost row of the spiral frame arrangement in FIG. When one of the frames (or the frame direction itself may be selected) is selected manually, the direction to which the frame belongs, that is, the frame direction is selected. A plurality of frames in the specific direction may be sequentially extracted in the time-series direction so as to be vertically cut into skewers in a downward direction. As the frames to be extracted at this time, all of the frames accumulated in the frame accumulation unit 21 in the direction may be extracted, or frames for a certain predetermined time may be extracted. Accordingly, the corresponding frame in the specific direction is extracted by the frame extraction unit 23, and the images are displayed on the display device 30 in order of time (or going back in time).

  Note that the speed for selecting a frame by the rotation key 41 (speed for moving on the spiral frame arrangement) may be a speed for reproducing the video in real time or a speed for rotating the video in real time. Further, the rotary key 41 may be configured as a double cushion key that has two key functions of a real-time key and a fast-forward key. Alternatively, the rotation key 41 may have a separate key, a real-time key and a fast-forward key. In any case, in the case of fast rotation, images are thinned out and reproduced, and the number of thinned out images is increased as the reproduction speed increases. When the rotation key 41 and the time key 42 are released, the display device 30 stops with the still image being displayed. However, a reproduction key may be separately provided to resume image reproduction. .

  FIG. 7 is an explanatory diagram conceptually showing an example of data processing in the image processing apparatus 20. Shooting is a case where the camera 10 is rotated by, for example, four or more rotations, and each rotation is performed in four directions of front, right, rear, and left. In FIG. 7A to FIG. 7C, the frame data is arranged in the order of time passage from top to bottom in the order of the first frame, the second frame,.

  FIG. 7A shows the frame data stored in the frame storage unit 21. Each frame data 11 includes attached data of shooting time and shooting direction.

In the example shown in the drawing, the shooting direction data is represented by the rotation direction cumulative angle when the front is set to 0 °, and the first frame at the time of first shooting in the first rotation is 0 ° along the rotation walk. The second frame during right shooting is 90 °, the third frame during rear shooting is 180 °, the fourth frame during left shooting is 270 °, and the fifth frame during 360 ° forward shooting is 360 °. The 6th frame when shooting right is 450 °, and so on.
The shooting time data is added to each frame, for example, “** hour ** minute ** second **” in units of 1/100 second.

  For example, the operator operates the rotation key 41 and the time key 42 of the operation switch 40 to select extraction of frame data of a video imaged in the right direction. Then, the frame extraction unit 23 captures the second frame, the sixth frame, and the tenth frame obtained by photographing the right side illustrated in FIG. 7B from the frame data stored in the frame storage unit 21 illustrated in FIG. Extract only frame data such as. The extracted frame data 11 of the second frame in the right direction, the sixth frame, the tenth frame,... Is sent to the image output unit 25, and as shown in FIG. It is reconfigured as a data group and output to the display device 30. Thereby, the video in the right direction is reproduced based on the reconstructed frames on the display device 30.

  The video processing system 100 of the present embodiment is an example of the configuration as described above. Next, an example of the video processing method according to the configuration example will be described with reference to FIGS. 8 and 9. First, the operation of the camera 10 will be described with reference to FIG.

  When the flow is started in FIG. 8, first, in step S10, it is determined whether or not the camera 10 has acquired a shooting start signal (hereinafter referred to as a trigger). For example, when the operator presses the power switch of the camera 10 or the like, a loop is waited until the trigger is acquired by the camera 10 and the determination is satisfied, and when the determination is satisfied, the process proceeds to the next step S20.

  In step S <b> 20, the camera 10 is rotated by the motor 50, and the situation around the inside and outside of the vehicle 3 is photographed in a plurality of directions. In step S30, the above-described shooting time data and shooting direction data are added to each frame of the video obtained in a plurality of directions in step S20.

  Thereafter, in step S40, it is determined whether or not the camera 10 has been rotated a predetermined number of times. The predetermined number of times may be set appropriately by the operator, for example, by the operation switch 40 or the like each time the operation is performed, or may be fixedly determined in advance. While the predetermined number of times has not been reached, the determination is not satisfied, and the procedure of step S40 is repeated to continue the rotation and photographing of the camera 10. When the predetermined number of times is reached and the determination is satisfied, the process proceeds to step S50, and the rotation and photographing of the camera 10 are stopped.

  After that, in step S60, the video signal 9 obtained by photographing is output to the frame storage unit 21, and the frame data 11 in a plurality of directions of video is stored in the frame storage unit 21 of the image processing apparatus 20. Then, it returns to step S10 and repeats the same procedure.

Next, an operation example of the image processing apparatus 20 will be described with reference to FIG.
When the flow is started in FIG. 9, first, in step S <b> 110, it is determined whether or not there has been an instruction to browse the video on the display device 30. For example, the loop waits until a browsing instruction is given by, for example, pressing the power switch of the display device 30 by the operator, and when there is a browsing instruction and the determination is satisfied, the process proceeds to the next step S115.

  In step S115, it is determined whether there is a video shot within a predetermined period. As this predetermined period, for example, it is conceivable that the period goes back for a predetermined fixed period from the browsing instruction time point. If the frame data 11 photographed within the predetermined period is not stored in the frame storage unit 21, the determination is not satisfied and the flow ends. If the frame data 11 is stored, the determination is satisfied, and the process proceeds to the next step S120.

  In step S120, it is determined whether or not the rotation key 41 of the operation switch 40 has been pressed by the operator. If the rotation key 41 is pressed, the determination is satisfied, and the routine goes to the next Step S125. If the determination is not satisfied, the process moves to step S145.

  In step S125, the most time is selected among the frames designated by pressing the “+” key or the “−” key of the rotation key 41, for example, a plurality of frames confirmed to be stored in the frame storage unit 21 in step S115. The image corresponding to the old one is reproduced and displayed on the display device 30. When the “+” key of the rotation key 41 is pressed, as described above, images of frames in time series from the specified frame are sequentially reproduced and displayed, and when the “−” key is pressed, the above designation is made. The images of the frames are sequentially reproduced and displayed along the retroactive direction in the reverse direction to the time series from the frames that have been made.

  In the next step S130, the operator stops pressing the rotation key 41 and determines whether or not the hand is released. If the rotation key 41 is kept pressed, the determination is not satisfied, the process returns to step S125, and the same procedure is repeated. If the pressing of the rotation key 41 is stopped, the determination is satisfied, and the routine goes to the next Step S135.

  In step S135, the frame direction when the rotation key 41 is stopped, that is, the shooting direction corresponding to the frame is fixed. In the next step S140, the reproduction is stopped while displaying the image corresponding to the frame at the time when the rotation key 41 is stopped in step S130, and a so-called reproduction pause state is set. Thereafter, the process proceeds to step S145.

  In step S145, it is determined whether or not the time key 42 of the operation switch 40 has been pressed by the operator. If the time key 42 is not pressed and the determination is not satisfied, the process returns to step S120 and the same procedure is repeated. If the time key 42 is pressed, the determination is satisfied, and the routine goes to the next Step S150.

  In step S150, a plurality of frames corresponding to the frame direction fixed in step S135 are extracted, and images corresponding to the extracted frames are sequentially reproduced on the display device 30. That is, when the “+” key of the time key 42 is pressed, the images corresponding to the extracted frames are sequentially displayed in the forward direction as described above. When the “−” key of the time key 42 is pressed, images corresponding to the extracted frames are sequentially displayed in the reverse direction (backward direction).

  Thereafter, in step S155, the operator stops pressing the time key 42 and determines whether or not the hand has been released. If the time key 42 is kept pressed, the determination is not satisfied, the process returns to step S120, and the same procedure is repeated. If the pressing of the time key 42 is stopped, the determination is satisfied, and the routine goes to the next Step S160.

  In step S160, the playback is stopped while the image corresponding to the frame at the time when the pressing of the time key 42 is stopped is displayed, and a so-called playback pause state is set. Then, it moves to step S120 and repeats the same procedure.

  In the video processing system 100 in the above embodiment, the imaging unit 10 (corresponding to a camera) that captures a video and outputs a corresponding video signal, and the shooting direction are switched to a plurality of directions (hereinafter referred to as a plurality of directions). In addition, a driving unit 50 (corresponding to a motor) that drives the imaging unit 10 to change its attitude, and a plurality of frames 9 acquired in the plurality of directions included in the video signal 9 from the imaging unit 10 Frame accumulating means 21 (corresponding to a frame accumulating section) for accumulating, and frame extracting means 23 for extracting a frame corresponding to a predetermined direction among a plurality of frames 9 accumulated in the frame accumulating means 21 (in the frame extracting section) And display means 30 (corresponding to a display device) capable of displaying the frame extracted by the frame extraction means 23.

  In the video processing system 100 of the present embodiment, the imaging unit 10 is driven by the driving unit 50. By the driving, the posture of the imaging unit 10 is changed, and the shooting direction is switched to a plurality of directions. As a result, the video signal 9 of the video imaged in the plurality of directions is input from the imaging unit 10 to the frame storage unit 21, and the plurality of frames 9 acquired in the plurality of directions are stored in the frame storage unit 21. From the plurality of frames 9 thus stored, the frame extracting means 23 extracts only the frame 11 corresponding to a predetermined direction (which may have a certain angle range; see a modification of (3) described later). Are displayed on the display means 30.

  As a result, the operator can selectively view only a certain one-way image from a plurality of directions. In other words, continuous monitoring video in the same direction can be realized. As described above, it is not necessary to prepare a special camera having a plurality of lenses by extracting and displaying a predetermined direction from images in a plurality of directions by driving one imaging means 10. Alternatively, the system configuration can be simplified and complicated control such as synchronization of multiple cameras is required as compared to the case where cameras with fixed directions are arranged in all directions and the images for each camera are switched and displayed. Absent.

  In addition to the above-described configuration, the video processing system 100 according to the embodiment further includes first operation means 41 (corresponding to a rotation key) that allows an operator to input a specific direction among the plurality of directions.

  Thus, the operator can specify a desired direction intended by himself / herself from a plurality of photographic images, and can selectively view only the images in that direction. In addition, by changing the direction in which the instruction is input as appropriate, an image in a different direction can be appropriately viewed each time. Furthermore, when it is desired to check whether there is an abnormality in a specific direction in all directions, it is difficult to capture a time-series change in slow rotation photography, but in this embodiment, a selected image in one direction is selected. It becomes easier to understand time-series changes in a specific direction.

  In the video processing system 100 in the above embodiment, in addition to the above-described configuration, the frame extraction unit 23 instructs the first operation unit 41 among the plurality of frames 9 stored in the frame storage unit 21. The frame 11 corresponding to the input specific direction is extracted, and the display means 30 displays the frame 11 corresponding to the specific direction extracted by the frame extraction means 23.

  At the time of shooting, the frames 9 extending in all directions are stored in the storage unit 21, and the frame 11 in the direction intended by the operator is extracted from the frames and displayed. Accordingly, even when the direction in which the operator inputs an instruction changes, an image corresponding to the direction in which the instruction is input can be reliably displayed each time.

  In the video processing system 100 in the above embodiment, in addition to the above-described configuration, the operator further includes second operation means 41 (corresponding to a rotary key) for displaying the frame 11 in time series, The display unit 30 displays the plurality of frames 9 stored in the frame storage unit 21 in time series in response to an instruction input from the second operation unit 41.

  Thereby, the operator does not display the video in a specific direction according to the instruction input of the first operation means 41, but displays the video in a plurality of directions along the time series corresponding to the posture change by the driving means 10. When it is desired to display sequentially, display corresponding to such needs can be performed. Therefore, the range of usage modes can be expanded for the operator, and convenience can be improved.

  In the video processing system 100 according to the above embodiment, in addition to the above-described configuration, the second operation means 41 further allows the operator to display the frames 11 in a specific direction in a time series reverse order. The reverse order instruction means 42 (corresponding to the time key) is displayed, and the display means 30 sets the frames 11 extracted by the frame extraction means 23 as time series in response to an instruction input from the reverse order instruction means 42. Display in reverse order.

  As a result, it is possible to view the video in the desired direction intended by the operator by going back in time. In particular, when watching a video in a desired direction, when a user wants to watch a certain scene again, the reverse order instruction means 42 can go back to the scene again to further improve convenience. it can.

  In the video processing system 100 according to the above embodiment, in addition to the above-described configuration, the driving unit 50 rotates the imaging unit 10 over the entire circumference in the circumferential direction, so that the shooting direction is changed over the entire circumferential direction. The frame accumulating unit 21 accumulates a plurality of frames 9 acquired in the entire circumference in the circumferential direction, and the frame extracting unit 23 includes the frame accumulating unit. Of the plurality of frames 9 related to the entire circumference in the circumferential direction accumulated in the means 21, the frame 11 corresponding to the predetermined direction is extracted.

  Thus, the operator can selectively view only a video in one direction (which may have a certain angle range) from the captured video over 360 ° in the circumferential direction. Further, by extracting and displaying a predetermined direction from all images obtained by driving one image pickup means 10 360 °, a large number of fixed-direction cameras are arranged in each direction over the entire circumference 360 °. Compared with the case where each video is switched and displayed, the system configuration can be simplified and complicated control is not required.

  In addition, this embodiment is not restricted above, A various deformation | transformation is possible. Hereinafter, such modifications will be described in order.

(1) When the accumulated frames are rearranged in advance according to the direction In other words, in the above embodiment, the operator operates the operation switch 40 to indicate a desired direction out of a plurality of directions. A frame in the indicated direction was extracted from frames in a plurality of directions of the stored video, and only the video corresponding to the frame in that direction could be selectively viewed. In this modification, a plurality of frames stored in the frame storage unit 21 are extracted for each of a plurality of predetermined setting directions before the operator operates the operation switch, and are arranged for each setting direction. Instead, it is reconfigured.

FIG. 10 shows an example of the switch panel surface of the operation switch 45 used in this modification.
The operation switch 45 includes a direction designation key 46.

The direction designation key 46 corresponds to the first operation means, and has a function of individually designating and selecting the direction in which frame data is to be reproduced, and selects each of the front, right, rear and left directions. The buttons 46a, 46b, 46c and 46d are provided.
The operation switch 45 of this modification also has a time key 47 having “+” and “−” keys, which is selected by the direction designation key 46, like the time key 42 of the operation switch 40 of the above embodiment. You may make it select in order of time to extract the flame | frame data of the image | video in a specific direction forward and reverse. In this case, it is possible to perform retroactive browsing and forward browsing as described above.

  FIG. 11 is a conceptual diagram showing a frame arrangement in which frame data in a plurality of directions is rearranged in each of the front, right, rear, and left directions and reconstructed in this modification. Corresponding to The frame storage unit 21 of the image processing apparatus 20 stores frame data arranged in a spiral manner with respect to the shooting direction and time, as shown in FIG. In the present modification, as described above, the frame data in FIG. 4 is rearranged in each of the front, right, rear, and left directions and reconstructed as shown in FIG.

  That is, when the operator presses the “Previous” button 46 a of the direction designation key 46, as shown in the uppermost row in FIG. 11, the first of the plurality of frame data stored in the frame storage unit 21 is captured. The frame, the fifth frame, the ninth frame, the thirteenth frame,... Are extracted by the frame extraction unit 23 and rearranged so as to be continuous in this order to be reconfigured. When the “right” button 46 b of the direction designation key 46 is pressed, the second frame, the sixth frame, the tenth frame, and the fourteenth frame in which the right side of the plurality of frame data stored in the frame storage unit 21 is photographed. Frames,... Are extracted by the frame extraction unit 23, and are rearranged so as to be reconstituted in this order.

  Similarly, when the “back” button 46 c of the direction designation key 46 is pressed, the third frame, the seventh frame, the fourteenth frame, the eighteenth frame, and the eighteenth frame, in which the rear side is photographed among the plurality of frame data accumulated in the frame accumulation unit 21. Frames,... Are extracted by the frame extraction unit 23, and are rearranged so as to be reconstituted in this order. Further, when the “left” button 46 d of the direction designation key 46 is pressed, the fourth frame, the eighth frame, the twelfth frame, the sixteenth frame in which the left side is photographed among the plurality of frame data stored in the frame storage unit 21. Frames,... Are extracted by the frame extraction unit 23, and are rearranged so as to be reconstituted in this order.

  When the above-described reconstruction is completed by the “front”, “right”, “rear”, and “left” buttons 46a to 46d of the direction designation key 46, the buttons “front”, “right”, “rear”, and “left” are automatically selected. The frame extraction unit 23 outputs the frame data of the video in the specific direction selected in “46a to 46d” to the display device 30 via the image output unit 25. That is, as shown in FIG. 11, a plurality of reconstructed frames 11 corresponding to the specific direction are output to the display device 30 in the forward direction along the flow of time, and the corresponding image is displayed on the display device 30. Is done.

  FIG. 12 is an explanatory diagram conceptually showing an example of data processing in the image processing apparatus 20 in the present modification, and corresponds to FIG. 7 described above. As in FIG. 7, in FIGS. 12A to 712 c, the frame data are arranged in the order of time passage from top to bottom in the order of the first frame, the second frame,.

  FIG. 12A shows frame data stored in the frame storage unit 21 as in FIG. 7A, and is attached with the above-described shooting direction data and shooting time data.

  In the present modification, unlike the above embodiment, the frame data stored in the four directions stored in the frame storage unit 21 is reconstructed for each direction as described above, without the operator operating the operation switch 40. That is, as shown in FIG. 12B, a frame data group of the first frame, the fifth frame, the ninth frame,... Taken forward, and the second frame, the sixth frame taken rightward, Frame data group of the 10th frame,... Frame data group of the 3rd frame, 7th frame, 11th frame,... Taken from the rear, 4th frame, 8th frame shot of the left side , The frame data group of the twelfth frame,... In the figure, some arrow lines are omitted for the sake of complexity.

  In this example, the operator presses the “front” button 46 a of the direction designation key 46 to select the front frame data group, and the selected front frame data group is displayed from the frame extraction unit 23 to the display device 30. Is output. Thus, the forward video is reproduced based on the frames output by the display device 30.

  The video processing system 100 according to the present modification is an example of the configuration as described above. Next, an example of the video processing method according to the configuration example will be described with reference to FIG. In this modification, the operation of the camera 10 is the same procedure as in FIG. 8 in the above embodiment, and the description is omitted. An example of the operation of the image processing apparatus 20 will be described with reference to FIG. 13 corresponding to FIG.

  When the flow is started in FIG. 13, first, in step S <b> 210 similar to step S <b> 110 in FIG. 9, it is determined whether or not there has been an instruction to browse the video on the display device 30. The loop waits until there is a viewing instruction. If there is a viewing instruction, the process proceeds to step S220 similar to step S115 in FIG. 9, and it is determined whether or not there is a video shot within a predetermined period. If the frame data is stored in the frame storage unit 21, the determination is satisfied, and the routine goes to the next Step S230.

  In step S230, the frame extraction unit 23 captures the four-direction frame data stored in the frame storage unit 21, and, as described above, the captured four-direction frame data is respectively forward, right, rear, and left. Rearrange and reorganize the data for each direction. Note that the steps S115 and S230 are not limited to being executed after a browsing instruction such as pressing the power switch of the display device 30 as described above, and are automatically executed without a browsing instruction. You may make it do.

  Thereafter, in step S240, it is determined whether or not the direction of the video to be reproduced has been selected by the operator. The loop waits until the determination is satisfied, and when the operator presses one of the “front”, “right”, “rear”, and “left” buttons of the direction designation key 46 of the operation switch 45, the direction of the frame to be reproduced is selected. The determination is satisfied and the routine goes to Step S250.

  In step S250, the frame data in the direction selected in step S240 is extracted from the frame data group reconstructed for each direction held in the frame extraction unit 23, and the extracted frame data is extracted from the frame extraction unit 23. Is output to the display device 30 via the image output unit 23, and the video in the direction selected by the display device 30 is reproduced and displayed.

  In the video processing system 100 in the above embodiment, in addition to the above-described configuration, the first operation means 46a to 46d ("front" of the direction designation key) that allow the operator to input a specific direction among the plurality of directions. Equivalent to “right”, “rear” and “left” buttons).

  Thus, the operator can specify a desired direction intended by himself / herself from a plurality of photographic images, and can selectively view only the images in that direction. In addition, by changing the direction in which the instruction is input as appropriate, an image in a different direction can be appropriately viewed each time.

  In the video processing system 100 according to the modified example, in addition to the above-described configuration, the frame extracting unit 23 further adds a plurality of frames 9 stored in the frame storing unit 21 to a plurality of predetermined setting directions. And rearrange them for each setting direction.

  After the frames 9 in all directions are accumulated in the frame accumulation means 21 at the time of shooting, the accumulated frames 9 are rearranged in advance in a plurality of setting directions without waiting for an operator's instruction. Keep it. As a result, when an instruction is input in the direction intended by the operator, it is sufficient to display the frame in the set direction that matches the direction as it is, and the processing up to the display can be speeded up. In particular, when the posture change by the driving unit 50 is so fast that the visual observation with the naked eye is impossible (in the case of so-called high-speed shooting), it is not necessary to consider display of images in a plurality of directions along a time series. Therefore, the prior rearrangement reconstruction as described above is effective. Further, in this case, the interval between shot images in the same direction can be set to, for example, about several frames / second and can be reproduced as a normal moving image, and a single direction can be monitored sufficiently continuously. In this case, in order to take a clear image even if it is rotated at a relatively high speed (several revolutions / second), a high-speed shutter is combined, and the shutter is released after each frame image is shot, so that the shutter is taken care of. The images may be configured as a series of images taken in this manner.

  In the video processing system 100 according to the modified example, in addition to the above-described configuration, the display unit 30 further includes the first operation unit 46a among the frames for each setting direction reconfigured by the frame extraction unit 23. The frame 11 in the setting direction corresponding to the specific direction input by the instructions at .about.d is displayed.

  Thereby, the frame 11 in the setting direction corresponding to the instruction input of the first operation means 46a to 46d is selected from the frames 11 reconstructed in advance based on the accumulated frames 9, and displayed quickly. Can do.

(2) When the camera is rocked instead of rotationally driven In the above embodiment and the modified example of (1), the camera 10 rotates around the entire circumference of 360 ° and images the surroundings. However, depending on the application of the camera 10, the camera 10 is rotated only once to photograph the surroundings, or the camera 10 is swung within a predetermined angle range to photograph the surroundings, or the surroundings are repeatedly swung. In some cases, 360 ° is rotated once or a plurality of times to photograph the surroundings. This modification is a case where such a camera 10 is swung to photograph the surroundings.

FIG. 14 is a conceptual diagram showing a frame configuration of a video image captured by the camera and its reconstruction in the present modification.
As shown in FIG. 14, in this example, the camera 10 is driven so as to face forward, right, left, and backward in the first swing, and right and forward in the second swing. Driven in the order of left, rear, and driven in the order of left, rear, right in the third swing, and forward, right, left in the fourth swing Moving in order, shooting in each direction.

  Correspondingly, the frame storage unit 21 of the image processing unit 20 includes a first frame, a second frame, a third frame, and a fourth frame obtained by photographing the front, the right, the left, and the rear by the first swing. And the 5th, 6th, 7th, and 8th frames shot right, front, left, and back with the second swing, and the left, back, and right shot with the third swing. The ninth frame, the tenth frame, and the eleventh frame, and the twelfth frame, the thirteenth frame, and the fourteenth frame obtained by photographing the front, right, and left by the fourth swing are stored in this order.

  Then, when the operator selects, for example, the forward direction using the rotary key 41 of the operation switch 40 similar to that in the above embodiment, as shown in the drawing, the frame data stored in the frame storage unit 21 as described above is forward. Only the first frame, the sixth frame, and the twelfth frame are extracted and reconstructed. Then, the reconstructed front frame group is output from the frame extraction unit 23 to the display device 30 via the image output unit 25. Thereby, even when the camera 10 swings and the surroundings are imaged, a desired forward image can be viewed on the display device 30. The same applies to the other directions. As a result, the same effect as the above embodiment can be obtained.

(3) When giving a predetermined angular width in each direction In the said embodiment and the modification of (1) and (2), the imaging | photography direction of the camera 10 is four directions of the front, right side, back, and left, Although an example has been described in which a single image of one frame is obtained in each direction per one revolution or one revolution, the present invention is not limited to this. That is, as shown in FIG. 15, even in the case where a plurality of frames, in the illustrated example, three frames are obtained within a predetermined angle range centered on the front, right, rear, and left directions. The present invention can be applied.

  In the example shown in FIG. 15, for each rotation of the camera 10, three consecutive frames are photographed in each of the front, right, rear, and left directions. In such a video frame configuration, for example, when playing a video in the right direction, the frame extraction unit 23 is the first frame to the right of the first rotation, as indicated by a white arrow in the figure. Extract in the order of 4 frames → next 5th frame → last 6th frame, and then extract in the order of 16th frame → 17th frame → 18th frame which is the first frame on the right of the second rotation, Thereafter, similar frame extraction may be repeated.

  Note that the driving and shooting of the camera 10 described above may be performed periodically, at the discretion of the operator, or automatically when some abnormality occurs. Further, the display device 30 is not limited to a vehicle-mounted device, and may be displayed at a distance remote from the vehicle 3 using a portable terminal or other stationary terminal.

  Although some of the embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications are made without departing from the spirit of the present invention. Is.

It is a conceptual diagram which shows the structural example of the vehicle carrying the video processing system of one Embodiment of this invention. It is an image figure which shows a mode that the camera of a video processing system image | photographs the inside and outside surroundings of a vehicle. It is a block diagram which shows an example of a function structure of the image processing apparatus of a video processing system. It is a conceptual diagram which shows the frame structure of the image | video imaged with the camera. It is a top view showing an example of the switch panel surface of an operation switch. It is a conceptual diagram which shows the mode of the selection of the frame from the frame arrangement | sequence by the rotation key and time key of an operation switch. It is explanatory drawing which shows an example of the data processing in an image processing apparatus notionally. It is a flowchart which shows the operation example of a camera. It is a flowchart which shows the operation example of an image processing apparatus. It is a top view which shows an example of the switch panel surface of the operation switch used in a 1st modification. It is a conceptual diagram which shows the frame arrangement | sequence which rearranged and rearranged the frame data of multiple directions for every direction of the front, right side, back, and left side. It is explanatory drawing which shows an example of the data processing in an image processing apparatus notionally. It is a flowchart which shows the operation example of an image processing apparatus. It is a conceptual diagram which shows the frame structure of the image | video imaged with the camera in a 2nd modification, and its reconstruction. It is a conceptual diagram which shows the frame structure of the image | video imaged with the camera in a 3rd modification, and the selection method of the flame | frame.

Explanation of symbols

3 Vehicle 10 Camera (equivalent to imaging means)
20 Image processing device 21 Frame storage unit (corresponding to frame storage means)
23 Frame extraction unit (equivalent to frame extraction means)
30 display device (equivalent to display means)
40 operation switch 41 rotation key (corresponding to first operation means and second operation means)
42 hour key (equivalent to reverse order instruction means)
46 Direction designation key (equivalent to the first operation means)
50 motor (equivalent to driving means)

Claims (8)

Imaging means for capturing video and outputting a corresponding video signal;
Driving means for driving the imaging means to change its posture so as to switch the photographing direction to a plurality of directions (hereinafter referred to as a plurality of directions);
Frame storage means for storing a plurality of frames acquired in the plurality of directions included in the video signal from the imaging means;
A frame extraction means for extracting a frame corresponding to a predetermined direction from the plurality of frames accumulated in the frame accumulation means;
And a display unit capable of displaying the frame extracted by the frame extraction unit. The video processing system according to claim 1,
An image processing system, comprising: a first operation unit that allows an operator to input a specific direction among the plurality of directions. The video processing system according to claim 2,
The frame extraction means includes
Out of a plurality of frames stored in the frame storage means, a frame corresponding to the specific direction input by the first operation means is extracted,
The display means includes
A video processing system, wherein the frame corresponding to the specific direction extracted by the frame extraction means is displayed. The video processing system according to claim 3,
The operator has a second operation means for displaying the frames in time series,
The display means includes
A video processing system for displaying a plurality of frames stored in the frame storage unit in time series in response to an instruction input from the second operation unit. The video processing system according to claim 2,
The frame extraction means includes
A video processing system, wherein a plurality of frames stored in the frame storage means are extracted for each of a plurality of predetermined setting directions, rearranged for each setting direction, and reconfigured. The video processing system according to claim 5,
The display means includes
Video processing for displaying a frame in the setting direction corresponding to the specific direction input by the first operation unit among frames in each setting direction reconstructed by the frame extraction unit system. The video processing system according to any one of claims 2 to 6,
The second operating means includes
The operator includes a reverse order instruction means for displaying frames in a specific direction in an order reverse to the time series,
The display means includes
A video processing system for displaying the frames extracted by the frame extraction unit in an order reverse to a time series in response to an instruction input by the reverse order instruction unit. The video processing system according to any one of claims 1 to 7,
The driving means includes
By rotating the imaging means over the entire circumference, the shooting direction is switched over the entire circumference.
The frame storage means includes
Accumulating a plurality of frames acquired in the entire circumference in the circumferential direction included in the video signal from the imaging means,
The frame extraction means includes
A video processing system, wherein a frame corresponding to a predetermined direction is extracted from a plurality of frames related to the entire circumference in the circumferential direction accumulated in the frame accumulation means.

Images