JP2011250928A - Device, method and program for space recognition for visually handicapped person - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a novel space recognition device for visually handicapped persons enabling a user to recognize a three-dimensional space with high accuracy through a sensory system (a tactile sensory system, an auditory sensory system, or the like) markedly lower in resolution than a visual system.SOLUTION: A partial pixel region of a main image imaged by a main camera of a stereo imaging apparatus is defined as a noted region, and distance values on every pixel are computed by carrying out stereo matching with sub-images imaged by a sub-camera on each pixel that composes the noted region. A statistical central value (e.g. an average value) is derived on a plurality of computed distance values, and a vibrator is driven on a larger output level as the central value becomes smaller. In response to the operation of the stereo imaging apparatus, the output level of the vibrator changes with the lapse of time. The user recognizes a depth feeling of an obstacle from the aging effect of a vibration level sensed by a tactile sensation.

Description

  The present invention relates to a space recognition device for visually impaired people, and more particularly to a space recognition device for visually impaired people using stereo vision.

  Conventionally, various studies have been made on spatial recognition systems using stereo vision for the purpose of assisting visually impaired people. Japanese Patent Laid-Open No. 2002-65721 (Patent Document 1) calculates a three-dimensional position of an object in a user's front space from an image captured by a stereo camera attached to the user's frontal head, An environment recognition support device for a visually impaired person is disclosed, in which a tactile display device touched by a user or a stereo headphone is driven. Japanese Patent Laying-Open No. 2003-79585 (Patent Document 2) converts the three-dimensional information of an obstacle obtained from an image photographed by a stereo camera into two-dimensional information, and the user's forehead based on the two-dimensional information. Disclosed is a walking aid for visually impaired persons, characterized in that the spatial information is transmitted to the user in a bodily manner by driving the protrusions of the actuator array applied to.

  However, it is known that the information transmission speed of the human auditory system is much slower than that of the visual system, and that of the tactile system is much slower than that of the auditory system. Regardless of how high resolution information is provided to the user based on accurate three-dimensional information obtained using vision, the resolution of the sensory system on the user side is insufficient. There was a problem that the sense of depth in the space could not be recognized accurately.

JP 2002-65721 A JP 2003-79585 A

  The present invention has been made in view of the above problems in the prior art, and the present invention provides a three-dimensional view to a user via a sensory system (such as an auditory system or a tactile system) that has a significantly lower resolution than the visual system. It is an object of the present invention to provide a novel space recognition device for visually handicapped persons capable of accurately recognizing a space.

  As a result of earnestly examining the novel space recognition device for the visually impaired, which enables the user to accurately recognize the three-dimensional space through a sensory system that has a remarkably lower resolution than the visual system, The idea was that only the information that the user really needed was extracted from the vast amount of 3D information in the space, and this was reduced to 1D information suitable for the resolution of sensory organs such as touch and hearing. . Based on this idea, the present inventor defines a partial pixel area of the main image captured by the main camera of the stereo imaging device as the attention area, and uses stereo vision for all the pixels constituting the attention area. Thus, a statistical representative value is derived for the plurality of distance values calculated in this way, and a system configuration for driving the output device based on the representative value is conceived and the present invention has been achieved.

  In other words, according to the present invention, there is provided a space recognition device for a visually impaired person including a stereo imaging device including a main camera and a sub camera, an information processing device, and an output device, wherein the information processing device is picked up by the main camera. Stereo matching is performed between the attention area setting unit that defines a part of the pixel area of the main image as the attention area, and the sub-image captured by the sub camera for each pixel constituting the attention area, and the attention area A stereo matching unit that detects pixels in the sub-image corresponding to each of the pixels, and a parallax calculated from coordinates of the pixels in the corresponding sub-image detected with the pixels in the region of interest. A distance value calculating unit that calculates a distance value of each pixel of the attention area, a representative value deriving unit that derives a statistical representative value for the plurality of calculated distance values, and the representative value An output signal generator for generating a large output level of the output signal as smaller, visually impaired spatial recognition device and an output device that drives on the basis of the output signal is provided.

  As described above, according to the present invention, a novel space recognition device for visually handicapped persons that allows a user to accurately recognize a three-dimensional space through a sensory system that has a significantly lower resolution than the visual system. Is provided.

The figure which shows the space recognition apparatus for visually impaired persons of this embodiment. The conceptual diagram for demonstrating the function of the attention area setting part in this embodiment. The figure which shows three modes of an attention area. The conceptual diagram for demonstrating the utilization method of three modes of an attention area. The figure which shows the stereo imaging device of the space recognition apparatus for visually impaired persons of this embodiment. The figure which shows the usage condition of the space recognition apparatus for visually impaired persons of this embodiment. The figure which shows the usage condition in "slit mode" of the space recognition apparatus for visually impaired persons of this embodiment. The conceptual diagram for demonstrating the effect of "slit mode" in this embodiment. The conceptual diagram for demonstrating the effect of "slit mode" in this embodiment. The conceptual diagram for demonstrating the effect of "slit mode" in this embodiment. The conceptual diagram for demonstrating the effect of "slit mode" in this embodiment. The conceptual diagram for demonstrating the effect of "slit mode" in this embodiment. The conceptual diagram for demonstrating the effect of "slit mode" in this embodiment. The conceptual diagram for demonstrating the effect of "slit mode" in this embodiment.

  Hereinafter, the present invention will be described with reference to embodiments shown in the drawings, but the present invention is not limited to the embodiments shown in the drawings. In the drawings referred to below, the same reference numerals are used for common elements, and the description thereof is omitted as appropriate.

  FIG. 1 shows a space recognition device 100 for a visually impaired person according to an embodiment of the present invention. The space recognition device 100 for the visually impaired (hereinafter simply referred to as the space recognition device 100) includes a stereo imaging device 10, an information processing device 20, an output device 30, and setting switching means 40. . The stereo imaging device 10 is a means for capturing an image of the object S in the traveling direction of a visually impaired user (hereinafter simply referred to as a user), and includes a main camera 12 and a sub camera installed in parallel equivalence. 14 is provided. The main camera 12 and the sub camera 14 are preferably configured as a 3CCD camera capable of capturing a digital moving image or a digital continuous still image with high resolution. Each image captured by the main camera 12 and the sub camera 14 is transferred to the information processing apparatus 20. The information processing device 20 calculates the separation distance between the object S and the stereo imaging device 10 (that is, the user) based on the two images captured by the main camera 12 and the sub camera 14, using stereo vision. An output signal corresponding to the separation distance is generated and transmitted to the output device 30.

  The information processing apparatus 20 in the present embodiment can be configured as a general-purpose computer. By executing an application program described in a programming language under the management of an appropriate operating system, the following functional units are provided. Realize. That is, the information processing device 20 includes an image input interface 21 for receiving an image transferred from the stereo imaging device 10, an attention area setting unit 22, a stereo matching unit 23, a distance value calculation unit 24, a representative A value deriving unit 25, an output signal generating unit 26, and a setting switching unit 27 are included. Hereinafter, functions performed by the respective functional means will be described with reference to FIGS.

  FIG. 2 is a conceptual diagram for explaining the function of the attention area setting unit 22 in the present embodiment. In the present embodiment, first, two images of the object S imaged by the main camera 12 and the sub camera 14 of the stereo imaging device 10 are input to the attention area setting unit 22 via the image input interface 21. FIG. 2A shows two images input to the attention area setting unit 22. In FIG. 2, a captured image of the main camera 12 (hereinafter referred to as a main image) is shown on the left side of the paper, and a captured image of the sub camera 14 (hereinafter referred to as a sub image) is shown on the right side of the paper. ing. As described above, since the main camera 12 and the sub camera 14 are installed in parallel equiposition, the epipolar lines of the main image and the sub image coincide with each other as shown in FIG. .

  Subsequently, as illustrated in FIG. 2B, the attention area setting unit 22 sets a part of the pixel area of the main image received from the stereo imaging device 10 as the attention area R (enclosed by a broken line). Note that the attention area R is preferably defined so that its center of gravity coincides with the optical axis of the main camera (indicated by “+” in the drawing). In the present embodiment, it is preferable that a plurality of attention areas R having different shapes and sizes are defined and configured so that the aspect of the attention area R can be appropriately switched by setting. This point will be described in detail later.

  All the pixel information constituting the attention area R of the main image set by the attention area setting section 22 and all the pixel information of the sub-image are sent to the stereo matching section 23. FIG. 2C conceptually shows the pixel information sent to the stereo matching unit 23. Here, the pixel information is information including the luminance value and coordinates of each pixel. The stereo matching unit 23 performs a stereo matching process for each pixel constituting the attention area R with the sub image for each pixel, and identifies pixels in the sub image corresponding to each pixel included in the attention area R. To do. The stereo matching process can be performed by a known method such as a method using dynamic programming. As a matching result, the stereo matching unit 23 calculates the distance value of information (hereinafter referred to as pixel pair information) that associates the coordinates of each pixel in the attention area R with the coordinates of the corresponding pixel in the sub-image. Send to part 24. The distance value calculation unit 24 calculates a distance value based on the coordinates of the pixel of the attention area R and the coordinates of the pixel of the sub-image included in the pixel pair information. In the present embodiment, the distance value is the length of a perpendicular line drawn from the position on the object S reflected in each pixel of the region of interest R to the line connecting the centers of the lenses of the main camera 12 and the sub camera 14 or its length. It can be defined as an approximate value. The distance value calculation unit 24 sets the parallax p as a value obtained by subtracting the deviation from the distance between both pixels obtained based on the coordinates of the pixel of the attention area R and the coordinates of the pixel of the sub-image, and A distance value can be calculated by dividing the product of the center-to-center distance of each lens and the focal length of the camera by the parallax p, or a distance value as an approximate value by multiplying the parallax p by an appropriate fixed coefficient. Can also be derived. The distance value is calculated for all the pixels constituting the attention area R, and the same number of distance values as the number of pixels constituting the attention area R are sent to the representative value deriving unit 25 as a measurement result.

  The representative value deriving unit 25 is a functional unit for reducing enormous spatial information included in the attention area R. That is, the representative value deriving unit 25 derives a statistical representative value for a plurality of distance values received from the distance value calculating unit 24. At this time, the measurement error in pixel units is automatically absorbed. In the present embodiment, as a representative value, any one of an average value, a median value, a mode value, and a minimum value can be set. The derived representative value is sent to the output signal generator 26.

  The output signal generation unit 26 converts the representative value into one-dimensional information (output level) adapted to the resolution of a sensory organ such as a sense of touch or hearing. A function for converting the representative value into an output level is prepared in advance in the output signal generation unit 26, and the output level is determined with reference to the function using the representative value as a parameter. In the present embodiment, the output level can be defined by frequency and / or amplitude (hereinafter referred to as frequency and the like). When the output level is defined by a frequency or the like, a function having a negative correlation between the representative value and the frequency is prepared, so that a higher frequency (or a larger amplitude) is determined as the representative value becomes smaller. The output signal generator 26 generates a periodic signal based on the determined frequency or the like. In addition, instead of the above function, an output signal generation table is prepared in which the representative value and the frequency have a negative correlation, and the output level is determined by referring to the table. May be.

  Finally, the output device 30 is driven by the output signal transmitted from the information processing device 20. In the present embodiment, the output level is defined by the frequency or the like, and the output device 30 can be configured as a vibrator such as a vibrator or a speaker. For example, when a vibrator is used as the vibrator, the vibrator vibrates at a higher frequency (or a larger amplitude) as the output level increases (that is, as the representative value decreases). It is possible to intuitively recognize the sense of distance of the object S via In addition, when a speaker is used as the vibrator, a sound wave having a high frequency (or a large amplitude) is generated as the output level increases (that is, as the representative value decreases). The sense of distance of the object S can be recognized intuitively. In addition, the output device 30 may be configured as a pressurizer, and the output level may be defined by pressure, so that the sense of distance of the object S can be transmitted via the user's pressure sense.

  When defining a function (or an output signal generation table) for converting the above-described representative value into an output level, it is necessary to assign an output range unique to the output device 30 to the range of the representative value. When the range of the representative value is wide, it becomes possible to comprehensively convey the sense of distance from the near view to the distant view to the user, but the information resolution is reduced accordingly. Therefore, in the present embodiment, it is preferable to define a plurality of modes (for example, “distant view mode”, “near view mode”, etc.) for the range of representative values and to be able to switch between the modes. For example, in the “distant view mode”, by setting the range of the representative value to “1 to 4 m”, the user can acquire a wider range of information about the surrounding situation than when the conventional white cane is used. Is possible. On the other hand, in the “near view mode”, by setting the range of the representative value to “0 to 2 m”, the user can acquire high-resolution information about the close environment. These modes are switched by the setting switching unit 27 controlling the output signal generating unit 26 in response to an input from the setting switching unit 40 operated by the user.

  As described above, the embodiment has been described in which the sense of distance of the obstacle is intuitively transmitted to the user. However, the present invention is not limited to this. For example, the output device 30 is configured as an audio output device, and the representative value is set. By defining in correspondence with the tone of the sound, the user can conceptually convey the sense of distance of the object S to the user, or convey the distance value corresponding to the representative value with a prepared language voice. The distance to the object S can be quantitatively transmitted to the user. The components of the space recognition device 100 according to the present embodiment have been described above. Next, the attention area R setting mode in the space recognition device 100 according to the present embodiment will be described below.

  In the space recognition apparatus 100 of the present embodiment, it is preferable that a plurality of setting modes are defined in advance for the attention area R as described above, and each mode can be switched. In the present embodiment, as the setting mode, for example, three modes of “wide area mode”, “local mode”, and “slit mode” can be defined. These three modes can provide the user with different utilization methods of the space recognition apparatus 100. Hereinafter, the three modes will be described with reference to FIGS. 3 and 4.

  3A to 3C show the attention area R in the three modes surrounded by a broken line. In the “wide area mode” shown in FIG. 3A, the entire area of the main image (or an area close thereto) is defined as the attention area R, and the “minimum value” is defined as the representative value. In the “local mode” shown in FIG. 3B, a very narrow area is defined as the attention area R, and any one of “average value”, “median value”, and “mode” is set as a representative value. Can be adopted. Further, in the “slit mode” shown in FIG. 3C, a rectangular area having a large aspect ratio (that is, a slit-like area) is defined as the attention area R. ”,“ Median ”, or“ mode ”can be employed. In the present embodiment, as shown in FIG. 3C, the region of interest R in the “slit mode” is configured such that its longitudinal direction is parallel to the epipolar line, and perpendicular to the epipolar line. It can also be configured to be inclined or inclined at an arbitrary angle with respect to the epipolar line. Switching of each mode is executed by the setting switching unit 27 controlling the attention area setting unit 22 and the representative value deriving unit 25 in response to an input from the setting switching unit 40 operated by the user. In the following description, the “local mode” and “slit mode” representative values are assumed to be “average values”.

  4A to 4C are conceptual diagrams for explaining a method of utilizing the three modes. The “wide area mode” shown in FIG. 4A can be used when it is desired to widely detect the presence of an obstacle. In the example illustrated in FIG. 4A, the attention area R includes an obstacle 52 (0.5 m ahead), an obstacle 54 (4 m ahead), and other backgrounds. In the “wide mode”, since “minimum value” is set as a representative value, a distance value (0...) That approximates the separation distance to the elongated bar-shaped obstacle 52 that is closest to the user in the visual field range. 5m) is derived as a representative value, and the output level is determined based on the distance value (0.5 m). As a result, the user can recognize the presence of the closest obstacle 52 within his visual field range together with the sense of distance. However, in the “wide area mode”, even if the obstacle is a rope or a wall, the output levels are equal if they are at the same distance as viewed from the user. Also, wherever the obstacle is in the visual field range, if they are at the same distance when viewed from the user, the output levels are equal. Therefore, in the “wide area mode”, the user cannot recognize what shape of an obstacle exists in which direction and in what manner.

  On the other hand, the “local mode” shown in FIG. 4B can be used when it is desired to accurately grasp the sense of distance to the object. In the example shown in FIG. 4B, the obstacle 54 (4 meters ahead) is reflected in most of the pixels in the attention area R. In the “local mode”, since “average value” is set as the representative value, a distance value (4 m) that approximates the separation distance from the user to the obstacle 54 is derived as the representative value, and the distance value (4 m) The output level is determined based on. As a result, the user can recognize the presence of the obstacle 54 on the optical axis of the main camera 12 together with the sense of distance. In addition, when the output signal generation unit 26 is configured to generate language speech (output signal) indicating the distance value corresponding to the representative value, the distance to the obstacle 54 is “4 m” in the “local mode”. Can be communicated to the user with a language voice.

  Conventionally, each time a white cane touches an object, the user recognizes the touch as a “point”, and recognizes the overall image of the object by integrating multiple captured “points” in time series. It can be said. In that sense, it can be said that the “local mode” of the present embodiment can be expected to be used in the same manner as a conventional white cane. However, the “local mode” of the present embodiment has an advantage over the conventional white cane. That is, in the case of a white cane, only an object within the distance range where the cane can reach can be recognized, whereas according to the “local mode” of the present embodiment, theoretically limited to the applicable distance range. There is no. Therefore, the user will be able to use the space recognition device 100 of the present embodiment as if it were a “stretchable white cane”.

  On the other hand, in the “slit mode” shown in FIG. 4C, the “average value” is set as the representative value, and therefore the “average value” of the distance values calculated from all the pixels in the attention area R is the output level. Is converted to Here, in the example shown in FIG. 4C, a part of the obstacle 52, a part of the obstacle 54, and other backgrounds are reflected in the pixels in the attention area R. The output level converted from the “average value” of the distance values calculated from all the pixels is obtained by leveling the sense of distance of the obstacle 52, the sense of distance of the obstacle 54, and the sense of distance of other backgrounds. It seems that there is no significance as information. However, according to the “slit mode” of the present embodiment, a more specific image (a sense of depth) of the obstacle can be provided to the user. This point will be described below with reference to FIGS.

  FIG. 5 shows the stereo imaging device 10 of the space recognition device 100 of the present embodiment. 5A shows the front surface of the stereo imaging device 10, and FIG. 5B shows the back surface of the stereo imaging device 10. In the example shown in FIG. 5, the stereo imaging device 10 is mounted using an elongated cylindrical casing 11 on the assumption that the user uses it with one hand. As shown in FIG. 5A, in the vicinity of both ends of the front surface of the housing 11, a main camera 12 and a sub camera 14 configured as an ultra-small 3CCD camera are installed in parallel equidistant positions with appropriate intervals. Further, a special surface processing 15 such as unevenness is applied to the central portion of the front surface of the housing 11, and when the user recognizes the surface processing 15 by touch, the front surface of the stereo imaging device 10 (that is, the imaging direction of the camera). ) Can be recognized.

  On the other hand, as shown in FIG. 5B, a vibrator 30 as an output device 30 is integrally fixed on the rear surface of the stereo imaging device 10, and in a state where the user holds the stereo imaging device 10, The vibration of the vibrator 30 can be sensed with a palm.

  Further, in the example shown in FIG. 5, the switch 40 as the setting switching unit 40 is formed integrally with the stereo imaging device 10. As shown in FIG. 5C, the user operates the switch 40 with his / her thumb as necessary, thereby performing the above-described “wide area mode”, “local mode”, “slit mode”, “distant view mode”, Various settings such as “near view mode” can be switched. Although this embodiment does not specifically limit the specific configuration of the switch 40, a visually handicapped person can confirm the current selection mode with a tactile sense using a multistage push switch, a universal switch, or the like. It would be preferable to configure.

  FIG. 6A is a diagram illustrating a usage mode of the space recognition device 100 that employs the stereo imaging device 10 illustrated in FIG. 5. In the example shown in FIG. 6A, the user is walking with the stereo imaging device 10 in the right hand and the information processing device 20 mounted on the waist belt. The stereo imaging device 10 and the information processing device 20 are connected by a signal line 16 and configured to be capable of bidirectional communication. The two-way communication between them may be realized by wireless communication. Furthermore, the stereo imaging device 10 and the information processing device 20 may be mounted and integrated in one housing.

  Note that the present invention is not limited to the above-described embodiment, and the output device 30 can be separated from the stereo imaging device 10. For example, as shown in FIG. It is also possible to attach the vibrator 30 to, for example. As shown in FIG. 6B, the output device 30 can be configured as a speaker device (for example, headphones), or a vibrator and a speaker device can be used in combination. Furthermore, as shown in FIG. 6B, an operation panel including a setting button with Braille attached to the housing of the information processing apparatus 20 can be formed as the setting switching unit 40.

  Next, the utility of the “slit mode” in the present embodiment will be described. There can be various objects in the living space that spreads in front of the user. Among them, the user particularly needs, for example, “pillar”, “wall”, “、”, “entrance”, “ It is information about "passage". If the user can perceive the sense of distance and depth of these objects with high accuracy, it will greatly help the user. The present inventor has come up with the “slit mode” described above, focusing on the fact that these objects have a spatial boundary in at least one of the horizontal direction and the vertical direction. That is, the “slit mode” is a mode capable of selectively detecting the presence of a spatial boundary extending in the horizontal direction or the vertical direction of an object, and the user can use this “slit mode” by using this “slit mode”. Therefore, a more specific spatial image can be acquired for the object. Hereinafter, this point will be described based on a specific example with reference to FIGS.

  FIG. 7 is a diagram illustrating a usage mode of the stereo imaging device 10 (shown in FIG. 5) in the “slit mode”. In FIG. 7, only the stereo imaging device 10 of the space recognition device 100 is shown for convenience of explanation. First, as shown in FIG. 7A, the user proceeds forward with the right hand holding the stereo imaging device 10 protruding in the traveling direction. At this time, by setting the space recognition device 100 to the “wide area mode”, the user can recognize that there is an obstacle in the traveling direction through the vibration of the vibrator 30.

  For example, as shown in FIG. 8, it is assumed that a narrow passage appears in the traveling direction of the user. In FIG. 8A, the field of view of the main camera set to the “wide area mode” is indicated by a solid thick frame, and the attention area R is indicated by a broken line frame. In the “wide area mode”, as shown in FIG. 8A, “wall”, “passage”, and “far view beyond the passage” are reflected in the attention area R. In the “wide area mode”, the “minimum value” of the distance value calculated for each pixel is derived as a representative value, so the distance value corresponding to the “wall” that is the obstacle closest to the user is the representative value. This is converted into vibration of the vibrator 30. The user stops by recognizing that there is some obstacle ahead due to the vibration.

  However, at this point, the user cannot know the details of the obstacle mode. Therefore, the user operates the setting switching unit 40 to switch the space recognition device 100 to the “slit mode”. In FIG. 8B, the field of view of the main camera set to the “slit mode” is indicated by a solid thick frame, and the attention area R is indicated by a broken line frame. Here, it should be noted that the longitudinal direction of the region of interest R in the “slit mode” corresponds to the longitudinal direction of the stereo imaging device 10. Thus, by making the longitudinal direction of the attention area R correspond to the longitudinal direction of the stereo imaging device 10, the user can intuitively recognize the direction of the “slit” in the “slit mode”.

  As shown in FIG. 7B, the user who has switched the space recognition device 100 to the “slit mode” moves the stereo imaging device 10 up and down in a state where the longitudinal direction of the stereo imaging device 10 is parallel to the horizontal direction. FIG. 9A shows the attention area R of a plurality of main images captured in the meantime in a time series with circled numbers 1-7. In FIG. 9A, the field of view of the main camera is omitted, and only the attention area R is indicated by a broken line frame (the same applies to FIGS. 10 to 14). On the other hand, FIG. 9B shows a time-series change in the output level (%) of the vibrator 30. In the example illustrated in FIG. 9, the space recognition apparatus 100 should be referred to as being set in the “distant view mode” described above (the same applies to FIGS. 10 and 11).

  In the “slit mode”, an average value of the distance values calculated for each pixel constituting each region of interest R is calculated, and the vibrator 30 vibrates at an output level corresponding to the average value. Since the contents reflected in the attention area R indicated by the numbers 1 to 7 are not significantly different from each other as shown in FIG. 9A, the average values derived from the attention areas R are almost equal. As a result, as shown in FIG. 9B, the output level of the vibrator 30 hardly changes in time series. However, the average value in the “slit mode” in this case is slightly larger than the minimum value obtained in the “wide area mode” shown in FIG. 8A because the “far view beyond the passage” is reflected. Therefore, the vibration level felt by the user is slightly weaker than that in the “wide area mode”. The user recognizes that there is an obstacle with some depth rather than the whole wall spreading forward, due to the temporal change of this sensation.

  Next, as shown in FIG. 7C, the user rotates the hand by 90 ° while keeping the space recognition device 100 in the “slit mode” so that the longitudinal direction of the stereo imaging device 10 is parallel to the vertical direction. Then, the stereo imaging device 10 is moved left and right. FIG. 10A shows the attention area R of a plurality of main images captured in the meantime in circles with numerals 1 to 5 in a time series. As shown in FIG. 10A, the contents reflected in each region of interest R change greatly in time series. Along with this, as shown in FIG. 10B, the output level of the vibrator 30 largely changes in time series. The user estimates from the time-dependent change in the vibration level of the vibrator 30 that there are two walls in front and that a depth of at least 4 m extends between the two walls. That is, the user can recognize that there is a narrow passage ahead.

  As shown in FIG. 7D, the user proceeds in the passage while moving the stereo imaging device 10 in the same manner as shown in FIG. 7D while holding the space recognition device 100 in the “slit mode”. FIG. 11A shows the attention areas R of a plurality of main images captured in the meantime in a time series with circled numbers 1-8. As shown in FIG. 11 (a), the output level of the vibrator 30 changes in time series as shown in FIG. 11 (b) in accordance with the change in the contents reflected in each region of interest R. The user can recognize the width of the passage from the change in the vibration level of the vibrator 30 with time, and can advance safely.

  Furthermore, further utility of the “slit mode” will be described with reference to FIGS. 12 to 14. In the examples illustrated in FIGS. 12 to 14, the space recognition device 100 should be referred to as being set to the “near view mode” described above.

  For example, it is assumed that an obstacle as shown in FIG. 12 appears in the traveling direction of the user. In this case, even if the user moves the stereo imaging device 10 left and right as shown in FIG. 12A, the user moves the stereo imaging device 10 up and down as shown in FIG. 12B. Even in this case, the output level of the vibrator 30 (that is, the vibration level felt by the user) does not change with time. Based on this, the user can recognize that the whole wall is facing forward.

  Next, it is assumed that an obstacle as shown in FIG. 13 appears in the traveling direction of the user. In this case, when the user moves the stereo imaging device 10 left and right, the output level of the vibrator 30 (vibration level felt by the user) does not change with time, as shown in FIG. When the imaging apparatus 10 is moved up and down, the output level of the vibrator 30 (vibration level felt by the user) changes with time in the manner shown in FIG. The user has a steeply sloped slope forward based on the difference between the temporal change in the sensation felt when moving the stereo imaging device 10 left and right and the temporal change in the sensation felt when moving up and down. I can recognize that.

  Next, it is assumed that an obstacle as shown in FIG. 14 appears in the traveling direction of the user. In this case, when the user moves the stereo imaging device 10 to the left or right, the output level of the vibrator 30 (vibration level felt by the user) does not change with time as shown in FIG. When the imaging apparatus 10 is moved up and down, the output level of the vibrator 30 (vibration level felt by the user) changes with time in the manner shown in FIG. What should be noted here is the difference between the temporal change in the output level shown in FIG. 14B and the temporal change in the output level at the steep slope shown in FIG. 13B. . The user can recognize that the obstacle standing ahead is not a steep slope but a staircase by detecting the difference between the two temporal changes. Let's go.

  As described above, according to the “slit mode” of the present embodiment, the user is based on the temporal change of the vibration level sensed by tactile sense as the stereo imaging device 10 is moved up and down and left and right. The presence of a spatial boundary extending in the horizontal direction or the vertical direction can be recognized together with the sense of distance, and as a result, the space can be recognized more accurately.

  Although the present invention has been described with the embodiment, the present invention is not limited to the above-described embodiment. In the embodiment described above, a configuration has been described in which a plurality of setting modes are defined in advance for the shape and size of the region of interest, and this is switched as appropriate. In the present invention, a dial type or the like is provided as another embodiment. By adopting the setting switching means, it is possible to adjust the height and width of the attention area. Furthermore, in the above-described embodiment, a configuration has been described in which a plurality of setting modes are defined in advance for the range of representative values to which the output range of the output device is assigned, and this is switched as appropriate. It is also possible to adopt a means to freely set a range of representative values. In addition, it is included in the scope of the present invention as long as the effects and effects of the present invention are exhibited within the scope of embodiments that can be considered by those skilled in the art.

DESCRIPTION OF SYMBOLS 10 ... Stereo imaging device 11 ... Housing 12 ... Main camera 14 ... Sub camera 15 ... Surface processing 16 ... Signal line 20 ... Information processing device 21 ... Image input interface 22 ... Attention area setting part 23 ... Stereo matching part 24 ... Distance Value calculating unit 25 ... representative value deriving unit 26 ... output signal generating unit 27 ... setting switching unit 30 ... output device (vibrator)
40. Setting switching means (switch)
52, 54 ... Obstacle 100 ... Space recognition device for visually impaired

Claims (12)

A spatial recognition device for a visually impaired person including a stereo imaging device including a main camera and a sub camera, an information processing device, and an output device,
The information processing apparatus includes:
A region-of-interest setting unit that defines a pixel region of a part of the main image captured by the main camera as a region of interest;
A stereo matching unit that performs stereo matching with a sub-image captured by a sub-camera for each pixel constituting the region of interest, and detects pixels in the sub-image corresponding to the pixels of the region of interest;
A distance value calculation unit that calculates a distance value of each pixel of the region of interest based on a parallax calculated from the coordinates of the pixel in the corresponding sub-image detected with each pixel of the region of interest;
A representative value deriving unit for deriving a statistical representative value for the plurality of calculated distance values;
An output signal generator that generates an output signal with a higher output level as the representative value becomes smaller;
A space recognition device for a visually impaired person including an output device that is driven based on the output signal.   The space recognition device for a visually impaired person according to claim 1, wherein the region of interest is defined as a slit-like region, and the representative value is selected from the group consisting of an average value, a median value, and a mode value.   The space recognition device for a visually impaired person according to claim 1, wherein the attention area is defined as a narrow area, and the representative value is selected from the group consisting of an average value, a median value, and a mode value.   The space recognition apparatus for a visually impaired person according to claim 1, wherein the attention area is defined as the entire area of the main image, and the representative value is a minimum value.   A slit mode in which the region of interest is defined as a slit-like region and the representative value is selected from the group consisting of an average value, a median value, and a mode value, the region of interest is defined as a narrow region, and the representative value is an average value and a median value And a local mode selected from the group consisting of the mode values, and a setting switching unit for switching three settings consisting of a wide area mode in which the region of interest is defined as the entire area of the main image and the representative value is the minimum value The space recognition device for a visually impaired person according to claim 1, further comprising:   The space recognition device for a visually impaired person according to claim 1, wherein the attention area is defined such that a center of gravity thereof coincides with an optical axis of the main camera.   The space recognition device for a visually impaired person according to any one of claims 1 to 6, wherein the output device is a vibrator.   The space recognition device for a visually impaired person according to claim 7, wherein the vibrator is a vibrator or a speaker device.   The space for visually impaired persons according to any one of claims 1 to 8, wherein the stereo imaging device and the output device are integrally mounted in a single portable case. Recognition device.   9. The stereo imaging device, the output device, and the information processing device are integrally mounted in a single portable case that can be carried with one hand. Space recognition device for the visually impaired. A computer-executable method for causing a computer to drive an output device to allow a user to recognize space, the method comprising:
A functional unit that defines a pixel region of a part of a main image captured by the main camera of a stereo imaging device including a main camera and a sub camera as a region of interest
A function unit that performs stereo matching with a sub-image captured by a sub-camera for each pixel constituting the region of interest, and detects pixels in the sub-image corresponding to the pixels of the region of interest;
A functional unit that calculates a distance value of each pixel of the region of interest based on a parallax calculated from the coordinates of the pixel in the corresponding sub-image that is detected and the pixel of the region of interest; Functional means for deriving a statistical representative value for a plurality of the distance values;
Functional means for generating an output signal having a larger output level as the representative value becomes smaller;
Functional means for driving the output device based on the output signal;
How to realize.   A computer-executable program for realizing each functional unit according to claim 11.