JP6063274B2 - Tactile presentation control device and tactile presentation control program - Google Patents

Description

  The present invention relates to a haptic presentation control apparatus and a haptic presentation control program, and more particularly to a haptic presentation control apparatus and a haptic presentation control program that improve user recognition accuracy for an object to be presented.

  Conventionally, a tactile means for presenting an object such as a GUI (Graphical User Interface) menu or chart to a user such as a visually impaired person is known. For example, there is a first method in which a large number of small piezoelectric actuators are two-dimensionally mounted, a screen drawn with a bitmap or the like is binarized, and presented on a tactile display in which static unevenness corresponding to the screen is formed. (For example, refer nonpatent literature 1). In this tactile display, the stimulation pins are driven up and down according to the layout information of the screen, and the user touches the object represented by the unevenness to recognize the layout and shape of the screen.

  As another tactile means, there is a second method in which an object constituting a screen in a markup description language is vibrated at a different frequency for each object (see, for example, Non-Patent Document 2). In addition, when presenting two-dimensional information using a tactile display or a tactile diagram, there is a third method for detecting a hue and reading out character information set in advance corresponding to the hue by speech synthesis (for example, Non-Patent Document 3).

KGS, “Dot View (registered trademark)”, http: // www. kgs-jpn. co. jp / b_dv2. html Sakai, Handa, Shimizu, “Tactile presentation method and content presentation method capable of different frequency vibration stimulation in object units”, IEICE, IEICE Technical Report WIT 2011-91 (2012-3) Uchida, Yamamoto, Shimada, Shinohara, Shimojo, Shimizu, "Transmission of tactile pattern information by voice support", Proceedings of the 33rd Sensation Agency Symposium, 13 (2007)

  However, in the first method described above, the image information is binarized, and the stimulation pin of the tactile display corresponding to the binarized level of each pixel is driven to be uneven so that the contour and area of the image are tactile. The method of making a different presentation with respect to an object indicating an important position or content in an image is not considered.

  In addition, the second method described above presents an object having a fixed shape described in a markup language by vibration, but has an arbitrary shape corresponding to an important position or content in an image. In order to vibrate the area, it is necessary to subdivide the objects, and to set the vibrations for each of the subdivided objects, and it is difficult to improve the user's recognition accuracy realistically.

  The third method described above is a method of detecting the hue of an image displaying content and converting character information set in advance corresponding to the hue into sound. In order to output the character information corresponding to the voice, it is necessary to arrange various colors in the image. Further, in the third method, different audio information cannot be given when there are a plurality of portions of the same color, and there is a limit to improving the user's recognition accuracy.

  Therefore, an object of the present invention is to provide a haptic presentation control apparatus and a haptic presentation control program for improving the recognition accuracy of a user for an object to be presented.

According to another aspect of the present invention, there is provided a tactile sensation presentation control apparatus that provides a tactile sensation to a user with a tactile sensation presentation unit in which a plurality of stimulation pins are arranged to recognize content by tactile sensation. Color area detecting means for detecting different color areas from an object, and a stimulus type corresponding to each color information obtained by the color area detecting means, and data for converting the data by including the set stimulus type in the color information A stimulus for extracting the stimulus type from the converted color information obtained by the conversion means and the data conversion means, and generating stimulus information for driving the stimulus pin of the tactile sense presenting means corresponding to the extracted stimulus type object extracting information generating means, the objects included in the markup description information corresponding to the content Comprises means out, and stimulation information corresponding to the color information, and stimulating the priority control means for controlling the priority of the stimulation information per object obtained by the object extraction unit.

The tactile sense presentation control program according to one aspect of the present invention is a tactile sense control program for causing a computer to execute a tactile sense presentation control process in which a user is stimulated by a tactile sense presenting means in which a plurality of stimulus pins are arranged to recognize content by tactile sense. In the presentation control program, the computer sets and sets a stimulus type corresponding to each color information obtained by the color area detecting means for detecting different color areas from the object of the image data included in the content, and the color area detecting means. Data conversion means for performing data conversion by including the stimulus type in the color information, extracting the stimulus type from the converted color information obtained by the data conversion means , and presenting the tactile sense corresponding to the extracted stimulus type stimulation information generating means for generating stimulation information for driving the stimulus pin means, said con Object extraction means for extracting an object included in markup description information corresponding to an item, stimulus priority for controlling the priority of stimulus information according to the color information and stimulus information for each object obtained by the object extraction means Function as a degree control means .

  ADVANTAGE OF THE INVENTION According to this invention, the recognition accuracy of the user with respect to the object to show can be improved.

It is a figure which shows the schematic structural example of the tactile sense presentation control apparatus in 1st Embodiment. It is a figure which shows the example of data conversion from hue information to vibration information. It is a figure which shows an example of the image data acquired by an image data acquisition means. It is a figure which shows the example of a schematic structure of the tactile sense presentation control apparatus in 2nd Embodiment. It is a figure for demonstrating the method of performing a vibration presentation locally in 2nd Embodiment. It is a figure which shows the example of a schematic structure of the tactile sense presentation control apparatus in 3rd Embodiment. It is a figure which shows an example which vibrates and displays the display area except a non-image area. It is a figure which shows an example which vibrates and displays the display area when the object containing a non-image area | region part overlaps. It is a flowchart which shows an example of a tactile sense presentation control process.

<About the present invention>
The present invention controls, for example, a tactile display (tactile presentation means) that presents two-dimensional information in a tactile sense with unevenness or vibration of a stimulation pin, and for example, GUI menus and charts used in contents such as digital broadcasts and web pages In the case of presenting an object, an individual object or a local object corresponding to an important position of image data such as a menu item attribute (for example, genre), a landmark on a map, a position of earthquake / tsunami information, etc. By driving and presenting the region at different vibration frequencies, the recognition accuracy of the object of a user such as a visually handicapped person can be improved, and the presentation information can be acquired quickly and efficiently.

  In the present invention, the stimulus pins arranged in two dimensions of the tactile presentation device are vibrated at different frequencies in units of objects described in a markup language or the like under the control of the tactile presentation control device, and the color for each object Information or the like is detected, and the detected information is converted into a preset vibration frequency to vibrate at a different frequency. As a result, it is possible to present a tactile sensation with locally different vibrations based on a predetermined condition (for example, priority) for an object in which object information described in markup and color information are mixed. In addition, the user in this embodiment is not limited to a visually impaired person, for example, All persons who use a tactile sense presentation control apparatus are included.

  Below, the form which implemented suitably the tactile sense presentation control device and the tactile sense control program is explained in detail using a drawing etc.

<Outline Configuration Example of Tactile Presentation Control Device in First Embodiment>
FIG. 1 is a diagram illustrating a schematic configuration example of the tactile sense presentation control apparatus according to the first embodiment. In FIG. 1, the tactile sense presentation control device 10 includes an image data acquisition unit 11, a storage unit 12, a color area detection unit 13, a data conversion unit 14, a vibration data generation unit (stimulus information generation unit) 15, a display. Means 16. Further, in the example of FIG. 1, the haptic presentation unit 17 controlled by the haptic presentation control device 10 is externally connected. However, the present invention is not limited to this, and is integrally provided in the haptic presentation control device 10. May be.

  The image data acquisition unit 11 inputs content to be displayed on the display unit 16 or presented by the tactile sense presentation unit 17 and acquires image data included in the input content. Here, the content is a data group including images, videos, character information, and the like such as digital broadcasts and Web pages, but is not limited thereto. The image data is, for example, a menu such as a GUI or a chart, but is not limited thereto. As the image data, data created in a format such as jpeg, gif (registered trademark), bmp, or tiff can be used, but the format type is not limited to this.

  The content can be acquired from an external device such as a LAN (Local Area Network) or the Internet from an external device, acquired from an external storage device such as a memory card, or the content stored in the storage unit 12 in advance. However, the present invention is not limited to this. For example, content may be generated in the haptic presentation control apparatus 10 and the generated content may be used.

  The storage unit 12 stores various information necessary for the present embodiment. For example, the storage unit 12 stores content to be displayed on the screen of the display unit 16 or presented to the tactile sense presenting unit 17. The storage unit 12 stores a conversion table for converting the color information into the vibration information in the data conversion unit 14 or stores the arrangement information of the stimulation pins in the tactile sense presentation unit 17, but is not limited thereto. Other setting information, execution results of each process, and the like are stored. In addition, the storage unit 12 can read various stored information as needed, and write processing results and the like.

  The color area detection unit 13 detects color information for each object corresponding to the image data obtained by the image data acquisition unit 12. For example, the color area detection unit 13 may acquire color information for each pixel included in the image data, or may detect a plurality of pixels as one area and detect the color information for each area.

  Here, the color information includes, for example, hue, luminance, saturation, saturation, and the like, but is not limited thereto. In the following description, hue information is used as an example of color information. For example, the color area detection unit 13 detects areas having different hues by hue analysis or the like for each pixel using a preset hue ring diagram or the like.

  The data conversion means 14 converts the hue information (color information) of each object detected by the color area detection means 13 into preset stimulus information. The stimulus information is stimulus information presented to the user by the tactile sense presenting means 17 in which a plurality of stimulus pins are arranged. As the stimulus information, for example, at least one of physical stimulus information such as vibration stimulus, electrical stimulus, temperature stimulus, and pressure stimulus is used, but the stimulus information is not limited to this. In the following description, vibration is used as an example of stimulus information. The data conversion means 14 converts the vibration frequency corresponding to each hue included in the image data.

  Note that the data conversion means 14 is a format format having separately hue information and luminance information for color information (for example, RGB, etc.) of image data having various format formats such as jpeg, gif, png, and bmp. (For example, it may be converted into HSV, HLS). When the format format before conversion is already the HSV format or HLS format, the above-described format conversion is not performed. A specific example of processing in the data conversion unit 14 will be described later.

  The vibration data generation means 15 generates stimulus information (vibration data) for giving a stimulus to the user using the tactile sense presentation means 17 based on the vibration frequency of each region obtained by the data conversion means 14.

  In general, the number of pixels corresponding to the resolution of the image data is different from the number of stimulation pins arranged two-dimensionally in the tactile sense presentation control device 15, and the number of stimulation pins is smaller than the number of pixels. Therefore, the vibration data generation means 15 performs pixel dropping by associating the resolution of the image data obtained by the image data acquisition means 11 with the number of stimulation pins of the tactile sense presentation means 17 stored in advance in the storage means 12 or the like. Do. Pixel dropping is a process of associating a plurality of pixel areas with one stimulation pin, for example, but is not limited to this. The vibration data generation means 15 sets the vibration frequency (cycle) for each stimulation pin.

  The display means 16 displays content or data including vibration information converted by the data conversion means 14 on the screen. The display means 16 includes, for example, a monitor and a display, but is not limited thereto. The display unit 16 shown in FIG. 1 is provided inside the tactile sense presentation control apparatus 10, but is not limited to this, and is connected via a communication network represented by a LAN or the Internet, for example. It may be a remote display means.

  The tactile sense presenting means 17 has a plurality of stimulation pins arranged two-dimensionally at a predetermined interval in the display area, and the tactile sense using a user's finger or the like is formed by moving the stimulation pins up and down to form irregularities on the surface. Realize the presentation. The tactile sense presenting means 17 is, for example, a tactile display and can use, for example, a dot view of KGS, but is not limited thereto.

  The tactile sense presenting means 17 has a function of presenting the above-described stimulus information. For example, in the case of presentation of vibration, vibration can be presented by controlling the concave / convex period of the corresponding stimulation pin based on the vibration frequency set for each hue. In the case of electrical stimulation, for example, electrical stimulation can be presented by applying a predetermined amount of current to the corresponding stimulation pin at predetermined intervals. In the case of temperature stimulation, the heat stimulation can be presented by transmitting heat from the heating element or the like to the corresponding stimulation pin. In the case of pressure stimulation, stimulation by pressure can be presented by adjusting the pressing force and the height of the stimulation pin when the corresponding stimulation pin is changed from the concave state to the convex state.

  For example, a PC (Personal Computer), a server, or a tablet terminal can be used as the haptic presentation control apparatus 10 described above, but the present invention is not limited to this.

<Data Conversion Example in Data Conversion Unit 14>
Next, an example of data conversion from color information to stimulus information in the data conversion means 14 described above will be specifically described. FIG. 2 is a diagram illustrating an example of data conversion from hue information to vibration information.

  In the first embodiment, color information (for example, hue (H)) for each pixel is detected from the acquired image data. The HSV color information is H (hue), V (saturation), and S (saturation), and each pixel has 8-bit data.

  In the first embodiment, the hue circle diagram 21 as shown in FIG. 2A is divided into arbitrary hue ranges, and a code for identifying each divided hue range is assigned. The hue data corresponds to 0 to 360 ° in the hue circle diagram 21. For example, as shown in the parentheses in FIG. 2A, the value is counted every 2 ° and the value is 0 to 180 °. You may associate.

  In the first embodiment, a hue-vibration conversion table 22 as shown in FIG. 2B is set in advance, and vibration information is assigned to each code in the hue range. The items of the hue-vibration conversion table 22 are, for example, “hue (°)”, “tag attribute value”, “identification code”, “vibration (Hz)”, but are not limited thereto. The tag attribute value indicates, for example, color attribute information corresponding to the hue range. The vibration is information for identifying a frequency (period) for vibrating the stimulation pin included in the tactile sense presenting means 17 for example, but is not limited to this, and the frequency is stored as it is. May be. In the first embodiment, a predetermined vibration frequency is set from the code corresponding to the hue of each pixel using the hue-vibration conversion table 22.

  For example, in the hue-vibration conversion table 22 shown in FIG. 2B, by setting the type of vibration frequency to 8 levels using 3 bits, vibrations f0 to f7 are applied to the hue range divided into eight. Can be set. In this case, the vibration frequency to be converted is set to a low frequency that is relaxed for warm colors such as red, and is set to a high frequency for cold colors such as blue. Although it is preferable to set the frequency according to sensitivity, it is not limited to this.

  In the hue-vibration conversion table 22 shown in FIG. 2B, the vibration information is set for all the hue ranges, but the present invention is not limited to this, and only for some hue ranges. Vibration information may be set.

  Furthermore, in the first embodiment, as shown in FIG. 2C, the display data for displaying on the display means 16 and the tactile sense for the hue information (8 bits) of one pixel included in the image data. Vibration data for vibrating the stimulation pin of the presentation means 17 is set. Specifically, as shown in FIG. 2C, when the hue (tag attribute value) of a certain pixel portion is Blue (blue), the data conversion unit 14 displays the upper 5 bits of the hue information (8 bits). The bits are used as display data as they are, and the lower 3 bits are replaced with a code obtained from the hue-vibration conversion table 22 in FIG.

  That is, in the first embodiment, the lower 3 bits of the hue information can be held as vibration data, and the hue information as shown in FIG. 2C is used to vibrate the stimulation pin in the tactile sense presenting means 17. In this case, the corresponding vibration can be generated using the lower 3 bits and the user can be presented by tactile sensation. Specifically, when the lower 3 bits of the hue information are, for example, the code “101”, the corresponding vibration frequency “f5” can be acquired from the hue-vibration conversion table 22 to vibrate the stimulation pin.

  Further, by using the upper 5 bits of the hue information, the display unit 16 can visually display the image data included in the content to the user. In the first embodiment, the vibration data is included in the hue information, which leads to compression of the transmission capacity.

  In the first embodiment, since the vibration data is included in the hue information, the actual hue is different from the value, and there is a possibility that the color reproduction is slightly distorted due to the influence of the lower 3 bits. However, since the upper 5 bits remain as the original hue information, a large error is not seen, and the display unit 16 or the like can display an image with no sense of incongruity.

  In order to reduce the influence of color distortion, when displaying on the display means 16 using hue information, the lower 3 bits of all 8 bits are corrected to a preset intermediate value. May be displayed. For example, when the lower 3 bits are “000” or “111”, they are corrected to “011”, “100”, or the like. As a result, color reproduction errors can be dispersed.

  Furthermore, in the first embodiment, after all the pixels in the image data are converted, the vibration data generating unit 15 applies the same color to the objects in the image as shown in FIG. Pixel dropping according to the number of stimulation pins of the tactile sense presenting means 17 is performed. In the first embodiment, data that vibrates at each frequency assigned to each stimulation pin is created. The stimulation pin vibration pattern data created by the vibration data generation means 15 drives each stimulation pin of the tactile sense presentation means 17.

  In the above example, the display data is the upper 5 bits and the vibration data is the lower 3 bits. However, the present invention is not limited to this. For example, the lower 2 bits and the lower 4 bits are used as the vibration data. Good. For example, when the lower 2 bits are used as vibration data, the above-described hue circle diagram 21 can be divided into a maximum of four types of hue ranges.

  In the hue data, the lower the number of bits to be replaced and the smaller the number of bits to be replaced, the more display information can be displayed on the display means 16 without changing the display data.

  Further, although the vibration data is set in the lower bits of the hue information, the present invention is not limited to this, and may be set using upper bits, for example.

  As described above, in the first embodiment, the object to be presented is detected by detecting the hue of each object included in the image data, and changing the same hue area into vibration and presenting it with the tactile sense presenting means 17 (tactile display). The recognition accuracy of the user can be improved, and the arrangement and contents of the object can be acquired quickly.

<Example of image data>
Here, FIG. 3 is a diagram illustrating an example of image data acquired by the image data acquisition unit. In the image data shown in FIG. 3, FIGS. 3A and 3B show examples of image data included in the content, and FIGS. 3C to 3E show markup description information. 2 shows an example of content that the user has. The markup description information is information in which object information, layout information, and the like corresponding to image data included in the content are described in a predetermined markup language, and is generated separately from the content.

  In the examples of FIGS. 3A, 3C, and 3E, Hokkaido image data (map information) is shown as the background image of the object, and the object for a predetermined region (region) in Hokkaido. Information is displayed in different colors and shapes. In the example of FIG. 3C, each object information (Obj1 to Obj3) displayed in different colors and shapes is described in a markup language. In the example of FIG. 3E, object information (Obj1 to Obj5) indicated by areas of different colors and object information (Obj6) indicated by a display range (frame) are described in a markup language. ing.

  In addition, in the examples of FIGS. 3B and 3D, image data of a Japanese map is shown, and each regional division (for example, Hokkaido, Tohoku, Kanto, Chubu, Kinki, China, Shikoku, Kyushu, (Okinawa) is displayed in different colors. 3D, which is visually the same as FIG. 3B, the object information (Obj1 to Obj5) shown in FIG. 3D is described in a markup language.

  For example, in the examples of FIG. 3A and FIG. 3B, the objects sorted based on the hue range can be presented with different vibration information. Further, in the examples of FIGS. 3C to 3E, information can be added to an object portion described in a markup language in addition to the objects classified by color.

  In the first embodiment, a region including color information is detected from each piece of image data as shown in FIGS. 3A to 3E and is presented by the tactile sense presenting means 17 based on the detected color information and the like. By making the vibration of the object part different, the user's recognition accuracy for the object to be presented can be improved.

<Outline Configuration Example of Tactile Presentation Control Device in Second Embodiment>
Next, a tactile sense presentation control apparatus according to the second embodiment will be described with reference to the drawings. FIG. 4 is a diagram illustrating a schematic configuration example of the tactile sense presentation control apparatus according to the second embodiment. In addition, in the example of FIG. 4, the same code | symbol shall be attached | subjected about the function structure similar to the tactile sense presentation control apparatus 10 mentioned above, and specific description here is abbreviate | omitted.

  FIG. 5 is a diagram for explaining a technique for locally presenting vibrations in the second embodiment. In the example of FIG. 5, FIG. 5A shows an example of markup description information, and FIG. 5B is a diagram showing a synthesis example of presentation information.

  In FIG. 4, the tactile sense presentation control device 30 includes an image data acquisition means 11, a storage means 12, a color area detection means 13, a data conversion means 14, a display means 16, an object extraction means 31, and a layout generation means. 32, vibration object extraction means 33, vibration data generation means 34, and vibration priority control means (stimulus priority control means) 35. In the example of FIG. 4, the haptic presentation unit 17 controlled by the haptic presentation control device 30 is externally connected. However, the present invention is not limited to this, and is integrally provided in the haptic presentation control device 30. May be.

  In the second embodiment, for example, the image data as shown in FIGS. 3C to 3E described above and an object described in a markup language are mixed, and locally in overlapping contents. Make a vibration presentation.

  The object extraction unit 31 inputs, for example, markup description information corresponding to the content input to the image data acquisition unit 11, and extracts an object included in the content from the input markup description information. Further, the object extraction means 31 may extract position information, shape, size and the like of each object.

  Here, in the markup description information, information such as the layout and vibration conditions of each object included in the content is described. Specifically, as shown in FIG. 5A, object identification information (<No>), object shape (<figure>), vibration condition (vibration frequency identification information) (<vib>), and the like are described. However, the contents to be described are not limited to this, and may include information such as position information and size of the object, for example. In addition, since the markup description information described above is generated separately from the content, for example, the content creator may generate the markup description information, or a third party who viewed the content may create the tactile presentation that acquired the content. Although the user of the control apparatus 30 may create, it is not limited to this.

  The layout generation unit 32 generates a screen layout for the object from the position information of each object extracted by the object extraction unit 31.

  The vibration object extraction unit 33 extracts an object to be presented by vibration from the objects extracted by the object extraction unit 31. Moreover, the vibration object extraction means 33 extracts the vibration conditions set for the extracted object.

  The vibration data generation means 34 drives the stimulation pin drive data of the tactile sense presentation means 17 that vibrates each of the image data and the object in the content in which the image data included in the content and the object included in the markup description information are mixed. Is generated.

  The vibration priority control unit 35 controls which condition is prioritized to vibrate the stimulation pin of the tactile sense presentation unit 17 when there is an overlap between the image data and the object.

  In the second embodiment, for example, in the case of content as shown in FIGS. 3C to 3E, vibration control by tactile sensation is performed on the overlapping display portion of the image data and the object included in the markup description information. Do.

  The signal at the point (a) in FIG. 4 output from the data conversion unit 14 in FIG. 4 is f1 to f4 with respect to regions having different hues in the image as shown in (a) in FIG. Different vibration information is set as follows. In the example of FIG. 5B, vibration information is set for a part of regions having different hues. This is a hue range corresponding to the hues in the hue-vibration conversion table 22 described above. This is because the vibration information is set only for.

  On the other hand, in the markup description information corresponding to the content (FIG. 5A), the object in the content is extracted by the object extraction means 31, and the screen layout of each object is generated by the layout generation means 32.

  In addition, the vibration object extraction means 33 extracts an object for which vibration conditions are set with a <vib> tag shown in FIG. The signal at the point (b) in FIG. 4 outputted from the vibration object extracting means 33 forms a vibration pattern with frequencies fa and fb as shown in (b) in FIG. 5 (B). Note that the fa and fb objects are generated based on the contents of the markup description information, and thus do not depend on the hue of the image data.

  Here, the layout generation data based on the image data and the markup description information shown in FIG. 5B is the tactile presentation stored in advance in the storage unit 12 in the vibration data generation unit 34 as in the first embodiment described above. Pixels are dropped according to the number of stimulation pins of the means 17. Further, the vibration priority control means 35 sets the final vibration frequency according to the priority of vibration presentation of the stimulation pin.

  Here, for example, the priority is preferably set to vibrate at a vibration frequency set for an important object displayed on the front surface of the screen by various methods, but is not limited thereto. For example, as a first priority control method, when the vibration frequency is set for the object included in the markup description information, the vibration by the tactile sense presentation unit 17 is the vibration condition for the object included in the markup description information. To give priority. For example, the signal at the point (c) in FIG. 4 output from the vibration priority control means 35 in FIG. 4 is image data (object) to vibrate at the vibration frequency f1, as shown in (c) of FIG. And the region of the object based on the markup description information to be vibrated at the vibration frequency fa. If the first priority control method is applied, the vibration frequency of the overlapping portion is fa.

  Further, as a second priority control method, when the vibration frequency of the object included in the markup description information is 0 Hz, or when the vibration frequency is not set in the markup description information, the vibration condition for the hue, that is, Priority is given to image data.

  As a third priority control method, for example, when an object included in the markup description information overlaps with image data, a tag (for example, <priority>) for specifying the priority is provided, and the setting value of the tag is set. It is a method to give priority according to. For example, when giving priority to the vibration condition set for the image data, “<priority> image data </ priority>” is described, but the description content is not limited to this.

  In the second embodiment, for example, all the vibration presentation areas are set based on the hue information of the image data, and all the objects included in the markup description information are set as non-image areas. An object included in the description information is not displayed as a tactile pattern, and a function of outputting a sound when the area of the object is touched can be provided.

  Here, the objects included in the markup description information often exist on the image data. Therefore, although vibration conditions between objects and vibration frequencies between hue information are set to vibration conditions that can be discriminated by tactile sense, vibrations generated from object vibration frequencies fa and fb and hue information of an image. It is preferable to set within a range in which the difference in vibration can be clearly distinguished even between the frequencies f1 to f4.

  Finally, the stimulation pin vibration pattern data generated by the vibration data generation unit 34 drives the corresponding stimulation pin of the tactile sense presentation unit 17 based on the vibration condition prioritized by the vibration priority control unit 35. As described above, in the second embodiment, when the image data and the object included in the markup description information coexist, the region to be preferentially presented with vibration is set, and the tactile sense providing means 17 is set with the set priority. By presenting, it is possible to quickly transmit important regions arranged in the image.

<Outline Configuration Example of Tactile Presentation Control Device in Third Embodiment>
Next, a tactile sense presentation control apparatus according to the third embodiment will be described with reference to the drawings. FIG. 6 is a diagram illustrating a schematic configuration example of the tactile sense presentation control apparatus according to the third embodiment. In FIG. 6, the functional components similar to those of the tactile sense presentation control device 30 in the second embodiment described above are denoted by the same reference numerals, and detailed description thereof is omitted here.

  FIG. 7 is a diagram illustrating an example in which the display area excluding the non-image area is presented by vibration. FIG. 7A shows an object included in the content and a description example of the vibration condition by the markup description information for the object, and FIG. 7B shows a diagram for explaining the vibration-free region. FIG. 8 is a diagram illustrating an example in which a display area is presented in a vibration manner when objects including non-image area portions overlap each other.

  In the third embodiment, as shown in FIG. 7A, in the object included in the markup description information included in the content, for example, a rectangle set in XY coordinates like the object (Obj2) in FIG. 7B. In the (rectangle) display area, vibration control is performed on a part of a non-image area that is normally displayed in a transparent state where no image is present, or a part where display areas of a plurality of objects overlap. Note that the portion of the non-image area that is normally displayed in a transparent manner is a hatched portion shown in (c) of FIG. A portion where the display areas of a plurality of objects overlap is a shaded portion shown in FIG.

  In the third embodiment, the tactile sense presentation control device 40 shown in FIG. 6 includes an image data acquisition unit 11, a storage unit 12, a color area detection unit 13, a data conversion unit 14, a display unit 16, and an object extraction unit. 31, layout generation means 32, vibration object extraction means 33, vibration priority control means 35, non-image area detection means (non-object area detection means) 41, and vibration data generation means 42. In the example of FIG. 6, the haptic presentation unit 17 controlled by the haptic presentation control device 40 is externally connected. However, the present invention is not limited to this, and is integrally provided in the haptic presentation control device 40. May be.

  In the tactile sense presentation control device 40, the non-image area detecting means 41 sets a rectangular area including an image object included in the image data using XY coordinates or the like. A non-existing non-image area portion is detected. For example, in the example of FIG. 7B, a non-image area of Obj2 is detected. However, the present invention is not limited to this. The non-image area detection unit 41 detects a non-image area portion for each object. To detect.

  A plurality of methods can be used as the detection method in the non-image area detection means 41. For example, in the first detection method, a non-image area is detected by extracting a code in which the non-image area is described with image data in a predetermined format (for example, gif or png).

  In the second detection method, for example, a non-image area pattern in which an image area portion and a non-image area portion are previously distinguished by binary data such as black and white is formed, and this pattern is formed as shown in FIG. Described in the markup description information as a non-object creation pattern. Then, the non-image area portion is detected by superimposing the non-object creation pattern and the object. Note that the non-image region detection method is not limited to the above-described method.

  Further, the vibration data generation means 42 in the third embodiment is for causing the stimulation pin provided in the tactile sense presentation means 17 to vibrate with respect to the display area excluding the non-image area detected by the non-image area detection means 41. Generate vibration data. Note that the vibration data generation means 42 performs the priority control of vibration in the vibration priority control means 35 after dropping pixels according to the number of stimulation pins of the tactile sense presentation means 17 as in the first and second embodiments. .

  The vibration priority control means 35 controls whether to vibrate the non-image area detected by the non-image area detection means 41 in addition to the priority control in the second embodiment described above, When the non-image area included in the area overlaps with the image area of another object, it is controlled whether or not the overlapping portion is vibrated.

  For example, when the non-image area is detected by the first detection method described above, the vibration priority control unit 35 controls the non-image area so as not to vibrate corresponding to the code of the image data. When the vibration priority control means 35 detects the non-image area by the second detection method described above, the image data object shown in FIG. 7B and the non-image area detection means 41 are detected. The non-image area obtained by superimposing the non-object creation pattern shown in (d) of FIG.

  Also, as shown in FIG. 8, when the rectangular area for Obj1 and the rectangular area for Obj2 overlap, the vibration data generating means 42 causes the frequencies fa and fb to vibrate in the areas in the object excluding the non-image areas, respectively. Generate vibration data. In the third embodiment, a tag (for example, the above-mentioned <priority> tag) for specifying the priority when there is an overlap between objects is provided, and the vibration data generating unit 42 sets a value set in the tag. Vibration data prioritized based on the above may be generated.

  In this case, if only the priority of the object is given, for example, if Obj1 has priority in FIG. 8, the image portion of Obj2 is made non-vibrating in the non-image area of Obj1. Therefore, the vibration priority control unit 35 gives priority to the vibration for the object overlapping with the non-image area in addition to the priority of the <priority> tag for the overlapping portion between the objects. Appropriate vibration can be presented.

  As described above, in the third embodiment, when a non-image area exists in the object of the image data included in the markup description information, the vibration of the non-image area is removed and the tactile sense providing unit 17 presents the image. It is possible to appropriately present only the portion where the. In addition, when the non-image area overlaps with another image area, the priority area is provided and vibration control is performed, so that an important area of the content can be quickly transmitted by touch. The first to third embodiments described above can be appropriately combined with a plurality of embodiments.

<Execution program (tactile presentation control program)>
Here, the haptic presentation control devices 10, 30, and 40 described above include, for example, a volatile storage medium such as a CPU (Central Processing Unit) and a RAM (Random Access Memory), and a nonvolatile storage such as a ROM (Read Only Memory). It can be configured by a computer provided with a medium, an input device such as a mouse, a keyboard, and a pointing device, a display means for displaying images and data, and an interface for communicating with the outside.

  Therefore, the above-described functions of the tactile sense presentation control devices 10, 30, and 40 can be realized by causing the CPU to execute a program describing these functions. These programs can also be stored and distributed in a recording medium such as a magnetic disk (floppy disk, hard disk, etc.), optical disk (CD-ROM, DVD, etc.), semiconductor memory, or the like.

  That is, an execution program (tactile presentation control program) for causing a computer to execute the processing in each configuration described above is generated, and the program is installed in, for example, a general-purpose PC, thereby realizing the above-described tactile presentation control process. be able to.

<Example of tactile sense presentation control processing>
Here, an example of the tactile sense presentation control process will be described using a flowchart. FIG. 9 is a flowchart illustrating an example of the tactile sense presentation control process.

  In the tactile sense presentation control process in the example of FIG. 9, the image data acquisition unit 11 acquires image data included in the input content (S01). Further, the color area detection means 13 detects a color area included in the image data (S02). In the process of S02, areas having different hues can be detected by, for example, hue analysis for each pixel using the hue circle diagram 21 or the like, but the present invention is not limited to this.

  Next, the data conversion means 14 converts the hue information obtained by the process of S02 into vibration information (S03). The vibration information converted by the process of S03 can be included in the lower bits of the hue data as described above. Next, the vibration data generation means 15 performs pixel dropping of the pixels and the stimulation pins in correspondence with the arrangement of the stimulation pins of the tactile sense presentation means 17, and generates vibration data for driving the corresponding stimulation pins (S04). .

  Further, the tactile sense presenting means 17 gives a predetermined vibration to a predetermined position of the stimulation pin based on the vibration data obtained from the vibration data generating means 15 and presents by tactile sense (S05).

  Here, in the tactile sense presentation control process, it is determined whether or not the content input in the process of S01 is displayed on the display means 17 (S06). When displaying (YES in S06), the content is displayed on the display means 17. It is displayed (S07). In the process of S07, hue data including vibration information can be used as the hue data corresponding to the image data included in the content. That is, it is possible to control the vibration for each object presented by the tactile sense presenting means 17 and the color information of the content displayed on the display means 17 using the hue data. After the process of S07 or when not displayed on the display means 17 in the process of S06 (NO in S06), the process is terminated.

  In the process of S05 described above, as shown in the second embodiment described above, the vibration condition is set in advance when the vibration condition is set for the object included in the markup description information corresponding to the input content. Alternatively, vibration control may be performed based on the priority condition, and presentation by the tactile sense presentation unit 17 may be performed.

  Further, in the above-described processing of S04, as shown in the above-described third embodiment, a non-image area is detected, and in the processing of S05, when an image area of a certain object overlaps with a non-image area of another object. Alternatively, vibration control may be performed based on preset priority conditions, and presentation by the tactile sense presenting means 17 may be performed.

  According to the above-described embodiment, the user's recognition accuracy for the object to be presented can be improved. In addition, the user can recognize the object quickly and reliably. For example, by converting the image data contained in the content into vibrations that differ for each color information, it is possible to quickly and appropriately transmit the position information such as maps and graphs and the contents of important areas all at once by touch. It is possible to present more effective two-dimensional information.

  Further, according to the present embodiment, when image data and an object included in markup description information are mixed, a region to be presented with vibration is determined according to the priority of both and converted into vibration, whereby the user Can quickly and properly recognize objects. Further, according to the present embodiment, when a non-image region exists in the object of the image data, the region excluding the non-image region part is converted into vibration and presented on the tactile display, so that the tactile sense can be used as a whole. Position information such as maps and graphs and contents of important areas can be transmitted quickly and appropriately, and effective two-dimensional information can be presented.

  Although the present embodiment has been described in detail above, the present invention is not limited to the specific embodiment, and various modifications and changes other than the above-described modifications are possible within the scope described in the claims. .

10, 30, 40 Tactile sense presentation control device 11 Image data acquisition means 12 Storage means 13 Color region detection means 14 Data conversion means 15, 34, 42 Vibration data generation means (stimulus information generation means)
16 Display means 17 Tactile sense presentation means 21 Hue ring diagram 22 Hue-vibration conversion table 31 Object extraction means 32 Layout generation means 33 Vibration object extraction means 35 Vibration priority control means (stimulus priority control means)
41 Non-image area detection means (non-object area detection means)

Claims (6)

In a tactile sensation presentation control device that causes a user to be stimulated by tactile sensation presentation means in which a plurality of stimulation pins are arranged to recognize content by tactile sensation,
Color area detecting means for detecting different color areas from the object of the image data included in the content;
A data conversion unit that sets a stimulus type corresponding to each color information obtained by the color area detection unit, includes the set stimulus type in the color information, and performs data conversion;
A stimulus information generating means for extracting the stimulus type from the converted color information obtained by the data converting means and generating stimulus information for driving a stimulus pin of the tactile sense presenting means corresponding to the extracted stimulus type; ,
Object extraction means for extracting an object included in markup description information corresponding to the content;
Stimulus priority control means for controlling the priority of the stimulus information corresponding to the color information and the stimulus information in units of objects obtained by the object extraction means;
A tactile sense presentation control device characterized by comprising: A tag that describes the priority of the stimulus information in the markup description information;
2. The tactile sense presentation control apparatus according to claim 1 , wherein the stimulus priority control means determines the stimulus type according to the priority set in the tag. Non-image area detecting means for detecting a non-image area that is an area other than image data among rectangular areas including objects,
The stimulus priority control means, the tactile presentation control apparatus according to claim 1 or 2 wherein the non-image areas are characterized by having a means for controlling so as not to present a stimulus. The stimulus priority control means includes
Wherein the non-image area, when the object area included in the markup description information overlaps, in claim 3, characterized in that priority is given to stimulate information included in the markup description information to the object region The tactile sense presentation control device described. The color information is color information, the stimulus information haptic presentation control apparatus according to any one of claims 1 to 4, characterized in that the vibration information of the stimulation pins. In a haptic presentation control program for causing a computer to execute a haptic presentation control process in which a user is stimulated by a tactile presentation means in which a plurality of stimulation pins are arranged and content is recognized by a tactile sense.
The computer,
Color area detecting means for detecting a different color area from an object of image data included in the content;
A data conversion unit that sets a stimulus type corresponding to each color information obtained by the color region detection unit, includes the set stimulus type in the color information, and performs data conversion ;
A stimulus information generating means for extracting the stimulus type from the converted color information obtained by the data converting means, and generating stimulus information for driving the stimulus pin of the tactile sense presenting means corresponding to the extracted stimulus type;
Object extraction means for extracting an object included in the markup description information corresponding to the content;
Stimulus priority control means for controlling the priority of the stimulus information corresponding to the color information and the stimulus information in units of objects obtained by the object extraction means;
Tactile presentation control program to function as