JP4173951B2 - Multisensory information transmitter and multisensory information receiver - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a multisensory information transmitting apparatus and a multisensory information receiving apparatus, and more particularly, to a multisensory information transmitting apparatus and a multisensory information receiving apparatus that transmit the state of an object included in an image to a user multisensory.
[0002]
[Prior art]
Conventionally, research has been conducted to transmit information to a user with a visual or auditory impairment using a sense other than visual or auditory sense.
[0003]
For example, as a sensory proxy for visually impaired users, “Optacon”, “TVSS” (“Visual impairment and its proxy technology” extracts contour information from images and transmits two-dimensional or three-dimensional information to tactile receptors. ”, Ichikawa et al., Nagoya University Press,“ Biomedical Engineering ”, Hoshimiya et al., Kyoritsu Publishing, etc.). These devices extract binarized information from image information obtained by photoelectric conversion, drive tactile stimulation elements arranged in two dimensions, and present a stimulus to a finger or back by vibration.
[0004]
In the 3D display, a tactile stimulation device using a stepping motor arranged in 2D was created by a computer, as in the “3D information display device for the blind” (medical welfare equipment technology R & D report). It is driven up and down corresponding to the object data and represents a pseudo three-dimensional shape.
[0005]
Furthermore, research on transmitting the three-dimensional shape of a virtual object to vision and tactile sensation has been made with recent progress of virtual reality. In the information transmission to the tactile sense, for example, there is one such as “PHAMTOM” (Sensable Technology Co., Ltd.) that touches and feels a three-dimensional shape, texture or the like with a finger. This device virtually reproduces a three-dimensional effect by feeding back spatial information created by a computer to a tactile receptor using a mechanical reaction force.
[0006]
[Problems to be solved by the invention]
In the field of broadcasting and communication, information is transmitted using visual and auditory media. Therefore, information is transmitted to the user through hearing as a medium for visually impaired users. However, no means has been established for tactile transmission of three-dimensional information such as maps and graphs in educational programs, or art objects, molecular models in educational programs, and topography. There was a limit to transmitting only. In addition, for a user who has a visual or auditory impairment, only the sense of touch is the only information acquisition means, and at present, information such as shape cannot be transmitted.
[0007]
On the other hand, a method of transmitting three-dimensional information and graphic information used in a conventional two-dimensional and three-dimensional tactile presentation device by digital data broadcasting is also conceivable. However, it is difficult to secure a sufficient transmission capacity with a limited broadcast transmission capacity. In addition, 2D and 3D tactile presentation devices can touch solids and figures based on spatial and planar position information, but users with visual impairments can understand details of solids and figures simply by touching them. There was a problem that it was difficult to do.
[0008]
The present invention has been made in view of the above points, and can transmit graphic information or three-dimensional information of an object included in an image to a tactile sensation presentation device, so that the state of the object included in the image can be sensed by the user. It is an object of the present invention to provide a multisensory information transmitting apparatus and multisensory information receiving apparatus that can be transmitted to a mobile phone.
[0009]
[Means for Solving the Problems]
Therefore, in order to solve the above-described problem, the multisensory information transmission device of the present invention is a multisensory information transmission device used in a multisensory information transmission system that multisenses the state of an object included in an image to a user. And a state information generation unit that generates state information of an object included in the image, and a state information transmission unit that transmits the generated state information.
[0010]
As described above, the state information of the object included in the image is generated, and the generated state information is transmitted to the receiving side separately from the image information, so that the state of the object included in the image is visually, auditory, tactile, etc. Can be presented with multiple senses.
[0011]
Further, in the multisensory information transmitting apparatus of the present invention, the state information generation unit divides the object included in the image into predetermined unit parts, and the position and characteristics of the divided parts for each part. A generating unit that detects and generates the state information according to the position and characteristics of each detected part; and adds address information to the generated state information, while adding the address information to image information including the object. And an address information adding unit to be added.
[0012]
As described above, the state information generation unit divides the object included in the image into predetermined units (for example, predetermined areas), and the state information according to the position and characteristics (for example, shape, hardness, etc.) of the divided parts. By generating, the state of the object included in the image can be presented on the receiving side with multiple senses such as visual, auditory, and tactile sensations.
[0013]
Also, in the multisensory information transmitting apparatus of the present invention, the state information generating unit includes a range specifying unit that specifies a predetermined range of the object, and one or more included in the specified predetermined range among the divided portions And a tag information adding unit for adding tag information to the detected one or more parts.
[0014]
In this way, by specifying a predetermined range of the object and adding tag information to one or more parts included in the predetermined range among the divided parts, information such as names and features can be added to each part. it can.
[0015]
In order to solve the above-described problem, the multisensory information receiving apparatus of the present invention is a multisensory information receiving apparatus used in a multisensory information transmission system that transmits the state of an object included in an image to a user multisensory. A state information receiving unit that receives state information of the object included in the image, and extracts the state of the object based on the received state information, and the state of the extracted object is displayed in the state presenting device. And a control unit that controls to be presented.
[0016]
In this way, the state information of the object included in the image is received separately from the image information, the state of the object is extracted based on the received state information, and the state presenting apparatus presents the state of the extracted object. By controlling in this way, the state of the object included in the image can be transmitted to the user with multiple senses.
[0017]
Further, in the multisensory information receiving apparatus of the present invention, the control unit is based on the extraction unit that extracts the position and the feature of each divided part from the received state information, and the position and the feature of each extracted part. A tactile sensation presentation control unit that causes the tactile sensation presentation apparatus to present the object by tactile sense, a position detection unit that detects a position touched by the user among the objects presented by the tactile sense, and a touch by the detected user. A tag information presentation control unit that extracts tag information added to a position and causes a tag information presentation device to present information related to the position touched by the user based on the extracted tag information. .
[0018]
In this way, the control unit extracts the position and feature for each part divided from the received state information, and causes the tactile sense presentation device to present the target object by tactile sense based on the extracted position and feature for each part. Therefore, the user can feel the shape of the object by touching the object presented by the tactile sense presentation device.
[0019]
In addition, the control unit detects a position touched by the user, and if tag information is added to the position touched by the user, the control unit presents information related to the position touched by the user to the tag information presenting device based on the tag information. Can be made. The tag information presentation device can be configured to output information on the position touched by the user, for example, by voice or Braille.
[0020]
DETAILED DESCRIPTION OF THE INVENTION
Next, embodiments of the present invention will be described with reference to the drawings. FIG. 1 shows a schematic diagram of an example of the multisensory information transmission system of the present invention. The multisensory information transmission system 1 in FIG. 1 is configured to include a transmission unit 2, a reception unit 3, and a display unit 4. The transmission unit 2 and the reception unit 3 are connected via a network 5 such as the Internet 5 and a broadcast network 6 such as TV broadcast, for example.
[0021]
The transmission unit 2 includes a figure / stereoscopic object information generation unit 11, a broadcast program content input unit 12, an Internet transmission unit 13, a data broadcast multiplexing unit 14, and a TV transmission unit 15. The figure / stereoscopic object information generation unit 11 detects physical information such as a figure, a solid position, hardness, surface property, and shape used in a broadcast program, and generates object information based on the detected physical information. For example, as a figure and a solid used in a broadcast program, there are works of art introduced in a broadcast program such as “Sunday Art Museum”.
[0022]
When transmitting the generated object information using the network 5, the figure / stereoscopic object information generation unit 11 supplies the object information to the Internet transmission unit 13. Also, when the generated object information is transmitted using the broadcast network 6, the figure / stereoscopic object information generation unit 11 supplies the object information to the data broadcast multiplexing unit 14.
[0023]
The Internet transmission unit 13 transmits the object information supplied from the figure / stereoscopic object information generation unit 11 to the network 5. On the other hand, the data broadcast multiplexing unit 14 inserts the object information supplied from the figure / stereoscopic object information generation unit 11 into the data broadcasting data and supplies the data to the TV transmission unit 15.
[0024]
The TV transmission unit 15 is supplied with normal broadcast program data from the broadcast program content input unit 12 and data broadcast data from the data broadcast multiplexing unit 14, and the supplied broadcast program data and data broadcast data are supplied. Is transmitted to the broadcast network 6 as a TV broadcast wave. The object information transmitted to the network 5 and the TV broadcast wave transmitted to the communication network 6 are received by the receiver 3.
[0025]
The receiving unit 3 includes an Internet receiving unit 16, a TV receiving unit 17, a data broadcast receiving unit 18, and a figure / three-dimensional object information reproducing unit 19. The internet receiver 16 receives object information via the network 5. The Internet receiving unit 16 supplies the received object information to the figure / stereoscopic object information reproducing unit 19.
[0026]
The TV receiver 17 receives TV broadcast waves from the broadcast network 6 and detects broadcast program data and data broadcast data from the received TV broadcast waves. The TV receiving unit 17 transmits the detected broadcast program data to a broadcast program content display unit 23 described later, and supplies the data broadcast data to the data broadcast receiving unit 18.
[0027]
The data broadcast receiving unit 18 detects the object information inserted by the transmission unit 2 from the supplied data broadcasting data, and supplies the detected object information to the figure / stereoscopic object information reproducing unit 19. The figure / stereoscopic object information reproducing unit 19 is supplied with the object information supplied from the Internet receiving unit 16 and the data broadcast receiving unit 18, and based on the object information, the diagram used for the broadcast program, the position of the solid, the hardness, Reproduce physical information such as surface properties and shape. The figure / stereoscopic object information reproduction unit 19 controls processing of a tactile / force-sensing presentation device 20, a voice / braille output device 21, and a screen output device 22, which will be described later, based on the reproduced physical information.
[0028]
The display unit 4 includes a tactile / force sense presentation device 20, a voice / braille output device 21, a screen output device 22, and a broadcast program content display unit 23. The tactile / force sense presenting device 20 presents a figure and a solid used in a broadcast program in accordance with the control from the figure / three-dimensional object information reproduction unit 19 by a tactile sense. The voice / braille output device 21 outputs information about a figure and a solid used in a broadcast program in voice or braille according to control from the figure / solid object information reproduction unit 19.
[0029]
The screen output device 22 outputs the figure and the solid used in the broadcast program as an image in accordance with the control from the figure / solid object data reproduction unit 19. The broadcast program content display unit 23 receives the broadcast program data transmitted from the TV receiving unit 17 and displays the broadcast program.
[0030]
As described above, the multisensory information transmission system 1 according to the present invention transmits physical information such as a figure, a solid position, hardness, surface property, and shape used in a broadcast program separately from normal broadcast program data to the network 5. , And can be transmitted using the broadcast network 6, and the receiving side can present the figure, the position of the solid, the hardness, the surface property, the shape, and the like with multiple senses such as sight, touch, and hearing.
[0031]
Hereinafter, details of the transmission unit 2 and the reception unit 3 constituting the multisensory information transmission system 1 of the present invention will be described. FIG. 2 shows a block diagram of an embodiment of the transmission unit of the present invention. 2 includes a figure / stereoscopic object information generation unit 11, a broadcast program content input unit 12, an Internet transmission unit 13, a data broadcast multiplexing unit 14, a TV transmission unit 15, an encoding unit 38, and a URL / data address addition unit. 39, a URL adding unit 40 is included. The figure / solid object information generation unit 11 includes a 2 / 3-dimensional object surface division unit 31, an object space position & parameter detection unit 32, a tag information addition position designation unit 33, a tag information space position detection unit 34, and a tag information input. A unit 35, a 2 / 3-dimensional object / tag space position comparison unit 36 and an object information generation unit 37
[0032]
The 2 / 3-dimensional object surface dividing unit 31 divides the surface of a figure and a solid (hereinafter referred to as “object”) used in a broadcast program into an appropriate number of portions (hereinafter referred to as “polygon”) and divides the surface. The polygons thus obtained are supplied to the object space position & parameter detection unit 32. The object space position & parameter detection unit 32 detects a texture (Texture) such as a spatial position, hardness, surface property, and shape of each polygon as physical information.
[0033]
The tag information addition position designation unit 33 designates a spatial position to which tag information representing a name, feature, etc. is added, and supplies the designated spatial position to the tag information space position detection unit 34. The tag information space position detection unit 34 selects polygons included in the designated space position, and detects the space position of each selected polygon. The tag information space position detection unit 34 supplies the space position of each selected polygon to the tag information input unit 35. The tag information input unit 35 adds tag information to each polygon selected by the tag information space position detection unit 34.
[0034]
The 2 / 3-dimensional object / tag space position comparison unit 36 is supplied with physical information such as the spatial position of each polygon from the object space position & parameter detection unit 32, and also receives tag information and its tag information from the tag information input unit 35. The spatial position of each polygon to which is added is supplied.
[0035]
The 2 · 3D object / tag space position comparison unit 36 compares the space position of each polygon detected by the object space position & parameter detection unit 32 with the space position of each polygon selected by the tag information space position detection unit 34. The comparison result is supplied to the object information generation unit 37. The 2 / 3-dimensional object / tag space position comparison unit 36 includes physical information such as the spatial position of each polygon supplied from the object space position & parameter detection unit 32, tag information supplied from the tag information input unit 35, and The spatial position of each polygon to which the tag information is added is also supplied to the object information generation unit 37.
[0036]
The object information generation unit 37 adds tag information to each polygon of the object that matches the spatial position of each polygon selected by the tag information space position detection unit 34 based on the supplied comparison result. Then, the object information generation unit 37 generates object information from the physical information of each polygon, tag information, etc., and supplies it to the encoding unit 38.
[0037]
The encoding unit 38 encodes the object information supplied from the object information generating unit 37 with VRML (Virtual Reality Modeling Language) or the like, and encodes the encoded object information (hereinafter referred to as “encoded object information”). , And supplied to the data broadcast multiplexing unit 14. The URL adding unit 40 adds a URL (Uniform Resource Locator) to the encoded object information supplied from the encoding unit 38 and supplies the encoded object information with the URL added to the Internet transmitting unit 13. The Internet transmission unit 13 transmits the encoded object information supplied from the URL adding unit 40 to a server arranged in the network 5 in, for example, the TCP / IP format based on the URL added to the encoded object information.
[0038]
When transmitting the encoded object information using the network 5, the URL / data address adding unit 39 supplies the URL to the data broadcast multiplexing unit 14 and transmits the encoded object information using the broadcast network 6. In this case, the index or address of the encoded object information is supplied to the data broadcast multiplexing unit 14.
[0039]
The data broadcast multiplexing unit 14 inserts the encoded object information supplied from the encoding unit 38 and the URL, index, or address supplied from the URL / data address adding unit 39 into the data broadcasting data, and sends it to the TV transmission unit 15. Supply. The TV transmission unit 15 is supplied with normal broadcast program data from the broadcast program content input unit 12 and data broadcast data from the data broadcast multiplexing unit 14, and the supplied broadcast program data and data broadcast data are supplied. Is transmitted to the broadcast network 6 as a TV broadcast wave.
[0040]
Here, the object information generation processing of the figure / stereoscopic object information generation unit 11 will be described in detail with reference to FIGS. FIG. 3 shows a flowchart of an example of the object information generation process. FIG. 4 is a diagram illustrating an example of an object divided into polygons. FIG. 5 is a diagram illustrating an example of the relationship between the spatial position information and the tag information.
[0041]
For example, when the object 42 shown in FIG. 4 is used in a broadcast program, the figure / three-dimensional object information generation unit 11 generates object information according to the procedure shown in the flowchart of FIG. In FIG. 3, in step S1, the 2 / 3-dimensional object surface dividing unit 31 divides the surface of the object 42 into an appropriate number of polygons. 1 to 38 described on the surface of the object 42 indicate polygon numbers (polygon NO). A polygon number is assigned to each polygon.
[0042]
Progressing to step S2 following step S1, the object space position & parameter detection unit 32 detects the spatial position of the polygon divided in step S1 by, for example, a laser distance measuring means. The spatial position of the detected polygon is recorded as spatial position information represented by (X, Y, Z) coordinates, as shown in the 2 / 3-dimensional object space position information of FIG. In step S2, physical information other than the spatial position of the polygon is also detected.
[0043]
Progressing to step S3 following step S2, the object space position & parameter detection unit 32 determines whether or not physical information of all polygons has been acquired. If it is determined that physical information of all polygons has not been acquired (NO in S3), the process proceeds to step S2, and the object space position & parameter detection unit 32 detects physical information of polygons for which physical information has not yet been acquired.
[0044]
If it is determined that physical information of all the polygons has been acquired (YES in S3), the process proceeds to step S4, where the tag information addition position designation unit 33 designates a spatial position to which tag information is added. For example, when tag information such as “horizontal tail” and “fuselage” is added to the A part 43 and the B part 44 surrounded by a circle or an ellipse of the object 42, respectively, the A part 43 and the B part 44 of the object 42 are separated by a laser. The tag information addition position is designated by scanning with the measuring means.
[0045]
Progressing to step S5 following step S4, the tag information space position detection unit 34 detects the space position of the polygon included in the tag information addition position specified in step S4. In step S6 following step S5, the tag information input unit 35 inputs text tag information.
[0046]
For example, when tag information “horizontal tail” and “fuselage” is added to the A portion 43 and the B portion 44 surrounded by a circle or an ellipse of the object 42, the spatial position of the polygon included in the A portion 43 is detected. , And input text tag information “horizontal tail”. Further, the position of the polygon included in the B part 44 is detected, and text tag information “body” is input.
[0047]
Progressing to step S7 following step S6, it is determined whether or not the input of all tag information for the tag information position designated in step S4 is completed. If it is determined that the input of all tag information for the tag information position specified in step S4 has not been completed (NO in S7), the process proceeds to step S5. If it is determined that the input of all tag information for the tag information position specified in step S4 is completed (YES in S7), the process proceeds to step S8.
[0048]
In step S8, it is determined whether or not the input of all necessary tag information has been completed, in other words, whether or not the position designation of the desired tag information to which tag information is to be added has been completed. If it is determined that the position designation of all tag information has been completed (YES in S8), the process proceeds to step S9. If it is determined that the position designation of all tag information has not been completed (NO in S8), the process proceeds to step S4.
[0049]
In step S9, the 2 / 3-dimensional object / tag space position comparing unit 36 compares the spatial position of each polygon detected in step S2 with the spatial position of each polygon detected in step S5. Progressing to step S10 following step S9, when the 2 / 3-dimensional object / tag space position comparison unit 36 detects polygons having the same spatial position (YES in S10), the process proceeds to step S11, and the 2 / 3-dimensional object of FIG. As shown in the space position information and the tag position information, tag information is added to the polygon detected in step S2.
[0050]
The tag position information is configured to include polygon number, spatial position information represented by (X, Y, Z) coordinates, and text tag information. If the 2 / 3-dimensional object / tag space position comparison unit 36 does not detect polygons having the same space position (NO in S10), the process proceeds to step S9. In step S12 following step S11, the object information generation unit 37 creates object information from physical information, tag information, and the like.
[0051]
Note that in steps S9 and S10, the spatial position of each polygon detected in step S2 and the spatial position of each polygon detected in step S5 may not completely match due to a measurement error or the like. It is desirable to set a range.
[0052]
Next, the detail of the receiving part 3 which comprises the multisensory information transmission system 1 of this invention is demonstrated. FIG. 6 shows a block diagram of an embodiment of the receiving unit and display unit of the present invention. 6 includes an Internet reception unit 16, a TV reception unit 17, a data broadcast reception unit 18, a figure / stereoscopic object information reproduction unit 19, a download control unit 51, and a URL / data address extraction unit 52. Is done.
[0053]
The figure / stereoscopic object information reproducing unit 19 includes an object information decoding unit 53, a tactile position pointer adding unit 54, an object tactile information generating unit 55, a screen presentation output control unit 56, a tag information extraction output control unit 57, and a tactile presentation output. A control unit 58 and a tactile sense presentation position detection unit 59 are included. The display unit 4 is configured to include a tactile / force sense presentation device 20, a voice / braille output device 21, a screen output device 22, a broadcast program content display unit 23, and a data program display unit 60.
[0054]
The Internet receiver 16 receives the encoded object information via the network 5 and supplies the received encoded object information to the download control unit 51. The TV receiver 17 receives TV broadcast waves from the broadcast network 6 and detects broadcast program data and data broadcast data from the received TV broadcast waves. The TV receiving unit 17 transmits the detected broadcast program data to the broadcast program content display unit 23 and supplies the data broadcast data to the data broadcast receiving unit 18.
[0055]
The data broadcast receiving unit 18 receives the supplied data broadcast data and supplies the data program data to the data program display unit 60 and the URL / data address extraction unit 52. The URL / data address extraction unit 52 extracts the URL, index or address of the encoded object information inserted into the data program data, and supplies the extracted URL to the download control unit 51, while extracting the extracted index or address. Are supplied to the data broadcast receiving unit 18.
[0056]
Based on the URL supplied from the URL / data address extraction unit 52, the download control unit 51 controls the Internet reception unit 16 to download the encoded object information from, for example, a server arranged in the network 5. The download control unit 51 supplies the downloaded encoded object information to the object information decoding unit 53.
[0057]
The data broadcast receiving unit 18 extracts the encoded object information inserted into the data for data broadcasting based on the index or address supplied from the URL / data address extracting unit 52. The data broadcast receiving unit 18 supplies the extracted object information to the object information decoding unit 53.
[0058]
The object information decoding unit 53 decodes the encoded object information encoded by VRML or the like supplied from the download control unit 51 or the data broadcast receiving unit 18, and based on the decoded object information, the physical information of the object, the tag Play information etc. The object information decoding unit 53 supplies the reproduced physical information and tag information to the object tactile information generation unit 55 and the screen presentation output control unit 56.
[0059]
The object tactile information generation unit 55 generates tactile information according to the supplied physical information, and supplies the generated tactile information to the tactile presentation output control unit 58. The tactile sensation presentation output control unit 58 drives and controls the haptic / force sensation presentation apparatus 20 such as PHAMTOM in accordance with the supplied tactile information. The tactile / force sense presenting device 20 presents the object by tactile sense according to the control of the tactile sense output control unit 58.
[0060]
The tactile sensation presentation position detection unit 59 detects position information of a touched part touched by the user's hand, fingertip, etc., and feeds back the detected position information to the object tactile information generation unit 55. The object tactile information generation unit 55 generates tactile information corresponding to the tactile part from the position information of the tactile part, and supplies the generated tactile information to the tactile presentation output control unit 58.
[0061]
That is, when the user moves the hand, the fingertip, etc., the position information of the touched portion is detected by the tactile sensation presentation position detecting unit 59, and the detected position information is fed back to the object tactile information generating unit 55 and fed back. Tactile information corresponding to the touched part is sequentially generated from the obtained position information.
[0062]
Then, by supplying the tactile information generated sequentially to the tactile sensation presentation output control unit 58 and controlling the reaction force by the tactile sensation / force sensation presentation device 20, it is possible to give the user a sense that the object exists, The user can recognize the three-dimensional shape.
[0063]
The object tactile information generation unit 55 supplies the position information and tag information supplied from the tactile sense position detection unit 59 to the tag information extraction output control unit 57. The tag information extraction / output control unit 57 uses the supplied position information and tag information to extract tag information of the touching portion, and drives and controls the voice / braille output device 21 according to the extracted tag information. The voice / braille output device 21 outputs voice information or braille information according to the control of the tag information extraction output control unit 57. That is, the name, characteristics, etc. of the touched part touched by the user's hand, fingertip, etc. can be acquired from the position information and tag information and transmitted to the user by voice or Braille.
[0064]
The tactile position pointer adding unit 54 creates information for adding the position of the tactile part detected by the tactile sense position detecting unit 59 on the screen as a pointer, and supplies the information to the screen presenting output control unit 56. . The screen presentation output control unit 56 causes the screen output device 22 to present the object reproduced by the object information decoding unit 53 and the pointer added by the tactile position pointer adding unit 54.
[0065]
The broadcast program content display unit 23 receives the broadcast program data transmitted from the TV receiving unit 17 and displays the broadcast program. The data program display unit 60 receives the data program data transmitted from the data broadcast receiving unit 18 and displays the received data program. The screen output device 22 presents an object and a pointer on the screen under the control of the screen presentation output control unit 56.
[0066]
As described above, figures and solid objects used in broadcast programs exist as objects that can be detected by touch in a virtual space in addition to vision, and the names and features of the parts are transmitted by voice and Braille depending on the touched position. Is done. Therefore, the user can confirm the name and characteristics of the touched position while touching the object that can be detected by tactile sensation, and can easily understand a three-dimensional shape such as a work of art.
[0067]
In this embodiment, the tactile / force-sensing presentation device 20 is used as an example of a tactile receptor that performs tactile presentation. However, the present invention is not limited to this, and is a tactile receptor formed by a micromachine or other microactuator. May be. Further, in this embodiment, an example in which object information is transmitted using the network 5 or the broadcast network 6 is described. However, the transmission using the network 5 and the broadcast network 6 according to the transmission capacity, the network network Alternatively, transmission using any one of the broadcast networks 6 is possible.
[0068]
The multisensory information transmitter described in the claims corresponds to the transmitter 2, the multisensory information receiver corresponds to the receiver 3, and the state information generator corresponds to the figure / stereoscopic object information generator 11. The control unit corresponds to the figure / solid object information reproduction unit 19.
[0069]
【The invention's effect】
As described above, according to the present invention, the state information of the object included in the image is generated, and the generated state information is transmitted to the receiving side separately from the image information, so that the object included in the image on the receiving side. Can be presented with multiple senses such as visual, auditory, and tactile sensations.
[0070]
Further, by designating a predetermined range of the object and adding tag information to one or more parts included in the predetermined range among the divided parts, information such as names and features can be added for each part.
[0071]
In addition, the state information of the object included in the image is received separately from the image information, the state of the object is extracted based on the received state information, and the state presenting apparatus presents the state of the extracted object. By controlling, the state of the object included in the image can be transmitted to the user with multiple senses.
[0072]
Moreover, the user can touch the shape of a target object by touching the target object presented by tactile sense. Furthermore, the information regarding the position touched by the user can be presented to the tag information presentation device by voice or Braille.
[0073]
[Brief description of the drawings]
FIG. 1 is a schematic diagram of an example of a multisensory information transmission system of the present invention.
FIG. 2 is a configuration diagram of an embodiment of a transmission unit according to the present invention.
FIG. 3 is a flowchart of an example of object information generation processing.
FIG. 4 is a diagram illustrating an example of an object divided into polygons.
FIG. 5 is a diagram illustrating an example of a relationship between space position information and tag information.
FIG. 6 is a configuration diagram of an embodiment of a receiving unit and a display unit of the present invention.
[Explanation of symbols]
1 Multisensory information transmission system
2 Transmitter
3 receiver
4 display section
5 network
6 Broadcasting network
11 Figure and 3D object information generator
12 Broadcast program content input section
13 Internet transmission department
14 Data Broadcast Multiplexer
15 TV transmitter
16 Internet receiver
17 TV receiver
18 Data broadcasting receiver
19 Figure and 3D object information playback unit
20 Tactile / force-sensing device
21 Voice / Braille output device
22 Screen output device
23 Broadcast program content display
31 2 / 3-dimensional object surface division
32 Object space position & parameter detector
33 Tag information addition position designation part
34 Tag information space position detector
35 Tag information input section
36 2 / 3D Object / Tag Space Position Comparison Unit
37 Object information generator
38 Code part
39 URL / data address adding section
40 URL addition part
51 Download controller
52 URL / data address extractor
53 Object Information Decoding Unit
54 Tactile position pointer addition unit
55 Object tactile information generator
56 Screen presentation output controller
57 Tag information extraction output control unit
58 Tactile presentation output control unit
59 Tactile sense presentation position detector
60 Data program display section

Claims (4)

A multisensory information transmitting device used in a multisensory information transmission system that multi-senses the state of an object included in an image that outputs a figure or a solid ,
A state information generating unit that generates state information of an object included in the image;
And a status information transmitting unit for transmitting the generated status information,
The state information generating unit includes a dividing unit that divides an object included in the image into predetermined unit parts;
A generating unit that detects the position and characteristics of the divided parts for each part, and generates the state information according to the detected position and characteristics of each part;
An address information adding unit for adding address information to the image information including the object, while adding address information to the generated state information;
A range designating unit for designating a predetermined range of the object;
A detection unit for detecting one or more parts included in the designated predetermined range among the divided parts;
A tag information adding unit for adding tag information to the detected one or more parts;
Multi-sensory information transmission apparatus that includes a. The multisensory information transmitting apparatus according to claim 1, wherein the characteristic of the divided portion is a texture including hardness, surface property, and shape. A multisensory information receiving device used in a multisensory information transmission system that multi-senses the state of an object included in an image that outputs a figure or a solid ,
A state information receiving unit for receiving state information of an object included in the image;
A controller that extracts the state of the object based on the received state information and controls the state presenting apparatus to present the state of the extracted object ;
The control unit is configured to extract a position and a feature for each of the divided parts from the received state information;
A tactile sensation presentation control unit that causes the tactile sensation presentation apparatus to present the object in a tactile sense based on the position and characteristics of each of the extracted parts;
A position detection unit for detecting a position touched by a user among the objects presented by the tactile sense;
A tag information presentation control unit that extracts tag information added to a position touched by the detected user and causes a tag information presentation device to present information related to the position touched by the user based on the extracted tag information;
Multi-sensory information receiving apparatus that includes a. 4. The multisensory information receiving apparatus according to claim 3, wherein the characteristic of the divided portion is a texture including hardness, surface property, and shape.

Images