KR20180110644A - Sensor information processing method and system between virtual world and real world - Google Patents

Abstract

Disclosed are a method and a system for efficiently processing sensor information between a virtual world and a real world. The method for processing sensor information comprises: acquiring first sensor information from a camera sensor in a real world; converting the acquired first sensor information into virtual world object characteristics applied to a virtual world or second sensor information applied to a virtual world; and applying the converted virtual world object characteristics or the second sensor information to the virtual world.

Description

TECHNICAL FIELD [0001] The present invention relates to a sensor information processing method and system for a virtual world and a real world,

The present invention relates to a sensor information processing method and system between a virtual world and a real world, and more particularly, to a method of defining a sensor information format between devices for a real world and a virtual world, and converting and providing sensor information .

In order to control the components of a virtual world such as an object, various sensors existing in the real world are used. Since there are many types of sensors in the real world, it is necessary to define the sensor information when applied to a virtual world. In addition, by knowing the capabilities of the sensors to acquire the sensor information, it is required to implement the sensor information more effectively in the virtual world.

The present invention proposes various sensor information that can be collected through a camera sensor and a microphone sensor, so that the conversion between the real world and the virtual world can be efficiently handled.

The present invention proposes a capability of each of the camera sensor and the microphone sensor in detail, thereby efficiently converting the conversion between the real world and the virtual world.

According to an embodiment of the present invention, there is provided a sensor information processing method comprising: obtaining first sensor information from a sensor in a real world; Converting the acquired first sensor information into virtual world object characteristics applied to a virtual world or second sensor information applied to a virtual world; And applying the transformed virtual world object feature or second sensor information to a virtual world, wherein the real world sensors correspond to a sensor capability specification and global coordinates dependent on the environment of the real world are set .

The real world sensor may include a camera sensor, and the camera sensor may be set to a camera sensor capability type.

The camera sensor capability type includes at least one of a supported resolution flag, a supported resolution, a resolution list, a resolution list type, a resolution type, a width, a height, a focal length range flag, a focal length range, a value range type, A color filter array flag, a color filter array type, a color space flag, a color space type, a bit depth range, an aperture value range, a shutter speed range, an ISO speed range flag, an ISO speed range, May include at least one of a flag, a bit depth range, a spectrum range flag, a spectrum range, a column range flag, a column range, a white balance temperature range flag, a white balance temperature range, a white balance tint flag, and a white balance tint range.

The real world sensor may include a microphone sensor, and the microphone sensor may be set to a microphone sensor capability type.

The microphone sensor capability type may include at least one of a microphone type, a transducer array type, a probe type, a polar pattern, a frequency range, a response type flag, a response frequency, and a pickup sensitivity.

The camera sensor is specified according to a camera sensor type (CameraSensorType)

The camera sensor type includes at least one of a camera orientation, a camera position, a camera altitude, a focal length, an aperture, a shutter speed, a filter, a camera direction flag, A camera position flag, a camera height flag, a focal length flag, an aperture flag, a shutter speed flag, and a filter flag.

The microphone sensor is specified in accordance with a microphone sensor type, and the microphone sensor type includes a direction flag, an altitude flag, a position flag, a sample rate flag, a resolution flag, a direction, an altitude, , Byte order, sign, resolution, big indian, little indian, low audio data size, and low audio data size.

The recording medium according to an embodiment of the present invention is readable in an electronic device in which sensor information of a sensor in a real world for applying sensor information to a virtual object in a virtual world is recorded, Sensors correspond to the sensor capability specification, and global coordinates that depend on the environment of the real world can be set.

The real world sensor may include a camera sensor, and the camera sensor may be set to a camera sensor capability type.

The camera sensor capability type includes at least one of a supported resolution flag, a supported resolution, a resolution list, a resolution list type, a resolution type, a width, a height, a focal length range flag, a focal length range, a value range type, A color filter array flag, a color filter array type, a color space flag, a color space type, a bit depth range, an aperture value range, a shutter speed range, an ISO speed range flag, an ISO speed range, May include at least one of a flag, a bit depth range, a spectrum range flag, a spectrum range, a column range flag, a column range, a white balance temperature range flag, a white balance temperature range, a white balance tint flag, and a white balance tint range.

The real world sensor may include a microphone sensor, and the microphone sensor may be set to a microphone sensor capability type.

The microphone sensor capability type may include at least one of a microphone type, a transducer array type, a probe type, a polar pattern, a frequency range, a response type flag, a response frequency, and a pickup sensitivity.

The camera sensor is specified according to a camera sensor type (CameraSensorType)

The camera sensor type includes at least one of a camera orientation, a camera position, a camera altitude, a focal length, an aperture, a shutter speed, a filter, a camera direction flag, A camera position flag, a camera height flag, a focal length flag, an aperture flag, a shutter speed flag, and a filter flag.

The microphone sensor is specified in accordance with a microphone sensor type, and the microphone sensor type includes a direction flag, an altitude flag, a position flag, a sample rate flag, a resolution flag, a direction, an altitude, , Byte order, sign, resolution, big indian, little indian, low audio data size, and low audio data size.

A sensor information processing system according to an embodiment of the present invention includes a media processor, and the media processor acquires first sensor information from a sensor in the real world, and outputs the acquired first sensor information to a virtual world And converts the converted virtual world object characteristic or the second sensor information into a virtual world, wherein the real world sensors are connected to a sensor Global coordinates that correspond to the capability specification and depend on the environment of the real world can be set.

The real world sensor may include a camera sensor, and the camera sensor may be set to a camera sensor capability type.

The real world sensor may include a microphone sensor, and the microphone sensor may be set to a microphone sensor capability type.

The camera sensor may be specified according to a camera sensor type (CameraSensorType).

The microphone sensor may be specified according to a microphone sensor type (Microphone Sensor Type).

According to an embodiment of the present invention, various sensor information that can be collected through the camera sensor and the microphone sensor are proposed, thereby efficiently processing the conversion between the real world and the virtual world.

According to an embodiment of the present invention, the capability between each of the camera sensor and the microphone sensor is proposed in detail, so that the conversion between the real world and the virtual world can be efficiently handled.

FIG. 1 illustrates a sensor information processing method and system between a virtual world and a real world according to an embodiment of the present invention.
2 is a diagram illustrating a media processor for applying sensor information obtained from a camera sensor and a microphone sensor in a real world to a virtual world according to an embodiment of the present invention.
3 is a flowchart illustrating a process of converting sensor information between a real world and a virtual world according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 illustrates a sensor information processing method and system between a virtual world and a real world according to an embodiment of the present invention.

According to an embodiment of the present invention, a virtual world of a digital world, such as digital contents provider, gaming, simulation, DVD, etc., and a sensor, actuator, vision, rendering, robotics, independent living, social / welfare system, , Interoperability between real world such as insurance, travel, real estate, and right management.

Virtual worlds integrate media technologies such as instant messaging, video, 3D, VR, AI, chat, and voice.

The media processor of the present invention may be implemented with a control including sensor effect metadata, sensor device capabilities / commands, user sensor preference, various delivery formats, It supports control information and sensor information.

A media processor in accordance with an embodiment of the present invention processes control information such as capabilities and preferences for sensors and actuators. In particular, the control information may include a device capability description, user preference information, and the like.

In addition, the media processor according to an embodiment of the present invention exchanges sensor information for interaction devices. Specifically, the media processor supports command formats that control actuators and data formats for receiving sensor information from sensors.

According to one embodiment of the present invention, the media processor supports a data format for communication between the adaptation engine and the capabilities / preferences of actuators / sensors in the real world. The control information includes user actuator preference information, user sensor preference information, an actuator capability description, and a sensor capability description. The control information may be used to tune sensor information and actuator commands for control of the virtual world / real world by providing external information to the adaptation engine.

According to one embodiment of the present invention, the syntax and semantics required to provide interoperability to control devices (actuators, and sensors) in the real world as well as the virtual world are provided.

The present invention provides a control information description language (CIDL) as an XML schema-based language to describe the basic structure of control information. The Device Capability Description Vocabulary (DCDV) is a term used to describe the capabilities of actuators such as lamps, fans, vibrators, motion chairs, scent generators, etc. Identify the XML representation. For example, the maximum wind speed, airflow level, etc. of a fan can be defined in the DCDV specification.

The Sensor Capability Description Vocabulary (SCDV) is a set of information that can be used for a variety of sensors including a light sensor, a temperature sensor, a velocity sensor, a global position sensor, an intelligent camera sensor ), And so on. For example, the capabilities of a global position sensor means a maximum operating temperature image of 90 degrees, a minimum operating temperature of minus 30 degrees, a sensitivity of 0.01 degrees, and a position accuracy of 0.01 degrees, which can be defined in the SCDV specification.

The Sensory Effect Preference Vocabulary (SEPV) specifies interfaces for describing a user's personal preferences for specific sensor effects such as light, wind, aroma, and vibration. For example, in the SEPV specification, the maximum intensity of a vibrating chair may be defined as 600 Hz. The Sensor Adaptation Preference Vocabulary (SAPV) specifies interfaces for describing individual sensor preferences and individual types of sensor information. For example, optical sensor adaptation is achieved by detecting between a minimum brightness of 10 lux and a maximum brightness of 400 lux.

According to another embodiment, an adaptation engine and a data format for communication between actuators / sensors in a real world or virtual world object are provided.

According to an embodiment of the present invention, by defining an XML Schema-based language named Interaction Information Description Language (IIDL), a format for interfacing actuators and sensors is provided, The syntax and semantics of the data format for the data format are specified. IIDL provides a basic structure with common information for communication with various actuators and sensors. A Device Command Vocabulary (DCV) provides a standardized format for commanding individual actuators, and a Sensed Information Vocabulary (SIV) is used to obtain environmental information from the real world or to obtain Information is used to hold information from any one of the sensors affecting virtual world objects.

The adaptation engine includes user's sensory preferences (USP), sensory device capabilities (SDC), sensor capabilities (SC) to control devices in both the real world and the virtual world, And sensor adaptation preferences (SAP).

Referring to FIG. 1, a media processor can convert sensor information between a virtual world and a real world. The media processor can also be represented as an engine. The media processor can perform the first adaptation that converts the sensor information acquired from the real world into the sensor information applied to the virtual world or converts the sensor information acquired from the real world into the virtual object characteristic of the virtual world. The media processor may perform a second adaptation that converts the sensor effect data or the virtual object characteristic of the virtual world into an actuator command applied to the real world. Here, the adaptation can be expressed by an engine.

2 is a diagram illustrating a media processor for applying sensor information obtained from a camera sensor and a microphone sensor in a real world to a virtual world according to an embodiment of the present invention.

Referring to FIG. 2, a process of collecting sensor information from a real world camera sensor 202 and a real world microphone sensor 203 is shown. The camera sensor 202 may provide the acquired sensor information to the media processor 201. [ Likewise, the microphone sensor 203 can provide the acquired sensor information to the media processor 201. Sensor information obtained from the camera sensor 202 and the microphone sensor 203 may be processed through a first adaptation performed in the media processor 201. [ Here, the first adaptation can be expressed by an engine.

The media processor 201 may convert the sensor information acquired from the camera sensor 202 and the microphone sensor 203 through the first adaptation and apply it to the object 204 of the virtual world.

The present invention provides a format for sensor information that is collected or processed from the camera sensor 202 and the microphone sensor 203. The present invention also provides a format for the sensor capabilities of the camera sensor 202 and the microphone sensor 203.

3 is a flowchart illustrating a process of converting sensor information between a real world and a virtual world according to an embodiment of the present invention.

In step 301, the media processor may obtain first sensor information from a sensor in the real world. In the present invention, a real-world sensor includes a camera sensor for acquiring an image and a microphone sensor for collecting audio.

In step 302, the media processor converts the first sensor information into a virtual world object feature applied to the virtual world or second sensor information applied to the virtual world.

In step 303, the media processor applies the virtual world object feature or the second sensor information to the virtual world.

Hereinafter, the data format and the detailed description of the sensor information and the sensor capability described in the present invention will be described in detail. Hereinafter, the first embodiment and the second embodiment will be described separately.

≪ Embodiment 1 >

(1) Additional metadata for media orchestration

Depending on the application, the characteristics of the sensor features, sensor capabilities, and captured image (video) are different. More detailed metadata about the basic features of the captured image is needed so that the media processor can successfully process (e.g., group, merge, split, etc.) the acquired data.

The present invention provides information for a media processor for a feature of a captured image (video).

Table 1 shows the syntax.

imageCharacteristics { Number of bits Mnemonic imageStreamFlag One bslbf imageCharacteristics imageCharacteristicsAttributes if (imageStreamFlag == 1) { charUpdateFlag One fsbf if (charUpdateFlag == 1) { imageCharacteristics imageCharacteristicsAttributes } } imageCharacteristicsAttributes { Number of bits Mnemonic colorImageFlag One bslbf filterFlag One bslbf colorSpaceFlag One bslbf bitDepthFlag One bslbf waveLengthFlag One bslbf temperatureFlag One bslbf tintFlag One bslbf if (filterFlag == 1) { filter 4 bslbf } if (colorSpaceFlag == 1) { colorSpaceIDLength vluimsbf colorSpaceID colorSpaceIDLength * 8 UTF-8 } if (bitDepthFlag == 1) { bitDepth 8 fsbf } if (waveLengthFlag == 1) { minWaveLength 14 fsbf maxWaveLength 14 fsbf } if (temperatureFlag == 1) { 온도 14 fsbf } if (tintFlag == 1) { tint 16 simsbf }

Table 2 shows the semantics

Name Definition imageStreamFlag '1' if video or image streaming
1 if it is a video or image stream. imageCharacteristics Defines characteristics of captured images (s) Defines characteristics of captured images charUpdateFlag '1' if any characteristic of currently captured image is different from the previous one If the current captured image is different from the previous captured image, colorImageFlag '0' if color, '1' if grayscale 0 if the captured image is color and 1 if the captured image is gray. filterFlag '1' if any filter is applied
If the filter is applied, 1 colorSpaceFlag '1' if colorSpace is defined 1 if a color space is defined bitDepthFlag '1' if bitDepth is defined
If the bit depth (bitDepth) is defined, waveLengthFlag '1' if waveLength range of captured image is defined
If the waveLenght range of the captured image is defined, temperatureFlag '1' if white balance temperature is defined If white balance temperature is defined, tint '1' if white balance tint is defined If white balance tint is defined, colorSpaceIDLength Length of colorSpaceID Color space ID length colorSpaceID Identifying color space used in image capturing Color space ID used when capturing an image bitDepth Current bit depth setting Set current bit depth waveLength Current wavelength setting in nanometer Current wavelength setting in nanometers minWaveLength Minimum wavelength in nanometer Minimum wavelength in nanometers maxWaveLength Maximum wavelength in nanometer Maximum wavelength in nanometers 온도 Current white balance temperature setting in Kelvin Current white balance temperature setting in Calvin tint Current white balance tint setting Current white balance tint setting

For certain types of surveillance, such as fire detection, night watch, etc., the features of the captured image are different. For example, thermal IR images used for fire detection or body heat detection at airports have other characteristics. Thus, by providing more metadata, the media processor can group, merge, and categorize images (video) from different sources.

Table 3 shows the syntax for the camera sensor type

<! - ##################################################### ## ->
<! - Camera Sensor Type ->
<! - ##################################################### ## ->
<complexType name = "CameraSensorType">
<complexContent>
<extension base = "iidl: SensedInfoBaseType">
<sequence>
<element name = "CameraGlobalPosition" type = "siv: GlobalPositionSensorType" minOccurs = "0"/>
<element name = "CameraOrientation" type = "siv: OrientationSensorType" minOccurs = "0"/>
<element name = "CameraAltitude" type = "siv: AltitudeSensorType" minOccurs = "0"/>
<element name = "CameraLocalPosition" type = "siv: PositionSensorType" minOccurs = "0"/>
<attribute name = "focalLength" type = "float" use = "optional"/>
<attribute name = "aperture" type = "float" use = "optional"/>
<attribute name = "shutterSpeed" type = "float" use = "optional"/>
<attribute name = "filter" type = "mpeg7: termReferenceType" use = "optional"/>
<attribute name = "ISOSpeed" type = "float" use = "optional"/>
<attribute name = "ExposureValue" type = "float" use = "optional"/>
<attribute name = "ColorFilter" type = "siv: ColorFilterArrayListType" use = "optional"/>
<attribute name = "Video" type = "boolean" use = "optional"/>
<attribute name = "SensorType" type = "boolean" use = "optional"/>
<attribute name = "ColorSpaceType" type = "string" use = "optional"/>
<attribute name = "BitDepth" type = "unsigned8" use = "optional"/>
<attribute name = "SpectrumRange" type = "siv: ValueRangeType" use = "optional"/>
<attribute name = "ThermalRange" type = "siv: ValueRangeType" use = "optional"/>
<attribute name = "WhiteBalanceTemp" type = "float" use = "optional"/>
<attribute name = "WhiteBalanceTint" type = "signed8" use = "optional"/>
</ sequence>
</ extension>
</ complexContent>
</ complexType>

<complexType name = "ValueRangeType">
<sequence>
<element name = "MaxValue" type = "float"/>
<element name = "MinValue" type = "float"/>
</ sequence>
</ complexType>

<simpleType name = "ColorFilterArrayListType">
<restriction base = "string">
<enumeration value = "Bayer"/>
<enumeration value = "RGBE"/>
<enumeration value = "CYYM"/>
<enumeration value = "CYGM"/>
<enumeration value = "RGB Bayer"/>
<enumeration value = "RGBW # 1"/>
<enumeration value = "RGBW # 2"/>
<enumeration value = "RGBW # 3"/>
</ restriction>
</ simpleType>
CameraSensorType { Number of bits Mnemonic CameraGlobalPositionFlag One bslbf CameraLocalPositionFlag One bslbf SupportedResolutionsFlag One bslbf FocalLengthFlag One bslbf ApertureFlag One bslbf ShutterSpeedFlag One bslbf FilterFlag One bslbf ISOSpeedFlag One bslbf ExposureValueFlag One bslbf ColorFilterArrayFlag One bslbf VideoFlag One bslbf SensorType One bslbf ColorSpaceFlag One bslbf BitDepthFlag One bslbf SpectrumRangeFlag One bslbf ThermalRangeFlag One bslbf WhiteBalanceTempFlag One bslbf WhiteBalanceTintFlag One bslbf SensedInfoBaseType SensedInfoBaseType if (CameraGlobalPositionFlag == 1) { CameraGlobalPosition GlobalPositionSensorType CameraAltitude AltitudeSensorType CameraOrientation OrientationSensorType } if (CameraLocalPositionFlag == 1) { CameraLocalPositionFlag PositionSensorType CameraOrientation OrientationSensorType } if (SupportedResolutionsFlag) { SupportedResolutions ResolutionListType } if (FocalLengthFlag) { FocalLength 32 fsbf } if (ApertureFlag) { Aperture 32 fsbf } if (ShutterSpeedFlag) { ShutterSpeed 32 fsbf } if (FilterFlag) { Filter 4 bslbf } if (ISOFlag) { ISOSpeed 32 fsbf } if (ExposureValueFlag) { ExposureValue 32 fsbf } if (ColorFilterArrayFlag) { ColorFilterArrayType ColorFilterArrayListType } if (ColorSpaceFlag) { ColorSpaceTypeLength vluimsbf ColorSpaceType ColorSpaceTypeLength * 8 UTF-8 } if (BitDepthFlag) { BitDepth 8 ValueRangeType } if (SpectrumRangeFlag) { SpectrumRange 32 ValueRangeType } if (ThermalRangeFlag) { ThermalRange ValueRangeType } if (WhiteBalanceTempFlag) { WhiteBalanceTemp 32 ValueRangeType } if (WhiteBalanceTintFlag) { WhiteBalanceTint 8 simsbf } } ResolutionListType { LoopResolution vluimsbf for (k = 0; k <LoopResolution; k ++) { Resolution [k] ResolutionType } } ResolutionType { Width 32 uimsbf Height 32 uimsbf } ValueRangeType { MaxValue 32 fsbf MinValue 32 fsbf }

Table 4 shows the semantics of the camera sensor capability type (CameraSensorCapabilityType)

Name Definition CameraSensorType Tool for describing sensed information with respect to a camera sensor.
Tools for describing sensor information for camera sensors CameraGlobalPosition Defines global positioning sensor based position information of Camera
Defines a global position sensor based on camera position information CameraAltitude Defines altitude of Camera Defines the altitude of the camera. CameraOrientation Defines orientation of Camera Defines the orientation of the camera. CameraLocalPosition Defines relative location of camera Defines the relative location of the camera SupportedResolutionsFlag This field, which is present in the binary representation, signals the presence of the SupportedResolutions element. This field signals the presence of a SupportedResolutions element and has a binary representation. 1, this element exists, and if it is 0, this element does not exist SupportedResolutions Describes a list of resolutions that the camera can support. ResolutionListType Describes a type of resolution list which consists of resolution type elements. ResolutionType Describes a type of resolution which is composed of a Width element and a Height element. Describes a resolution type consisting of a Width element and a Height element. Width Describes the width of the resolution that the camera can perceive. Height Describes the height of the camera's perceived resolution. FocalLengthFlag This field, which is present in the binary representation, signals the presence of the FocalLength element. A value of "1" means that this element is present and 0 means that this element is nopresent.
This field signals the presence of a focal length element and has a binary representation. If 1, this element exists, and if it is 0, this element does not exist FocalLength Describes the distance between the lens and the image sensor when the subject is in focus, in terms of millimeters (mm).
If the subject is in focus for mm units, the distance between the lenses and the image sensor is described. ValueRangeType Defines the range of values that the sensor can perceive. Describes the range of values that the image sensor can recognize. MaxValue Describes the maximum value that the sensor can perceive. MinValue Describes the minimum value that the sensor can perceive. ApertureFlag This field, which is present in the binary representation, signals the presence of the Aperture element. A value of "1" means that this element is present and 0 means that this element is nopresent.
This field signals the presence of the aperture element and has a binary representation. If 1, this element exists, and if it is 0, this element does not exist
Aperture Describes the aperture of a camera It is expressed as F-stop, eg F2.8. It may also be expressed as f-number notation such as f / 2.8.
Describes the iris of the camera. This is expressed as F-stop as F2.8. Or, this is expressed as f-number notation as f / 2.8.
ShutterSpeedFlag This field, which is present in the binary representation, signals the presence of the ShutterSpeed element. This field signals the presence of a shutter speed element and has a binary representation. If 1, this element exists, and if it is 0, this element does not exist
ShutterSpeed Describes the time the shutter remains open when taking a photograph in terms of seconds.
Describes how long the shutter stays open when taking pictures in seconds. filterFlag This field, which is present in the binary representation, signals the presence of the filter attribute. A value of "1" means the attribute shall be used and 0 means the attribute shall not be bused.
This field signals the presence of a filter attribute and has a binary representation. 1, the filter attribute should be used; if 0, the filter attribute is not used
filter Describes kinds of camera filters as a reference to a classification scheme that uses the mpeg7: termReferenceType defined in 7.6 of ISO / IEC 15938-5: 2003. The CS may be used for this purpose, the CameraFilterTypeCS defined in A.2.12.6.
Describes the types of camera filters with reference to a classification scheme using MPEG7.
ISOSpeedFlag This field, which is present in the binary representation, signals the presence of the ISOSpeed element. This field signals the existence of an ISO speed element and has a binary representation. This value is a value of "1" means that this element is present and 0 means that this element is nopresent. If 1, this element exists, and if it is 0, this element does not exist ISOSpeed Describes ISO speed based on ISO 12232: 2006. ISO ExposureValueFlag This field, which is present in the binary representation, signals the presence of the ExposureValue element. A value of "1" means that this element is present and 0 means that this element is nopresent.
This field signals the presence of an ExposureValue element and has a binary representation. If 1, this element exists and if it is 0, this element does not exist
ExposureValue Describes the exposure value. Describes the exposure value.
VideoFlag Describes image shooting mode. * j * for still image and "1" for video. 0 is for still images, 1 is for video. SensorType Describes type of sensor used. * j * for monochrome sensor, "1" for color sensor. 0 is a monochrome sensor, and 1 is a color sensor. ColorFilterArrayFlag This field, which is present in the binary representation, signals the presence of the ColorFilterArrayType element. A value of "1" means that this element is present and 0 means that this element is nopresent.
This field signals the presence of a color filter array type element (ColorFilterArrayType element) and has a binary representation. If 1, this element exists, and if it is 0, this element does not exist
ColorFilterArray Describes the color filter array applied to the image sensor of a camera0000
0001 Bayer
0010]
0011 CYYM
0100 CYGM
0101 RGBW Bayer
0110 RGBW # 1
0111 RGBW # 2
1000 RGBW # 3
1001-1111 Reserved

Describes a color filter array applied to the image sensor of a camera. ColorSpaceFlag This field, which is present in the binary representation, signals the presence of the ColorSpaceType element. A value of "1" means that this element is present and 0 means that this element is not prent.
This field signals the presence of a color space element and has a binary representation. 1, this element exists, and if it is 0, this element does not exist ColorSpaceType Describes the color space applied.
Describes applied color space BitDepthFlag This field, which is present in the binary representation, signals the presence of the BitDepth element. A value of "1" means that this element is present and 0 means that this element is nopresent.
This field signals the presence of a bit depth element and has a binary representation. 1, this element exists, and if it is 0, this element does not exist
BitDepth Describes applied bit depth Describes applied bit depth
SpectrumRangeFlag This field, which is present in the binary representation, signals the presence of the SpectrumRange element. A value of "1" means that this element is present and 0 means that this element is nopresent.
This field signals the presence of a Spectrum Range element and has a binary representation. 1, this element exists, and if it is 0, this element does not exist SpectrumRange Describes applied spectrum range that the camera perceived in terms of valueRangeType. Its default unit is nanometer (nm) .NOTE The minValue and the maxValue in the SensorCapabilityBaseType are not used for this sensor.

Describe the spectral range perceived by the camera sensor for valueRangeType. The basic unit of this is the nanometer.

Maximum and minimum values in SensorCapabilityBaseType are not used for camera sensors

ThermalRangeFlag This field, which is present in the binary representation, signals the presence of the ThermalRange element. A value of "1" means that this element is present and 0 means that this element is nopresent.
This field signals the presence of a ThermalRange element and has a binary representation. 1, this element exists, and if it is 0, this element does not exist
ThermalRange Describes applied thermal response range that the camera perceived in terms of valueRangeType. Its default unit is Celsius (C) .NOTE The minValue and the maxValue in the SensorCapabilityBaseType are not used for this sensor.

Describe the thermal response range recognized by the camera sensor for valueRangeType. The basic unit of this is the temperature (℃). The minimum and maximum values in SensorCapabilityBaseType are not used by the camera sensor

WhiteBalanceTempFlag This field, which is present in the binary representation, signals the presence of the WhiteBalanceTemp element. A value of "1" means that this element is present and 0 means that this element is nopresent.
This field signals the presence of a White Balance Temp element and has a binary representation. If 1, this element exists and if it is 0, this element does not exist
WhiteBalanceTemp Describes applied white balance temperature in Kelvin (K).
Describes the white balance temperature applied in Calvin units.
WhiteBalanceTintFlag This field, which is present in the binary representation, signals the presence of the WhiteBalanceTint element. This field signals the presence of a white balance tint element and has a binary representation. If 1, this element exists and if it is 0, this element does not exist
WhiteBalanceTint Describes the applied white balance tint value. Describes the applied white balance tint value.

<Use Case> For certain types of surveillance such as fire detection, night watch, etc., the features of the captured image are different. For example, thermal IR images used for fire detection or body heat detection at airports have other characteristics. Thus, by providing more metadata, the media processor can group, merge, and categorize images (video) from different sources.

(2) Response of IoMT & W

For IoMT & W, the present invention provides sufficient metadata to describe the data obtained by MThings. Syntax and semantics are given in binary format and can be converted or used in XML format, depending on the IoMT & W system and application.

There are many elements to describe the physical characteristics of IoMT & W devices / entities. The present invention provides location metadata of IoMT & W devices / entities.

(A) Geographical location

The geolocation metadata includes longitude, latitude, and orientation when acquiring a focal point. The geo-location metadata may be pre-set by the operator or may be obtained using appropriate sensors.

Table 5 shows the syntax.

Syntax Number of bits Mnemonic GeographicalPositioning { GeoPosition GeoPositionAttributes StationaryFlag One bslbf if (StationaryFlag == 1) { GeoPosition GeoPositionAttributes } } Syntax Number of bits Mnemonic GeoPositionAttributes { Longitude 32 fsfb Latitude 32 fsfb Altitude 32 fsfb AltUnitType One bslbf Yaw 32 Fsfb Pitch 32 fsfb Roll 32 fsfb YPRUnitType One bslbf } }

(B) Relative position The position of another geographic location, IoMT & W, which is absolute data, is provided relative to a preset point of origin.

Table 6 shows the syntax.

Syntax Number of bits Mnemonic LocalPositioning { LocalPosition LocalPositionAttributes StationaryFlag One bslbf if (StationaryFlag == 1) { LocalPosition LocalPositionAttributes } } Syntax Number of bits Mnemonic LocalPositionAttributes { X 32 fsfb Y 32 fsfb Z 32 fsfb XYZUnitType One bslbf Yaw 32 fsfb Pitch 32 fsfb Roll 32 fsfb YPRUnitType One bslbf } }

(C) Data Acquisition Sensor Feature Metadata Other geographic location, absolute data, IoMT & W location is provided relative to a preset point of origin.

The image (video) acquisition metadata is classified into two. One is the camera configuration metadata, and the other is the acquired image feature parameter.

(i) Camera settings

Table 7 shows the syntax.

Syntax Number of bits Mnemonic CameraSetting { focalLengthFlag One bslbf apertureFlag One bslbf shutterSpeedFlag One bslbf filterFlag One bslbf if (focalLengthFlag == 1) { focalLength 32 fsbf } if (apertureFlag == 1) { aperture 32 fsbf } if (shutterSpeedFlag == 1) { shutterSpeed 32 fsbf } if (filterFlag == 1) { filter 4 bslbf } } Name Definition focalLength Focal Length at the time of acquisition
Focal length when acquiring an image aperture Aperture setting Aperture setting shutterSpeed Shutter speed setting Shutter speed setting filter Applied filter Applied filter

(ii) Image characteristics Image characteristics

Table 8 shows the syntax.

Syntax Number of bits Mnemonic imageCharacteristicsData { imageStreamFlag One bslbf imageCharacteristics imageCharacteristicsAttributes if (imageStreamFlag == 1) { charUpdateFlag One bslbf if (charUpdateFlag == 1) { imageCharacteristics imageCharacteristicsAttributes } } Syntax Number of bits Mnemonic imageCharacteristicsAttributes { colorImageFlag One bslbf filterFlag One bslbf colorSpaceFlag One bslbf bitDepthFlag One bslbf waveLengthFlag One bslbf temperatureFlag One bslbf tintFlag One bslbf if (filterFlag == 1) { filter 4 bslbf } if (colorSpaceFlag == 1) { colorSpaceIDLength vluimsbf colorSpaceID colorSpaceIDLength * 8 UTF-8 } if (bitDepthFlag == 1) { bitDepth 8 fsbf } if (waveLengthFlag == 1) { minWaveLength 14 fsbf maxWaveLength 14 fsbf } if (temperatureFlag == 1) { 온도 14 fsbf } if (tintFlag == 1) { tint 16 simsbf } Name Definition imageStreamFlag '1' if video or image streaming
1 if it is a video or image stream. imageCharacteristics Defines characteristics of captured images (s) Defines characteristics of captured images charUpdateFlag '1' if any characteristic of currently captured image is different from the previous one If the current captured image is different from the previous captured image, colorImageFlag '0' if color, '1' if grayscale 0 if the captured image is color and 1 if the captured image is gray. filterFlag '1' if any filter is applied
If the filter is applied, 1 colorSpaceFlag '1' if colorSpace is defined 1 if a color space is defined bitDepthFlag '1' if bitDepth is defined
If the bit depth (bitDepth) is defined, waveLengthFlag '1' if waveLength range of captured image is defined
If the waveLenght range of the captured image is defined, temperatureFlag '1' if white balance temperature is defined If white balance temperature is defined, tintFlag '1' if white balance tint is defined If white balance tint is defined, colorSpaceIDLength Length of colorSpaceID Color space ID length colorSpaceID Identifying color space used in image capturing Color space ID used when capturing an image bitDepth Current bit depth setting Set current bit depth waveLength Current wavelength setting in nanometer Current wavelength setting in nanometers minWaveLength Minimum wavelength in nanometer Minimum wavelength in nanometers maxWaveLength Maximum wavelength in nanometer Maximum wavelength in nanometers 온도 Current white balance temperature setting in Kelvin Current white balance temperature setting in Calvin tint Current white balance tint setting Current white balance tint setting

(iii) Sound Sound

The following provides a sound transducer (microphone) feature for acquiring sound waves.

Table 9 shows the syntax.

Syntax Number of bits Mnemonic soundTransducerAttributes { transducerTypeFlag One bslbf transducerArrayFlag One bslbf probeTypeFlag One bslbf polarPatternTypeFlag One bslbf frequencyRangeFlag One bslbf frequencyResponseFlag One bslbf sensitivityFlag One bslbf } if (transducerTypeFlag == 1) { transducerType 4 fsbf } if (transducerArrayFlag == 1) { transducerArray 4 fsbf } if (probeTypeFlag == 1) { probeType 4 fsbf } if (polarPatternTypeFlag == 1) { polarPattern 4 fsbf } if (frequencyRangeFlag == 1) { minFrequency 24 fsbf maxFrequency 24 fsbf } if (frequencyResponseFlag == 1) { minResponseFrequency 24 fsbf maxResponseFrequency 24 fsbf } if (sensitivityFlag == 1) { sensitivity 8 fsbf }

Name Definition transducerType Defines type of transducer
Describe the type of transducer
0000 Reserved
0001 Condenser
0010 Dynamic
0011 Ribbon
0100 Carbon
0101 Piezoelectric
0110 Fiber Optic
0111 Laser
1000 Liquid
1001 MEMS
1010-1111 Reserved transducerArray Defines array types of transducer probes Describes array types of transducer probes
0000 Reserved
0001 single array
0010 linear array
0011 curvilinear
0100 phased
0101 annular
0110 matrix array
0111-1111 Reserved probeType Defines probing type of transducer Describes the probing type of a transducer
0000 Reserved
0001 linear probe
0010 sector probe
0011]
0100 trapezoid probe
0101-1111 Reserved polarPattern Defines the polar pattern of a transducer transducer.
0000 Reserved
0001 Omnidirectional
Bi-directional (or Figure 8)
0011 Subcardioid
0100 Cardioid
0101 Hypercardioid
0110 Supercardioid
0111 Shotgun
1000-1111 Reserved minFrequency Minimum pickup frequency in Hz
Minimum pickup frequency in Hz maxFrequency Maximum pickup frequency in Hz Maximum pickup frequency in Hz frequencyResponseFlag '0' if Flat frequency response
'1' if Tailored frequency response
0, it is a flat frequency response, and when it is 1, it is a tailored frequency response. minResponseFreqeuncy Minimum Pick response frequency in Hz
Minimum pickup response frequency in Hz maxResponseFrequency Maximum Pick response frequency in Hz sensitivity Pick sensitivity of transducer in pickup sensitivity of transducer in mV / PamV / Pa

* Camera ability

The image (video) acquisition metadata is classified into two. One is the camera setting metadata, and the other is the acquired image feature parameter.

Table 11 shows the syntax.

<complexType name = "CameraSensorCapabilityType">
<complexContent>
<extension base = "cidl: SensorCapabilityBaseType">
<sequence>
<element name = "SupportedResolutions" type = "scdv: ResolutionListType" minOccurs = "0"/>
<element name = "FocalLengthRange" type = "scdv: ValueRangeType" minOccurs = "0"/>
<element name = "ApertureRange" type = "scdv: ValueRangeType" minOccurs = "0"/>
<element name = "ShutterSpeedRange" type = "scdv: ValueRangeType" minOccurs = "0"/>
<element name = "ISOSpeedRange" type = "scdv: ValueRangeType" minOccurs = "0"/>
<element name = "ExposureValueRange" type = "scdv: ValueRangeType" minOccurs = "0"/>
<element name = "ColorFilterArrayType" type = "scdv: ColorFilterArrayListType" minOccurs = "0"/>
<element name = "Video" type = "boolean" minOccurs = "0"/>
<element name = "SensorType" type = "boolean" minOccurs = "0"/>
<element name = "ColorSpaceType" type = "string" minOccurs = "0"/>
<element name = "BitDepthRange" type = "scdv: ValueRangeType" minOccurs = "0"/>
<element name = "SpectrumRange" type = "scdv: ValueRangeType" minOccurs = "0"/>
<element name = "ThermalRange" type = "scdv: ValueRangeType" minOccurs = "0"/>
<element name = "WhiteBalanceTempRange" type = "scdv: ValueRangeType" minOccurs = "0"/>
<element name = "WhiteBalanceTintRange" type = "scdv: ValueRangeType" minOccurs = "0"/>
</ sequence>
</ extension>
</ complexContent>
</ complexType>

<complexType name = "ResolutionListType">
<sequence>
<element name = "Resolution" type = "scdv: ResolutionType" maxOccurs = "unbounded"/>
</ sequence>
</ complexType>

<complexType name = "ResolutionType">
<sequence>
<element name = "Width" type = "nonNegativeInteger"/>
<element name = "Height" type = "nonNegativeInteger"/>
</ sequence>
</ complexType>

<complexType name = "ValueRangeType">
<sequence>
<element name = "MaxValue" type = "float"/>
<element name = "MinValue" type = "float"/>
</ sequence>
</ complexType>

<simpleType name = "ColorFilterArrayListType">
<restriction base = "string">
<enumeration value = "Bayer"/>
<enumeration value = "RGBE"/>
<enumeration value = "CYYM"/>
<enumeration value = "CYGM"/>
<enumeration value = "RGB Bayer"/>
<enumeration value = "RGBW # 1"/>
<enumeration value = "RGBW # 2"/>
<enumeration value = "RGBW # 3"/>
</ restriction>
</ simpleType>

Table 12 shows the syntax.

CameraSensorCapabilityType { Number of bits Mnemonic SupportedResolutionsFlag One bslbf FocalLengthRangeFlag One bslbf ApertureRangeFlag One bslbf ShutterSpeedRangeFlag One bslbf ISORangeFlag One bslbf ExposureValueRangeFlag One bslbf ColorFilterFlag One bslbf VideoFlag One bslbf SensorType One bslbf ColorSpaceFlag One bslbf BitDepthRangeFlag One bslbf SpectrumRangeFlag One bslbf ThermalRangeFlag One bslbf WhiteBalanceTempRangeFlag One bslbf WhiteBalanceTintRangeFlag One bslbf SensorCapabilityBase SensorCapabilityBaseType if (SupportedResolutionsFlag) { SupportedResolutions ResolutionListType } if (FocalLengthRangeFlag) { FocalLengthRange ValueRangeType } if (ApertureRangeFlag) { ApertureRange ValueRangeType } if (ShutterSpeedRangeFlag) { ShutterSpeedRange ValueRangeType } if (ISOSpeedRangeFlag) { ISOSpeedRange ValueRangeType } if (ExposureValueRangeFlag) { ExposureValueRange ValueRangeType } if (ColorFilterArrayFlag) { ColorFilterArrayType ColorFilterArrayListType } if (ColorSpaceFlag) { ColorSpaceTypeLength vluimsbf ColorSpaceType ColorSpaceTypeLength * 8 UTF-8 } if (BitDepthRangeFlag) { BitDepthRange ValueRangeType } if (SpectrumRangeFlag) { SpectrumRange ValueRangeType } if (ThermalRangeFlag) { ThermalRange ValueRangeType } if (WhiteBalanceTempRangeFlag) { WhiteBalanceTempRange ValueRangeType } if (WhiteBalanceTintRangeFlag) { WhiteBalanceTintRange ValueRangeType } } ResolutionListType { LoopResolution vluimsbf for (k = 0; k <LoopResolution; k ++) { Resolution [k] ResolutionType } } ResolutionType { Width 32 uimsbf Height 32 uimsbf } ValueRangeType { MaxValue 32 fsbf MinValue 32 fsbf }

Table 13 shows the semantics for CameraSensorCapabilityType.

Name Definition CameraSensorCapabilityType Tool for describing a camera sensor capability.

Tools for explaining camera sensor capabilities SupportedResolutionsFlag This field, which is present in the binary representation, signals the presence of the SupportedResolutions element. A value of "1" means that this element is present and 0 means that this element is nopresent.
This field signals the presence of a SupportedResolutions element and has a binary representation. 1, this element exists, and if it is 0, this element does not exist SupportedResolutions Describes a list of resolutions that the camera can support. ResolutionListType Describes a type of resolution list which consists of resolution type elements. ResolutionType Describes a type of resolution which is composed of a Width element and a Height element. Describes a resolution type consisting of a Width element and a Height element. Width Describes the width of the resolution that the camera can perceive. Height Describes the height of the camera's perceived resolution. FocalLengthRangeFlag This field, which is present in the binary representation, signals the presence of the FocalLengthRange element. This field signals the presence of a focal length element and has a binary representation. If 1, this element exists, and if it is 0, this element does not exist FocalLengthRange Describes the range of the focal length that the camera sensor can perceive in terms of ValueRangeType. Its default unit is millimeters (mm) .NOTE The minValue and the maxValue in the SensorCapabilityBaseType are not used for this sensor.

If the subject is in focus for mm units, the distance between the lenses and the image sensor is described.
Minimum and maximum values in SensorCapabilityBaseType are not used for camera sensors ValueRangeType Defines the range of values that the sensor can perceive. MaxValue Describes the maximum value that the sensor can perceive. MinValue Describes the minimum value that the sensor can perceive. ApertureRangeFlag This field, which is present in the binary representation, signals the presence of the ApertureRange element. This field signals the presence of the aperture element and has a binary representation. A value of "1" means that this element is present and 0 means this element is not present. If 1, this element exists, and if it is 0, this element does not exist ApertureRange Describes the range of the camera sensor that can be used in the sensor.

Describes the range of apertures that the camera sensor can perceive for valueRangeType. The minimum and maximum values in SensorCapabilityBaseType are not used by the camera sensor
ShutterSpeedRangeFlag This field, which is present in the binary representation, signals the presence of the ShutterSpeedRange element. A value of "1" means that this element is present and 0 means that this element is nopresent.
This field signals the presence of a ShutterSpeedRange element and has a binary representation. If 1, this element exists, and if it is 0, this element does not exist
ShutterSpeedRange Describes the range of the shutter speed that the camera sensor can perceive in terms of valueRangeType. Its default unit is seconds (sec) .NOTE The minValue and the maxValue in the SensorCapabilityBaseType are not used for this sensor.

Describes the range of shutter speeds that the camera sensor can perceive for valueRangeType

The minimum and maximum values in SensorCapabilityBaseType are not used by the camera sensor ISOSpeedRangeFlag This field, which is present in the binary representation, signals the presence of the ISOSpeedRange element. This field signals the presence of an ISO speed range element (ISOSpeedRange element) and has a binary representation. If 1, this element exists, and if it is 0, this element does not exist ISOSpeedRange Describes the range of ISO Speed based on ISO 12232: 2006 that the camera sensor can perceive in terms of valueRangeType.NOTE The minValue and the maxValue in the SensorCapabilityBaseType are not used for this sensor.

Describes ISO speed range based on ISO. The minimum and maximum values in SensorCapabilityBaseType are not used by the camera sensor ExposureValueRangeFlag This field, which is present in the binary representation, signals the presence of the ExposureValueRange element. A value of "1" means that this element is present and 0 means that this element does not present.
This field signals the presence of an ExposureValueRange element and has a binary representation. If 1, this element exists and if it is 0, this element does not exist
ExposureValueRange Describes the range of the exposure value that the camera sensor can perceive in terms of valueRangeType.NOTE The minValue and the maxValue in the SensorCapabilityBaseType are not used for this sensor.

ValueRangeType describes the range of exposure values that the camera sensor can recognize. The minimum and maximum values in SensorCapabilityBaseType are not used by the camera sensor VideoFlag A value of 0 means that this camera sensocan only shoot still image. A value of " 1 "
0 if the camera sensor shoots only the still image. If the camera sensor records video, SensorType A value of * j * means that this camera can only perceive monochrome image. A value of " 1 " means that the camera can perceive the color image.
0 if the camera sensor can recognize monochrome images. Happy? If the color image can be recognized, ColorFilterArrayFlag This field, which is present in the binary representation, signals the presence of the ColorFilterArrayType element. A value of "1" means that this element is present and 0 means that this element is nopresent.
This field signals the presence of a color filter array type element (ColorFilterArrayType element) and has a binary representation. If 1, this element exists, and if it is 0, this element does not exist ColorFilterArrayType Describes the color filter array applied to the image sensor of the camera.
0000 Reserved
0001 Bayer
0010]
0011 CYYM
0100 CYGM
0101 RGBW Bayer
0110 RGBW # 1
0111 RGBW # 2
1000 RGBW # 3
1001-1111 Reserved ColorSpaceFlag This field, which is present in the binary representation, signals the presence of the ColorSpaceType element. A value of "1" means that this element is present and 0 means that this element is nopresent.
This field signals the presence of a color space element and has a binary representation. 1, this element exists, and if it is 0, this element does not exist ColorSpaceType Describes the color space applied.
Describes applied color space BitDepthRangeFlag This field, which is present in the binary representation, signals the presence of the BitDepthRange element. A value of "1" means that this element is present and 0 means that this element is nopresent. BitDepthRange Describes the range of the camera sensor in terms of value in the range of values. RangeType.NOTE The minValue and the maxValue in the SensorCapabilityBaseType are not used for this sensor.

Describes the range of bit depths that the camera sensor can perceive for valueRangeType.

The minimum and maximum values in SensorCapabilityBaseType are not used by the camera sensor SpectrumRangeFlag This field, which is present in the binary representation, signals the presence of the SpectrumRange element. A value of "1" means that this element is present and 0 means that this element is nopresent.
This field signals the presence of a Spectrum Range element and has a binary representation. 1, this element exists, and if it is 0, this element does not exist SpectrumRange Describes the spectrum range that the camera sensor can perceive in terms of valueRangeType. Its default unit is nanometer (nm) .NOTE The minValue and the maxValue in the SensorCapabilityBaseType are not used for this sensor.

Describe the spectral range perceived by the camera sensor for valueRangeType. The basic unit of this is the nanometer.

Maximum and minimum values in SensorCapabilityBaseType are not used for camera sensors
ThermalRangeFlag This field, which is present in the binary representation, signals the presence of the ThermalRange element. A value of " 1 " means that this element is present and 0 means that the elent is not present.
This field signals the presence of a ThermalRange element and has a binary representation. 1, this element exists, and if it is 0, this element does not exist
ThermalRange Describes the thermal response range that the camera sensor can perceive in terms of valueRangeType. Its default unit is Celsius (C) .NOTE The minValue and the maxValue in the SensorCapabilityBaseType are not used for this sensor.

Describe the thermal response range recognized by the camera sensor for valueRangeType. The basic unit of this is the temperature (℃). The minimum and maximum values in SensorCapabilityBaseType are not used by the camera sensor
WhiteBalanceTempRangeFlag This field, which is present in the binary representation, signals the presence of the WhiteBalanceTempRange element. A value of "1" means that this element is present and 0 means that this element is nopresent.
This field signals the presence of a White Balance Temp element and has a binary representation. If 1, this element exists and if it is 0, this element does not exist
WhiteBalanceTempRange Describes the white balance temperature range that the camera sensor can perceive in terms of valueRangeType. Its default unit is Kelvin (K) .NOTE The minValue and the maxValue in the SensorCapabilityBaseType are not used for this sensor.

Describes the white balance temperature range that the camera sensor can perceive for valueRangeType. Calvin is the basic unit. Minimum and maximum values are not used in SensorCapabilityBaseType
WhiteBalanceTintFlag This field, which is present in the binary representation, signals the presence of the WhiteBalanceTintRange element. A value of "1" means that this element is present and 0 means that this element is nopresent.
This field signals the presence of a white balance tint element and has a binary representation. If 1, this element exists and if it is 0, this element does not exist

WhiteBalanceTintRange Describes the range of white balance tint values that the camera can perceive in terms of valueRangeType.NOTE The minValue and the maxValue in the SensorCapabilityBaseType are not used for this sensor.

Describes the range of white balance tint values that the camera sensor can perceive for valueRangeType. Minimum and maximum values in SensorCapabilityBaseType can not be used in camera sensor

*sound

The following provides a sound transducer feature (microphone) for sound wave acquisition.

Table 14 shows the semantics.

<complexType name = "microphoneCapabilityType">
<complexContent>
<extension base = "cidl: SensorCapabilityBaseType">
<sequence>
<element name = "micorphoneType" type = "scdv: mcrophoneListType" minOccurs = "0"/>
<element name = "transcuderArrayType" type = "scdv: transducerArrayListType" minOccurs = "0"/>
<element name = "probtType" type = "scdv: probeListType" minOccurs = "0"/>
<element name = "polarPatternType" type = "scdv: polarPatternListType" minOccurs = "0"/>
<element name = "frequencyRange" type = "scdv: frequencyRangeType" minOccurs = "0"/>
<element name = "responseType" type = "scdv: frequencyRangeType" minOccurs = "0"/>
<element name = "pickSensitivity" type = "float" minOccurs = "0"/>
</ sequence>
</ extension>
</ complexContent>
</ complexType>

<simpleType name = "microphoneListType">
<restriction base = "string">
<enumeration value = "condenser"/>
<enumeration value = "dynamic"/>
<enumeration value = "ribbon"/>
<enumeration value = "carbon"/>
<enumeration value = "piezoelectric"/>
<enumeration value = "fiber optic"/>
<enumeration value = "laser"/>
<enumeration value = "liquied"/>
<enumeration value = "MEMS"/>
</ restriction>
</ simpleType>

<simpleType name = "transducerArrayListType">
<restriction base = "string">
<enumeration value = "single array"/>
<enumeration value = "linear array"/>
<enumeration value = "curvilinear"/>
<enumeration value = "phased"/>
<enumeration value = "annular"/>
<enumeration value = "matrix array"/>
<enumeration value = "MEMS"/>
</ restriction>
</ simpleType>

<simpleType name = "probeListType">
<restriction base = "string">
<enumeration value = "linear"/>
<enumeration value = "sector"/>
<enumeration value = "convex"/>
<enumeration value = "carbon"/>
<enumeration value = "trapezoid"/>
</ restriction>
</ simpleType>

<simpleType name = "polarPatternListType">
<restriction base = "string">
<enumeration value = "omnidirectional"/>
<enumeration value = "bi-directional"/>
<enumeration value = "subcardioid"/>
<enumeration value = "cardioid"/>
<enumeration value = "hypercardioid"/>
<enumeration value = "supercardioid"/>
<enumeration value = "shotgun"/>
</ restriction>
</ simpleType>

<complexType name = "frequencyRangeType">
<sequence>
<element name = "minFrequency" type = "float"/>
<element name = "maxFrequency" type = "float"/>
</ sequence>
</ complexType>
microphoneCapabilityType { Number of bits Mnemonic microphoneTypeFlag One bslbf transducerArrayFlag One bslbf probeTypeFlag One bslbf polarPatternTypeFlag One bslbf frequencyRangeFlag One bslbf frequencyResponseTypeFlag One bslbf sensitivityFlag One bslbf SensorCapabilityBase SensorCapabilityBaseType if (microphoneTypeFlag == 1) { microphoneType microphoneListType } if (transducerArrayFlag == 1) { transducerArrayType trnasducerArrayListType } if (probeTypeFlag == 1) { probeType 4 probeListType } if (polarPatternTypeFlag == 1) { polarPattern 4 polarPatternListType } if (frequencyRangeFlag == 1) { frequencyRange frequencyRangeType } if (responseTypeFlag == 1) { responseFrequency frequencyRangeType } if (sensitivityFlag == 1) { pickSensitivity 32 fsbf } microphoneListType { microphoneType 4 bslbf } transducerArrayListType { transducerArrayType 4 blsbf } probeListType { probeType 4 blsbf } polarPatternListyType { polarPattern 4 blsbf } frequencyRangeType { minFrequency 32 uimsbf maxFrequency 32 uimsbf }

Table 15 shows the semantics.

Name Definition microphoneType Defines type of microphone
Describe the type of microphone
0000 Reserved
0001 Condenser
0010 Dynamic
0011 Ribbon
0100 Carbon
0101 Piezoelectric
0110 Fiber Optic
0111 Laser
1000 Liquid
1001 MEMS
1010-1111 Reserved transducerArrayType Defines array types of transducer probes Describes array types of transducer probes
0000 Reserved
0001 single array
0010 linear array
0011 curvilinear
0100 phased
0101 annular
0110 matrix array
0111-1111 Reserved probeType Defines probing type of transducer Describes the probing type of a transducer
0000 Reserved
0001 linear probe
0010 sector probe
0011]
0100 trapezoid probe
0101-1111 Reserved polarPattern Defines the polar pattern of a transducer transducer.
0000 Reserved
0001 Omnidirectional
Bi-directional (or Figure 8)
0011 Subcardioid
0100 Cardioid
0101 Hypercardioid
0110 Supercardioid
0111 Shotgun
1000-1111 Reserved frequencyRange Pickup frequency range in Hz
Pickup frequency in Hz responseTypeFlag '0' if Flat frequency response
'1' if Tailored frequency response
0, it is a flat frequency response, and when it is 1, it is a tailored frequency response. responseFrequency Pick response frequency range for tailored frequency response

Pickup response frequency range for microphone tailored frequency response minFreqeuncy Minimum frequency in Hz
Minimum frequency in Hz maxFrequency Maximum frequency in Hz Maximum frequency in Hz pickSensitivity Pick sensitivity of transducer in pickup sensitivity of transducer in mV / PamV / Pa

The following semantics describe the microphone capability. The microphone has an ID of " MCID_001 ". It is a condenser microphone with a cardioid pattern of 20Hz-20kHz with a tailored frequency pickup range between 20Hz-8kHz.

Table 16 shows the syntax.

<cidl: SensorDeviceCapability xsi: type = "scdv: microphoneCapabilityType" id = "MCID_001">
<microphoneType>"condenser"</microphoneType>
<polarPatternType>"cardioid"</polarPatternType>
<scdv: frequencyRange>
<scdv: minFrequency> 20 </ scdv: minFrequency>
<scdv: maxFrequency> 20000 </ scdv: maxFrequency>
</ scdv: frequencyRange>
<scdv: responseType>
<scdv: minFrequency> 20 </ scdv: minFrequency>
<scdv: maxFrequency> 8000 </ scdv: maxFrequency>
</ scdv: responseType>
</ cidl: SensorDeviceCapability>

(3) Additional sound metadata for media orchestration

Depending on the type of application or input sound transducer, the characteristics of the capture sound are different. More detailed metadata about the characteristics of the transducer is needed so that the media processor can successfully process (e.g., group, merge, split, etc.) the acquired data.

The present invention provides information for the media processor with respect to the features of the input sound transducer.

Table 17 shows the syntax.

soundTransducerAttributes { Number of bits Mnemonic transducerOrientationFlag One bslbf transducerLocationFlag One bslbf transducerTypeFlag One bslbf transducerArrayFlag One bslbf probeTypeFlag One bslbf polarPatternTypeFlag One bslbf frequencyRangeFlag One bslbf frequencyResponseFlag One bslbf sensitivityFlag One bslbf if (transducerOrientationFlag == 1) { transducerOrientation See above OrientationSensorType } if (transducerLocationFlag == 1) { transducerLocation See above GlobalPositionSensorType } if (transducerTypeFlag == 1) { transducerType 4 fsbf } if (transducerArrayFlag == 1) { transducerArray 4 fsbf } if (probeTypeFlag == 1) { probeType 4 fsbf } if (polarPatternTypeFlag == 1) { polarPattern 4 fsbf } if (frequencyRangeFlag == 1) { minFrequency 24 fsbf maxFrequency 24 fsbf } if (frequencyResponseFlag == 1) { minResponseFrequency 24 fsbf maxResponseFrequency 24 fsbf } if (sensitivityFlag == 1) { sensitivity 8 fsbf }

Table 18 shows the semantics.

Name Definition transducerOrientation Defines direction of sound transducer
Defines the direction of the sound transducer transducerLocation Defines location of sound transducer Defines the location of the sound transducer transducerType Defines the type of transducer transducer.
0000 Reserved
0001 Condenser
0010 Dynamic
0011 Ribbon
0100 Carbon
0101 Piezoelectric
0110 Fiber Optic
0111 Laser
1000 Liquid
1001 MEMS
1010-1111 Reserved transducerArray Defines array types of transducer probes Defines the array type of the transducer probe
0000 Reserved
0001 single array
0010 linear array
0011 curvilinear
0100 phased
0101 annular
0110 matrix array
0111-1111 Reserved probeType Defines the probing type of the transducer transducer.
0000 Reserved
0001 linear probe
0010 sector probe
0011]
0100 trapezoid probe
0101-1111 Reserved polarPattern Defines polar pattern of transducer Defines the polar pattern of the transducer
0000 Reserved
0001 Omnidirectional
Bi-directional (or Figure 8)
0011 Subcardioid
0100 Cardioid
0101 Hypercardioid
0110 Supercardioid
0111 Shotgun
1000-1111 Reserved minFrequency Minimum pickup frequency in Hz
Minimum pickup frequency in Hz maxFrequency Maximum pickup frequency in Hz Maximum pickup frequency in Hz frequencyResponseFlag '0' if Flat frequency response
'1' if Tailored frequency response
If 0, it is a flat frequency response, and if it is 1, it is a tailored frequency response. minResponseFreqeuncy Minimum Pick response frequency in Hz
Minimum pickup response frequency in Hz maxResponseFrequency Maximum Pick response frequency in Hz sensitivity Pick sensitivity of transducer in mV / Pa
Pickup sensitivity of the transducer in mV / Pa

Depending on the characteristics of the transducer, the media processor can identify the purpose of the acquired sound wave and properly process it. For example, if the transducer type is a condenser with a tailored 2-20 kHz cardioid pattern in the frequency range between 2-8 kHz, the transducer (microphone) can be recognized as a live vocal. Similarly, when the frequency range is 5 MHz to 10 MHz, the transducer is used for diagnostics ultrasonic.

Table 19 shows the syntax.

microphoneSensorType { Number of bits Mnemonic microphoneGlobalPositionFlag One bslbf microphoneLocalPositionFlag One bslbf microphoneTypeFlag One bslbf transducerArrayFlag One bslbf probeTypeFlag One bslbf polarPatternTypeFlag One bslbf frequencyRangeFlag One bslbf frequencyResponseTypeFlag One bslbf sensitivityFlag One bslbf SensedInfoBaseType SensedInfoBaseType if (microphoneGlobalPositionFlag == 1) { microphoneGlobalPosition GlobalPositionSensorType microphoneAltitude AltitudeSensorType microphoneOrientation OrientationSensorType } if (microphoneLocalPositionFlag == 1) { microphoneLocalPositionFlag PositionSensorType microphoneOrientation OrientationSensorType } if (microphoneTypeFlag == 1) { microphoneType microphoneListType } if (transducerArrayFlag == 1) { transducerArrayType trnasducerArrayListType } if (probeTypeFlag == 1) { probeType 4 probeListType } if (polarPatternTypeFlag == 1) { polarPattern 4 polarPatternListType } if (frequencyRangeFlag == 1) { frequencyRange frequencyRangeType } if (responseTypeFlag == 1) { responseFrequency frequencyRangeType } if (sensitivityFlag == 1) { pickSensitivity 32 fsbf } microphoneListType { microphoneType 4 bslbf } transducerArrayListType { transducerArrayType 4 blsbf } probeListType { probeType 4 blsbf } polarPatternListyType { polarPattern 4 blsbf } frequencyRangeType { minFrequency 32 uimsbf maxFrequency 32 uimsbf }

Table 20 shows semantics.

Name Definition microphoneGlobalPosition Defines global positioning sensor based position information of microphone

Define a global position sensor based on microphone position information microphoneAltitude Defines altitude of microphone Defines the altitude of the microphone microphoneOrientation Defines orientation of microphone Defines the direction of the microphone microphoneLocalPosition Defines relative location of microphone Defines the relative position of the microphone transducerLocation Defines location of sound transducer Defines the location of the sound transducer microphoneType Defines type of microphone Defines the type of microphone
0000 Reserved
0001 Condenser
0010 Dynamic
0011 Ribbon
0100 Carbon
0101 Piezoelectric
0110 Fiber Optic
0111 Laser
1000 Liquid
1001 MEMS
1010-1111 Reserved transducerArrayType Defines array types of transducer probes Defines array types of transducer probes
0000 Reserved
0001 single array
0010 linear array
0011 curvilinear
0100 phased
0101 annular
0110 matrix array
0111-1111 Reserved probeType Defines probing type of transducer Defines transducer probing type
0000 Reserved
0001 linear probe
0010 sector probe
0011]
0100 trapezoid probe
0101-1111 Reserved polarPattern Defines polar pattern of transducer Defines transducer polar pattern
0000 Reserved
0001 Omnidirectional
Bi-directional (or Figure 8)
0011 Subcardioid
0100 Cardioid
0101 Hypercardioid
0110 Supercardioid
0111 Shotgun
1000-1111 Reserved frequencyRange Pickup frequency range in Hz
Pickup frequency range in Hz responseTypeFlag '0' if Flat frequency response
'1' if Tailored frequency response

If 0, it is a flat frequency response, and if it is 1, it is a tailored frequency response. responseFrequency Pick response frequency range for tailored frequency response

Pickup response frequency range for tailored frequency response microphones minFreqeuncy Minimum frequency in Hz
Minimum frequency in Hz maxFrequency Maximum frequency in Hz Maximum frequency in Hz pickSensitivity Pick sensitivity of transducer in pickup sensitivity of transducer in mV / PamV / Pa

The following semantics describe the microphone capability. The microphone has an ID of " MCID_001 ". It is a condenser microphone with a cardioid pattern of 20Hz-20kHz with a tailored frequency pickup range between 20Hz-8kHz.

Table 21 shows the semantics.

<cidl: SensorDeviceCapability xsi: type = "scdv: microphoneCapabilityType" id = "MCID_001">
<microphoneType>"condenser"</microphoneType>
<polarPatternType>"cardioid"</polarPatternType>
<scdv: frequencyRange>
<scdv: minFrequency> 20 </ scdv: minFrequency>
<scdv: maxFrequency> 20000 </ scdv: maxFrequency>
</ scdv: frequencyRange>
<scdv: responseType>
<scdv: minFrequency> 20 </ scdv: minFrequency>
<scdv: maxFrequency> 8000 </ scdv: maxFrequency>
</ scdv: responseType>
</ cidl: SensorDeviceCapability>

&Lt; Embodiment 2 >

* Sensor Capability Description

According to one embodiment of the present invention, the sensor capability of each sensor is described. In order to determine the position of the sensors, a global coordinate is defined for the sensor which depends on the user's real world environment. The sensor capability specification of each device is derived from the sensor capability basic type (SensorCapabilityBaseType).

The origin of the global coordinates for the sensor is at the user's location in the right handed coordinate system. The Y axis represents the direction of gravity and the Z axis is the direction of the upper right corner of the screen. The X axis is in the opposite direction to the user's position.

* Sensor capability base type (Sensor capability base type)

Table 22 shows the semantics for the Sensor capability base type.

<complexType name = "SensorCapabilityBaseType" abstract = "true">
<complexContent>
<extension base = "dia: TerminalCapabilityBaseType">
<sequence>
<element name = "Accuracy" type = "cidl: AccuracyType" minOccurs = "0"/>
</ sequence>
<attributeGroup ref = "cidl: sensorCapabilityBaseAttributes"/>
</ extension>
</ complexContent>
</ complexType>

<complexType name = "AccuracyType" abstract = "true"/>

<complexType name = "PercentAccuracy">
<complexContent>
<extension base = "cidl: AccuracyType">
<attribute name = "value" type = "mpeg7: zeroToOneType"/>
</ extension>
</ complexContent>
</ complexType>

<complexType name = "ValueAccuracy">
<complexContent>
<extension base = "cidl: AccuracyType">
<attribute name = "value" type = "float"/>
</ extension>
</ complexContent>
</ complexType>

Table 23 shows the semantics.

SensorCapabilityBaseType { Number of bits Mnemonic AccuracyFlag One bslbf TerminalCapabilityBase TerminalCapabilityBaseType if (AccuracyFlag) { Accuracy AccuracyType } SensorCapabilityBaseAttributes SensorCapabilityBaseAttributesType } AccuracyType { AccuracySelect 2 bslbf    if (AccuracySelect == 00) {       PercentAccuracy 32 fsbf    } else if (AccuracySelect == 01) {       ValueAccuracy 32 fsbf    } }

Table 24 shows the semantics for SensorCapabilityBaseType.

Name Definition Sensor capability basic type

SensorCapabilityBaseType SensorCapabilityBaseType shall extend dia: TeminalCapabilityBaseType as defined in ISO / IEC 21000-7 and provides a base abstract type for a subset of types of sensor device capability metadata types.

The sensor capability base type is extended to the terminal capability base type (TeminalCapabilityBaseType) and provides a base abstract type for a subset of the types defined as part of the sensor device capability metadata types. Accuracy flag
AccuracyFlag This field, which is present in the binary representation, signals the presence of the activation attribute. A value of "1" means the attribute shall be used and 0 means the attribute shall be used.

This field signals the presence of an activation attribute and has a binary representation. If 1, the activation attribute is used; if 0, the activation attribute is not used. Accuracy Describes the degree of closeness of a quantity to its actual value in AccuracyType.

Describe the degree of closeness between the measured quantity and the actual value in the accuracy type. Sensor Capabilities Basic Properties
sensorCapabilityBase Attributes Describes a group of attributes for the sensor capabilities.

Describe a group of attributes for sensor capabilities

Table 25 shows the semantics for AccuracyType.

Name Definition Accuracy Type

AccuracyType
Becomes a parent type providing a choice of accuracy in either relative value or absolute value.

It can be a parent type that provides a choice that describes the accuracy in either relative or absolute values. Select Accuracy
AccuracySelect This field, which is present only in the binary representation, will be used with accuracy. * j * means that the PercentAccuracy type shall be used, and "1" means that the ValueAccuracy type shall be used.
This field describes whether an accuracy scheme is used and has a binary representation. If 0, the PercentAccuracy type is used; if 1, the ValueAccuracy type is used. Percent Accuracy PercentAccuracy Describes the degree of closeness of a measured quantity to its actual value in a relative way using a value ranging from 0 to 1.0.

Describes the relative proximity of the measured value to the actual value using values ranging from 0 to 1.0. Value value Provides an actual value in a relative way for accuracy where 0 means 0% accuracy and value 1.0 means 100% accuracy. ISO / IEC 15938-5: 2003.

Provides the actual value in a relative manner for accuracy with a value of 1 for 0% accuracy and 100% accuracy. Value Accuracy ValueAccuracy Describes the degree of closeness of a quantity to its actual value in an absolute value of given unit.
Describes the proximity between the measured amount and the actual value for the absolute value of a given unit value
Value Provides an actual value in an absolute way, where the value means the possible range of error as (-value, + value) of given unit.

A value of (-value, + value) for a given unit, meaning a range of possible errors, and provides the actual value in an absolute manner.

Using the sensor capability basic type (SensorCapabilityBaseType), individual sensor device capability types are provided.

* Sensor capability base attributes

Table 26 shows the syntax for the Sensor capability base attributes.

<attributeGroup name = "sensorCapabilityBaseAttributes">
<attribute name = "unit" type = "mpegvct: unitType" use = "optional"/>
<attribute name = "maxValue" type = "float" use = "optional"/>
<attribute name = "minValue" type = "float" use = "optional"/>
<attribute name = "offset" type = "float" use = "optional"/>
<attribute name = "numOfLevels" type = "nonNegativeInteger" use = "optional"/>
<attribute name = "sensitivity" type = "float" use = "optional"/>
<attribute name = "SNR" type = "float" use = "optional"/>
</ attributeGroup>

SensorCapabilityBaseAttributesType { Number of bits Mnemonic    unitFlag One bslbf   maxValueFlag One bslbf minValueFlag One bslbf offsetFlag One bslbf numOfLevelsFlag One bslbf sensitivityFlag One bslbf SNRFlag One bslbf if (unitFlag) { unit 8 bslbf    } if (maxValueFlag) { maxValue 32 fsbf    } if (minValueFlag) { minValue 32 fsbf    } if (offsetFlag) { offset 32 fsbf    } if (numOfLevelsFlag) { numOfLevels 16 uimsbf    } if (sensitivityFlag) { sensitivity 32 fsbf    } if (SNRFlag) { SNR 32 fsbf    } }

Table 28 shows the semantics for SensorCapabilityBaseAttributes.

Name Definition Sensor Capabilities Basic Properties

sensorCapabilityBase Attributes Describes a group of attributes for the sensor capabilities.

Describe a group of attributes for sensor capabilities Unit flag

unitFlag This field, which is present in the binary representation, signals the presence of the activation attribute. A value of "1" means the attribute shall be used and 0 means the attribute shall not be bused.

This field signals the presence of an activation attribute and has a binary representation. If 1, the activation attribute is used; if 0, the activation attribute is not used. The maximum value flag maxValueFlag This field, which is present in the binary representation, signals the presence of the activation attribute. A value of "1" means the attribute shall be used and 0 means the attribute shall not be used.

This field signals the presence of an activation attribute and has a binary representation. If 1, the activation attribute is used; if 0, the activation attribute is not used. The minimum value flag minValueFlag This field, which is present in the binary representation, signals the presence of the activation attribute. A value of "1" means the attribute shall be used and 0 means the attribute shall not be bused.

This field signals the presence of an activation attribute and has a binary representation. If 1, the activation attribute is used; if 0, the activation attribute is not used. Offset flag offsetFlag This field, which is present in the binary representation, signals the presence of the activation attribute. A value of "1" means the attribute shall be used and 0 means the attribute shall not be bused.

This field signals the presence of an activation attribute and has a binary representation. If 1, the activation attribute is used; if 0, the activation attribute is not used. Level Flag Number numOfLevelsFlag This field, which is present in the binary representation, signals the presence of the activation attribute. A value of "1" means the attribute shall be used and 0 means the attribute shall not be bused.
This field signals the presence of an activation attribute and has a binary representation. If 1, the activation attribute is used; if 0, the activation attribute is not used. Sensitivity flag sensitivityFlag This field, which is present in the binary representation, signals the presence of the activation attribute. A value of "1" means the attribute shall be used and 0 means the attribute shall not be used This field signals the presence of an activation attribute and has a binary representation. If 1, the activation attribute is used; if 0, the activation attribute is not used. SNR flag
SNRFlag This field, which is present in the binary representation, signals the presence of the activation attribute. A value of "1" means the attribute shall be used and 0 means the attribute shall be used.
This field signals the presence of an activation attribute and has a binary representation. If 1, the activation attribute is used; if 0, the activation attribute is not used. unit
unit Describes the unit of the sensor's measuring value.
The unit of the measurement is a reference to a classification scheme as provided in A.2.1 of ISO / IEC 23005-6, if a unit other than the default unit specified in the semantics of the maxValue and minValue is used for the values of maxValue and minValue are used.
Describes the unit of measurement of the sensor and is specified with reference to the classification scheme term provided by UnitTypeCS. Units other than the base unit specified by the semantics of the maximum and minimum values may be used. Maximum value maxValue Describes the maximum value that the sensor can perceive. The terms will be different according to the individual sensor type.
Describe the maximum value that the sensor can recognize. This can vary depending on each sensor type
MinValue Describes the minimum value that the sensor can perceive. The terms will be different according to the individual sensor type.
Describes the minimum value the sensor can recognize. This can vary depending on each sensor type
Offset offset Describes the number of value locations added to a base value in order to get a specific absolute value.

To obtain a specific absolute value, we describe the value of the position added to the base value.
Level number numOfLevels Describes the number of values that the sensor can perceive in the maximum and minimum values.
EXAMPLE The value 5 means the sensor can perceive 5 steps from minValue to maxValue.

The numerical value of the level that the sensor can recognize between the maximum value and the minimum value. If the value is 5, the sensor can recognize from the minimum value to the maximum value in 5 steps. Sensitivity Describes the minimum magnitude of input signal required to produce a specified output signal in given unit.

Describes the minimum magnitude of the input signal required to produce a specific output signal in a given unit. SNR Describes the ratio of the signal power to the noise power.
Describe the signal power and the ratio of the noise power giving distortion to the signal

Sensor Capabilities Basic properties (SensorCapabilityBaseAttributes) provide the following example. An arbitrary sensor device having any_specific_sensor_device_capability_type has an ID of "ans01" and indicates a maximum value of 100, a minimum value of 10, a level of 20, an offset of -3, a sensitivity of 0.8, and an SNR of 99DB. The measurement unit of a specific sensor device is dB.

Table 29 shows the syntax.

"cidl: SensorDeviceCapability xsi: type =" scdv: any_specific_sensor_device_capability_type "id =" ans01 "maxValue =" 100 "minValue =" 10 "numOfLevels =" 20 "offset =" -3 "sensitivity =" 0.8 "SNR = = "urn: mpeg: mpeg-v: 01-CI-UnitTypeCS-NS: dB"/>

* Camera Sensor Capability Type

Hereinafter, a syntax and semantics for the camera sensor capability are proposed. The camera sensor capabilities support the capabilities of camera sensors, spectral camera sensors, color camera sensors, depth camera sensors, stereo camera sensors, and thermal camera sensors.

Table 30 shows the syntax.

<complexType name = "CameraSensorCapabilityType">
<complexContent>
<extension base = "cidl: SensorCapabilityBaseType">
<sequence>
<element name = "SupportedResolutions" type = "scdv: ResolutionListType" minOccurs = "0"/>
<element name = "FocalLengthRange" type = "scdv: ValueRangeType" minOccurs = "0"/>
<element name = "ApertureRange" type = "scdv: ValueRangeType" minOccurs = "0"/>
<element name = "ShutterSpeedRange" type = "scdv: ValueRangeType" minOccurs = "0"/>
<element name = "ISOSpeedRange" type = "scdv: ValueRangeType" minOccurs = "0"/>
<element name = "ExposureValueRange" type = "scdv: ValueRangeType" minOccurs = "0"/>
<element name = "ColorFilterArrayType" type = "scdv: ColorFilterArrayListType" minOccurs = "0"/>
<element name = "Video" type = "boolean" minOccurs = "0"/>
<element name = "SensorType" type = "boolean" minOccurs = "0"/>
<element name = "ColorSpaceType" type = "string" minOccurs = "0"/>
<element name = "BitDepthRange" type = "scdv: ValueRangeType" minOccurs = "0"/>
<element name = "SpectrumRange" type = "scdv: ValueRangeType" minOccurs = "0"/>
<element name = "ThermalRange" type = "scdv: ValueRangeType" minOccurs = "0"/>
<element name = "WhiteBalanceTempRange" type = "scdv: ValueRangeType" minOccurs = "0"/>
<element name = "WhiteBalanceTintRange" type = "scdv: ValueRangeType" minOccurs = "0"/>
</ sequence>
</ extension>
</ complexContent>
</ complexType>

<complexType name = "ResolutionListType">
<sequence>
<element name = "Resolution" type = "scdv: ResolutionType" maxOccurs = "unbounded"/>
</ sequence>
</ complexType>

<complexType name = "ResolutionType">
<sequence>
<element name = "Width" type = "nonNegativeInteger"/>
<element name = "Height" type = "nonNegativeInteger"/>
</ sequence>
</ complexType>

<complexType name = "ValueRangeType">
<sequence>
<element name = "MaxValue" type = "float"/>
<element name = "MinValue" type = "float"/>
</ sequence>
</ complexType>

<simpleType name = "ColorFilterArrayListType">
<restriction base = "string">
<enumeration value = "Bayer"/>
<enumeration value = "RGBE"/>
<enumeration value = "CYYM"/>
<enumeration value = "CYGM"/>
<enumeration value = "RGB Bayer"/>
<enumeration value = "RGBW # 1"/>
<enumeration value = "RGBW # 2"/>
<enumeration value = "RGBW # 3"/>
</ restriction>
</ simpleType>

Table 31 shows the syntax.

CameraSensorCapabilityType { Number of bits Mnemonic SupportedResolutionsFlag One bslbf FocalLengthRangeFlag One bslbf ApertureRangeFlag One bslbf ShutterSpeedRangeFlag One bslbf ISORangeFlag One bslbf ExposureValueRangeFlag One bslbf ColorFilterFlag One bslbf VideoFlag One bslbf SensorType One bslbf ColorSpaceFlag One bslbf BitDepthRangeFlag One bslbf SpectrumRangeFlag One bslbf ThermalRangeFlag One bslbf WhiteBalanceTempRangeFlag One bslbf WhiteBalanceTintRangeFlag One bslbf SensorCapabilityBase SensorCapabilityBaseType if (SupportedResolutionsFlag) { SupportedResolutions ResolutionListType } if (FocalLengthRangeFlag) { FocalLengthRange ValueRangeType } if (ApertureRangeFlag) { ApertureRange ValueRangeType } if (ShutterSpeedRangeFlag) { ShutterSpeedRange ValueRangeType } if (ISOSpeedRangeFlag) { ISOSpeedRange ValueRangeType } if (ExposureValueRangeFlag) { ExposureValueRange ValueRangeType } if (ColorFilterArrayFlag) { ColorFilterArrayType ColorFilterArrayListType } if (ColorSpaceFlag) { ColorSpaceTypeLength vluimsbf ColorSpaceType See ISO 10646 UTF-8 } if (BitDepthRangeFlag) { BitDepthRange ValueRangeType } if (SpectrumRangeFlag) { SpectrumRange ValueRangeType } if (ThermalRangeFlag) { ThermalRange ValueRangeType } if (WhiteBalanceTempRangeFlag) { WhiteBalanceTempRange ValueRangeType } if (WhiteBalanceTintRangeFlag) { WhiteBalanceTintRange ValueRangeType } } ResolutionListType { LoopResolution vluimsbf for (k = 0; k <LoopResolution; k ++) { Resolution [k] ResolutionType } } ResolutionType { Width 32 uimsbf Height 32 uimsbf } ValueRangeType { MaxValue 32 fsbf MinValue 32 fsbf }

Table 32 shows the semantics for CameraSensorCapabilityType.

Name Definition Camera sensor capability type
CameraSensorCapabilityType Tool for describing a camera sensor capability.
Tools for explaining camera sensor capabilities Supported resolution flags
SupportedResolutionsFlag This field, which is present in the binary representation, signals the presence of the SupportedResolutions element. This field signals the presence of a supported resolution element and has a binary representation. 1, this element exists, and if it is 0, this element does not exist Supported Resolutions SupportedResolutions Describes the camera support.
Describes a list of resolutions supported by the camera. Resolution list type ResolutionListType Describes a type of resolution list, which is composed of ResolutionType elements.
Describes the type of resolution list composed of resolution type elements Resolution Type ResolutionType Describes a type of resolution which is composed of a Width element and a Height element.
Describes the type of resolution consisting of a width element and a height element. Width Describes a width of resolution that the camera can perceive.
Describes the width of the resolution the camera can recognize Height Height Describes a height of resolution that the camera can perceive
Describes the height of resolution the camera can recognize Focal length range flag FocalLengthRangeFlag This field, which is present in the binary representation, signals the presence of the FocalLengthRange element. A value of "1" means that this element is present and 0 means that this element is nopresent.
This field signals the presence of a focal length range element and has a binary representation. If 1, this element exists, and if it is 0, this element does not exist Focal Length Range FocalLengthRange Describes the range of the focal length that the camera sensor can perceive in terms of ValueRangeType. Its default unit is millimeters (mm).
NOTE The minValue and the maxValue in the SensorCapabilityBaseType are not used for this sensor.

Describes the range of focal lengths that the camera sensor can recognize for the ValueRangeType. The basic unit of this is mm. 최대 Maximum and minimum values in the basic capability type (SensorCapabilityBaseType) are not used by the camera sensor Value Range Type ValueRangeType Defines the range of the value that the sensor can perceive.

Describes the range of values that the camera sensor can recognize Max Value MaxValue Describes the maximum value that the sensor can perceive.
Describes the maximum value that the camera sensor can recognize MinValue Describes the minimum value that the sensor can perceive.
Describes the minimum value that the camera sensor can perceive Aperture range flag ApertureRangeFlag This field, which is present in the binary representation, signals the presence of the ApertureRange element. A value of " 1 " means that this element is present and 0 means that the elent is not present.
This flag signals the presence of an ApertureRange element and has a binary representation. 1 if this element exists, 0 if this element does not exist Aperture Range Describes the range of the aperture that the camera can perceive in terms of valueRangeType.
NOTE The minValue and the maxValue in the SensorCapabilityBaseType are not used for this sensor.

Describes the range of apertures that the camera sensor can recognize for the value range type (valueRangeType).

Sensor Capability The minimum and maximum values in the default type (SensorCapabilityBaseType) are not used by the camera sensor
Shutter Speed Range Flag ShutterSpeedRangeFlag This field, which is present in the binary representation, signals the presence of the ShutterSpeedRange element. A value of "1" means that this element is present and 0 means that this element is nopresent.
This field signals the presence of a ShutterSpeedRange element and has a binary representation. If 1, this element exists, and if it is 0, this element does not exist Shutter Speed Range ShutterSpeedRange Describes the range of the shutter speed that the camera sensor can perceive in terms of valueRangeType. Its default unit is seconds (sec).
NOTE The minValue and the maxValue in the SensorCapabilityBaseType are not used for this sensor.
Describes the range of shutter speeds that the camera sensor can recognize for the value range type (valueRangeType). The basic unit of this is seconds.
Sensor Capability The minimum and maximum values in the default type (SensorCapabilityBaseType) are not used by the camera sensor
ISO speed range flag ISOSpeedRangeFlag This field, which is present in the binary representation, signals the presence of the ISOSpeedRange element. A value of "1" means that this element is present and 0 means that this element is nopresent.
This field signals the presence of an ISO speed range element (ISOSpeedRange element) and has a binary representation. If 1, this element exists, and if it is 0, this element does not exist ISO speed range ISOSpeedRange Describes the range of ISO Speed based on ISO 12232: 2006 that the camera can perceive in terms of valueRangeType.
NOTE The minValue and the maxValue in the SensorCapabilityBaseType are not used for this sensor.
This field describes the range of ISO speeds based on ISO. Sensor Capability The minimum and maximum values in the default type (SensorCapabilityBaseType) are not used by the camera sensor Exposure value range flag ExposureValueRangeFlag This field, which is present in the binary representation, signals the presence of the ExposureValueRange element. A value of "1" means that this element is present and 0 means that this element is nopresent.
This field signals the presence of an ExposureValueRange element and has a binary representation. If 1, this element exists, and if it is 0, this element does not exist Exposure Value Range ExposureValueRange Describes the range of exposure values that the camera sensor can perceive in terms of valueRangeType.
NOTE The minValue and the maxValue in the SensorCapabilityBaseType are not used for this sensor.

Describes the range of exposure values that the camera sensor can perceive for the value range type (valueRangeType).

Sensor Capability The minimum and maximum values in the default type (SensorCapabilityBaseType) are not used by the camera sensor
Video Flag VideoFlag A value of * j * means that this camera sensor can only shoot still image. A value of " 1 "
If 0, the camera sensor shoots the still image, and if 1, the camera sensor can record the video. Sensor type SensorType A value of * j * means that this camera can only perceive monochrome image. A value of " 1 " means that the camera can perceive the color image.
0, the camera sensor recognizes the monochrome image, and if 1, the camera sensor recognizes the color image Color Filter Array Flag ColorFilterArrayFlag This field, which is present in the binary representation, signals the presence of the ColorFilterArrayType element. This field signals the presence of a color filter array type element (ColorFilterArrayType element) and has a binary representation.

If 1, this element exists, and if it is 0, this element does not exist Color Filter Array Type ColorFilterArrayType Describes the color filter array to the image sensor of a camera
Describes a color filter array applied to the image sensor of a camera.

0000 Reserved
0001 Bayer
0010]
0011 CYYM
0100 CYGM
0101 RGBW Bayer
0110 RGBW # 1
0111 RGBW # 2
1000 RGBW # 3
1001-1111 Reserved Color space flag ColorSpaceFlag This field, which is present in the binary representation, signals the presence of the ColorSpaceType element. A value of "1" means that this element is present and 0 means that this element is nopresent.

This field signals the presence of a color space type element (ColorSpaceType element) and has a binary representation.

If 1, this element exists, and if it is 0, this element does not exist Color Space Type ColorSpaceType Describes the color space applied.

Describes applied color space
Bit depth range flag BitDepthRangeFlag This field, which is present in the binary representation, signals the presence of the BitDepthRange element. A value of "1" means that this element is present and 0 means that this element is nopresent.

This field signals the presence of a BitDepthRange element and has a binary representation.

If 1, this element exists, and if it is 0, this element does not exist Bit depth range BitDepthRange Describes the range of the bit depth that the camera can perceive in terms of valueRangeType.
NOTE The minValue and the maxValue in the SensorCapabilityBaseType are not used for this sensor.

Describes the range of bit depths that the camera sensor can perceive for the value range type (valueRangeType).

Sensor Capability The minimum and maximum values in the default type (SensorCapabilityBaseType) are not used by the camera sensor Spectrum range flag SpectrumRangeFlag This field, which is present in the binary representation, signals the presence of the SpectrumRange element. A value of "1" means that this element is present and 0 means that this element is not present
This field signals the presence of a Spectrum Range element and has a binary representation.

If 1, this element exists, and if it is 0, this element does not exist Spectrum range SpectrumRange Describes the spectrum range that the camera sensor can perceive in terms of valueRangeType. Its default unit is nanometer (nm).
NOTE The minValue and the maxValue in the SensorCapabilityBaseType are not used for this sensor.


Describes the spectral range that the camera sensor can perceive for the value range type (valueRangeType).

Sensor Capability The minimum and maximum values in the default type (SensorCapabilityBaseType) are not used by the camera sensor Column range flag ThermalRangeFlag This field, which is present in the binary representation, signals the presence of the ThermalRange element. A value of "1" means that this element is present and * j * 0means that this element is not present.

This field signals the presence of a ThermalRange element and has a binary representation.

If 1, this element exists, and if it is 0, this element does not exist Thermal Range ThermalRange Describes the thermal response range that the camera sensor can perceive in terms of valueRangeType. Its default unit is Celsius (℃).
NOTE The minValue and the maxValue in the SensorCapabilityBaseType are not used for this sensor.


Describes the thermal response that a camera sensor can perceive for a value range type (valueRangeType).

Sensor Capability The minimum and maximum values in the default type (SensorCapabilityBaseType) are not used by the camera sensor White Balance Temperature Range Flag WhiteBalanceTempRangeFlag This field, which is present in the binary representation, signals the presence of the WhiteBalanceTempRange element. A value of "1" means that this element is present and 0 means that this element is not present
This field signals the presence of a WhiteBalanceTempRange element and has a binary representation.

If 1, this element exists, and if it is 0, this element does not exist White Balance Temperature Range WhiteBalanceTempRange Describes the white balance temperature range that the camera sensor can perceive in terms of valueRangeType. Its default unit is Kelvin (K).
NOTE The minValue and the maxValue in the SensorCapabilityBaseType are not used for this sensor.

Describes the white balance temperature range that the camera sensor can recognize for the value range type (valueRangeType).

Sensor Capability The minimum and maximum values in the default type (SensorCapabilityBaseType) are not used by the camera sensor White balance tint flag WhiteBalanceTintFlag This field, which is present in the binary representation, signals the presence of the WhiteBalanceTintRange element. A value of "1" means that this element is present and 0 means that this element is not present
This field signals the presence of a white balance tint range element and has a binary representation.

If 1, this element exists, and if it is 0, this element does not exist White Balance Tint Range WhiteBalanceTintRange Describes the range of white balance tint values that the camera can perceive in terms of valueRangeType.
NOTE The minValue and the maxValue in the SensorCapabilityBaseType are not used for this sensor.


Describes the white balance tint value that the camera sensor can recognize for the value range type (valueRangeType).

Sensor Capability The minimum and maximum values in the default type (SensorCapabilityBaseType) are not used by the camera sensor

The camera sensing capability is described. The camera sensor may have an ID " CSCID_001 ". The camera sensor has a list of supported resolutions of 1280 x 720 (width x height) and 1920 x 1080. The maximum focal length of this camera sensor is 100mm, and the minimum focal length is 5mm. The maximum aperture of this camera sensor is F1.4, and the minimum aperture is F8. This camera sensor has a maximum shutter speed of 1 second and a minimum shutter speed of 0.001 seconds.

Table 33 shows the semantics.

<cidl: SensorDeviceCapability xsi: type = "scdv: CameraSensorCapabilityType" id = "CSCID_001">
<scdv: SupportedResolutions>
<scdv: Resolution>
<scdv: Width> 1280 </ scdv: Width>
<scdv: Height> 720 </ scdv: Height>
</ scdv: Resolution>
<scdv: Resolution>
<scdv: Width> 1920 </ scdv: Width>
<scdv: Height> 1080 </ scdv: Height>
</ scdv: Resolution>
</ scdv: SupportedResolutions>
<scdv: FocalLengthRange>
<scdv: MaxValue> 100 </ scdv: MaxValue>
<scdv: MinValue> 5 </ scdv: MinValue>
</ scdv: FocalLengthRange>
<scdv: ApertureRange>
<scdv: MaxValue> 1.4 </ scdv: MaxValue>
<scdv: MinValue> 8 </ scdv: MinValue>
</ scdv: ApertureRange>
<scdv: ShutterSpeedRange>
<scdv: MaxValue> 1 </ scdv: MaxValue>
<scdv: MinValue> 0.001 </ scdv: MinValue>
</ scdv: ShutterSpeedRange>
</ cidl: SensorDeviceCapability>

* Microphone sensor capability type

Hereinafter, the syntax and semantics of the capability specification of the microphone sensor are specified.

Table 34 shows the syntax for MicrophoneSensorCapabilityType.

<complexType name = "MicrophoneSensorCapabilityType">
<complexContent>
<extension base = "cidl: SensorCapabilityBaseType">
<sequence>
<element name = "micorphoneType" type = "scdv: mcrophoneListType" minOccurs = "0"/>
<element name = "transcuderArrayType" type = "scdv: transducerArrayListType" minOccurs = "0"/>
<element name = "probtType" type = "scdv: probeListType" minOccurs = "0"/>
<element name = "polarPatternType" type = "scdv: polarPatternListType" minOccurs = "0"/>
<element name = "frequencyRange" type = "scdv: frequencyRangeType" minOccurs = "0"/>
<element name = "responseType" type = "scdv: frequencyRangeType" minOccurs = "0"/>
<element name = "pickSensitivity" type = "float" minOccurs = "0"/>
</ sequence>
</ extension>
</ complexContent>
</ complexType>

<simpleType name = "microphoneListType">
<restriction base = "string">
<enumeration value = "condenser"/>
<enumeration value = "dynamic"/>
<enumeration value = "ribbon"/>
<enumeration value = "carbon"/>
<enumeration value = "piezoelectric"/>
<enumeration value = "fiber optic"/>
<enumeration value = "laser"/>
<enumeration value = "liquied"/>
<enumeration value = "MEMS"/>
</ restriction>
</ simpleType>

<simpleType name = "transducerArrayListType">
<restriction base = "string">
<enumeration value = "single array"/>
<enumeration value = "linear array"/>
<enumeration value = "curvilinear"/>
<enumeration value = "phased"/>
<enumeration value = "annular"/>
<enumeration value = "matrix array"/>
<enumeration value = "MEMS"/>
</ restriction>
</ simpleType>

<simpleType name = "probeListType">
<restriction base = "string">
<enumeration value = "linear"/>
<enumeration value = "sector"/>
<enumeration value = "convex"/>
<enumeration value = "carbon"/>
<enumeration value = "trapezoid"/>
</ restriction>
</ simpleType>

<simpleType name = "polarPatternListType">
<restriction base = "string">
<enumeration value = "omnidirectional"/>
<enumeration value = "bi-directional"/>
<enumeration value = "subcardioid"/>
<enumeration value = "cardioid"/>
<enumeration value = "hypercardioid"/>
<enumeration value = "supercardioid"/>
<enumeration value = "shotgun"/>
</ restriction>
</ simpleType>

<complexType name = "frequencyRangeType">
<sequence>
<element name = "minFrequency" type = "float"/>
<element name = "maxFrequency" type = "float"/>
</ sequence>
</ complexType> MicrophoneSensorCapabilityType { Number of bits Mnemonic microphoneTypeFlag One bslbf transducerArrayFlag One bslbf probeTypeFlag One bslbf polarPatternTypeFlag One bslbf frequencyRangeFlag One bslbf frequencyResponseTypeFlag One bslbf sensitivityFlag One bslbf SensorCapabilityBase SensorCapabilityBaseType if (microphoneTypeFlag == 1) { microphoneType microphoneListType } if (transducerArrayFlag == 1) { transducerArrayType trnasducerArrayListType } if (probeTypeFlag == 1) { probeType 4 probeListType } if (polarPatternTypeFlag == 1) { polarPattern 4 polarPatternListType } if (frequencyRangeFlag == 1) { frequencyRange frequencyRangeType } if (responseTypeFlag == 1) { responseFrequency frequencyRangeType } if (sensitivityFlag == 1) { pickSensitivity 32 fsbf } microphoneListType { microphoneType 4 bslbf } transducerArrayListType { transducerArrayType 4 blsbf } probeListType { probeType 4 blsbf } polarPatternListyType { polarPattern 4 blsbf } frequencyRangeType { minFrequency 32 uimsbf maxFrequency 32 uimsbf }

Table 35 shows the semantics for MicrophoneSensorCapabilityType.

Name Definition Microphone type
microphoneType Defines type of microphone
Defines the type of microphone
0000 Reserved
0001 Condenser
0010 Dynamic
0011 Ribbon
0100 Carbon
0101 Piezoelectric
0110 Fiber Optic
0111 Laser
1000 Liquid
1001 MEMS
1010-1111 Reserved Transducer array type transducerArrayType Defines array types of transducer probes
Defines the array type of the transducer probes.
0000 Reserved
0001 single array
0010 linear array
0011 curvilinear
0100 phased
0101 annular
0110 matrix array
0111-1111 Reserved Probe type probeType Defines probing type of transducer
Define the probing type of the transducer
0000 Reserved
0001 linear probe
0010 sector probe
0011]
0100 trapezoid probe
0101-1111 Reserved Polar pattern polarPattern Defines polar pattern of transducer
Define the polar pattern of the transducer
0000 Reserved
0001 Omnidirectional
Bi-directional (or Figure 8)
0011 Subcardioid
0100 Cardioid
0101 Hypercardioid
0110 Supercardioid
0111 Shotgun
1000-1111 Reserved Frequency range
frequencyRange Pickup frequency range in Hz
Pickup frequency range in Hz Response type flag

responseTypeFlag '0' if Flat frequency response
'1' if Tailored frequency response

0, it is a flat frequency response. When it is 1, it is a tailored frequency response. Response frequency
responseFrequency Pick response frequency range for tailored frequency response
Pickup response frequency range for tailored frequency response microphones Minimum frequency
minFreqeuncy Minimum frequency in Hz
Minimum frequency in Hz Max frequency maxFrequency Maximum frequency in Hz
Maximum frequency in Hz Pick-up sensitivity pickSensitivity Pick sensitivity of transducer in mV / Pa

Pickup sensitivity of the transducer in mV / Pa

The following example shows a specification of microphone capability with semantics. The ID of the microphone may be " MCID_001 ". In this case, the microphone is 20Hz-20kHz with the cardioid pattern of the frequency pickup range tailored between 20Hz-8kHz.

Table 36 shows the syntax.

<cidl: SensorDeviceCapability xsi: type = "scdv: microphoneCapabilityType" id = "MCID_001">
<microphoneType>"condenser"</microphoneType>
<polarPatternType>"cardioid"</polarPatternType>
<scdv: frequencyRange>
<scdv: minFrequency> 20 </ scdv: minFrequency>
<scdv: maxFrequency> 20000 </ scdv: maxFrequency>
</ scdv: frequencyRange>
<scdv: responseType>
<scdv: minFrequency> 20 </ scdv: minFrequency>
<scdv: maxFrequency> 8000 </ scdv: maxFrequency>
</ scdv: responseType>
</ cidl: SensorDeviceCapability>

* Sensor information description tool

The sensor information specification tool describes information acquired through each sensor. Sensor information can be generated at the output of the sensors. The sensor information basic type (SensedInfoBaseType) derives the sensor information type for each sensor.

* Global coordinate for sensors.

The reference coordinates for the sensors are defined by adopting the right coordinate system. The Y axis is the direction of gravity. The Z axis is perpendicular to the Y axis and represents the user's front direction. The X axis is perpendicular to the Y and Z axes and represents the right direction of the user. In the reference coordinates of the sensors, the default start is the user's position. The start in the coordinate system depends on the type of sensor.

Table 37 shows the syntax for the Sensed information base type.

<! - ##################################################### ## ->
<! - Sensed information base type ->
<! - ##################################################### ## ->
<complexType name = "SensedInfoBaseType" abstract = "true">
<sequence>
<element name = "TimeStamp" type = "mpegvct: TimeStampType" minOccurs = "0"/>
</ sequence>
<attributeGroup ref = "iidl: sensedInfoBaseAttributes"/>
</ complexType>
SensedInfoBaseTypeType { Number of bits Mnemonic TimeStampFlag One bslbf SensedInfoBaseAttributes SensedInfoBaseAttributesType If (TimeStampFlag) { TimeStamp TimeStampType } }

Table 38 shows the semantics of SensedInfoListType.

Name Definition Sensor information basic type
SensedInfoBaseType Provides the topmost type of base type hierarchy.
Provides the topmost type of base type hierarchy from which each sensor information is derived. Sensor Information Basic Properties sensedInfoBaseAttributes Describes a group of attributes for the sensed information.
Describe a group of attributes for sensor information Time stamp TimeStamp Provides the time information at which the sensed information is acquired. As defined in ISO / IEC 23005-6, there is a choice of three timing schemes, which are absolute time, clocktick time, and delta of clock tick time.
Provides time information when sensor information is acquired. One of the three time schemes is selected, and the three time schemes include the absolute time, the clock tick time, and the delta of the clock tick time. Timestamp flag
TimeStampFlag This field, which is present in the binary representation, signals the presence of the TimeStamp element. A value of "1" means the element shall be used and 0 means the element shall not be used. "

This field signals the presence of a timestamp element and has a binary representation. If 1, the timestamp element is used; if 0, the timestamp element is not used.

* Sensed information base attributes

Table 39 shows the syntax of Sensed information base attributes.

<! - ##################################################### ##### ->
<! - Definition of Sensed Information Base Attributes ->
<! - ##################################################### ##### ->
<attributeGroup name = "sensedInfoBaseAttributes">
<attribute name = "id" type = "ID" use = "optional"/>
<attribute name = "sensorIdRef" type = "anyURI" use = "optional"/>
<attribute name = "linkedlist" type = "anyURI" use = "optional"/>
<attribute name = "groupID" type = "anyURI" use = "optional"/>
<attribute name = "activate" type = "boolean" use = "optional"/>
<attribute name = "priority" type = "nonNegativeInteger" use = "optional" default = "0"/>
</ attributeGroup> SensedInfoBaseAttributesType {  Number of bits Mnemonic idFlag One bslbf sensorIdRefFlag One bslbf LinkedListFlag One bslbf groupIDFlag One bslbf priorityFlag One bslbf activateFlag One bslbf If (idFlag) { id See ISO 10646 UTF-8 } if (sensorIdRefFlag) { sensorIdRef See ISO 10646 UTF-8 } if (linkedlistFlag) { LinkedList See ISO 10646 UTF-8 } if (groupIDFlag) { groupID See ISO 10646 UTF-8 } If (priorityFlag) {     priority 32 uimsbf } if (activateFlag) { activate One bslbf } }

Table 40 shows the syntax of Sensed information base attributes.

Name Definition Sensor information basic attributes
sensedInfoBase Attributes Describes a group of attributes for the commands.

Describe the group of attributes for the command
ID
id Unique identifier for identifying individual sensed information
A unique identifier for identifying individual sensor information. Sensor ID reference sensorIdRef References A sensor that has generated the information included in this specific sensed information.
A reference to a sensor that generates information contained in a specific sensor information. Linked list Linked list Describes the multi-sensor structure that consists of a group of sensors in a way that includes a reference to the ID of the next sensor.

Describe a multiple sensor structure that includes a group of sensors, by way of each record containing a reference to the ID of the next sensor
Group IDgroupID Identifier for a group multi-sensor structure to which this specific sensor belongs.

Describe identifiers for groups of multiple sensor structures to which a particular sensor belongs.
Activate activate Describes whether the sensor is activated. A value of "true" means the sensor shall be activated and "false" means the sensor shall be deactivated.
In the binary representation, A value of "1" means the sensor shall be activated and 0 means the senr shall be deactivated

Describes whether the sensor is enabled. If true, the sensor is enabled; if false, the sensor is disabled.

If it is a binary representation, 1 means the sensor is active, 0 means the sensor is inactive. Priority Priority Describes a sensed information that is sensed in relation to other sensed information. A value of one indicates the highest priority and a lower value indicates lower priorities. The default value of the priority is one. If there are more than one sensed information with the same priority, the order of process can be determined by the adaptation engine itself.

When sensor information is adopted, the priority of sensor information for other sensor information sharing the same point at a specific time is described

A value of 1 indicates the highest priority, and a value greater than 1 indicates a lower priority. The default value for priority is 1. If there is more than one sensor information having the same priority, processing order of the sensor information is determined by the adaptation engine.


NOTE The priority may be used to apply the sensed information to the virtual world.

Priority is used to apply sensor information to virtual world object features defined within the group of sensors according to the capabilities of the adaptation VR.

EXAMPLE The adaptation RV processes depend on the individual sensed information of a group of sensors due to their limited order due to its limited capabilities. That is, the sensed information with the lower priority might get lost

The adaptation RV processes the individual sensor information of a group of sensors according to their priorities sorted in descending order due to limited capability. Sensor information with low priority is lost. Sensor information basic attribute type
SensedInfoBaseAttributesType Tool for describing sensed information base attributes.

A tool that describes the basic type of sensor information ID Flags IDFlag This field, which is present in the binary representation, signals the presence of the ID attribute. A value of "1" means the attribute shall be used and 0 means thattribute shall not be used.
This field signals the presence of an ID attribute and has a binary representation. If 1, the ID attribute is used; if 0, the ID attribute is not used. Sensor ID reference flag sensorIdRefFlag This field, which is present in the binary representation, signals the presence of the sensor ID reference attribute. A value of "1" means the attribute shall be used and 0 means thattribute shall not be used.
This field signals the presence of a sensor ID reference attribute and has a binary representation. 1, the sensor ID reference attribute is used, and if 0, the sensor ID reference attribute is not used Linked list flag linkedlistFlag This field, which is present in the binary representation, signals the presence of the linked list attribute. A value of "1" means the attribute shall be used and 0 means thattribute shall not be used.
This field signals the existence of a linked list attribute and has a binary representation. 1, the linked list attribute is used, if 0, the linked list attribute is not used Group ID flag groupIDFlag This field, which is present in the binary representation, signals the presence of the group ID attribute. A value of "1" means the attribute shall be used and 0 means thattribute shall not be used.

This field signals the presence of a group ID attribute and has a binary representation. If 1, the group ID attribute is used; if 0, the group ID attribute is not used. Priority flags priorityFlag This field, which is present in the binary representation, signals the presence of the priority attribute. A value of "1" means the attribute shall be used and 0 means thattribute shall not be used.
This field signals the presence of a priority attribute attribute and has a binary representation. If 1, the priority attribute is used; if 0, the priority attribute is not used Active flag activateFlag This field, which is present in the binary representation, signals the presence of the activation attribute. A value of "1" means the attribute shall be used and 0 means thattribute shall not be used.

This field signals the presence of an activation attribute and has a binary representation. If 1, the activation attribute is used; if 0, the activation attribute is not used.

* Camera sensor type

Hereinafter, a basic sensor type for sensing with a camera will be described. Various types of cameras, such as infrared cameras or spectrum cameras, are specified using a camera sensor type.

Table 41 shows the syntax of the Camera Sensor Type.

<! - ##################################################### ## ->
<! - Camera Sensor Type ->
<! - ##################################################### ## ->
<complexType name = "CameraSensorType">
<complexContent>
<extension base = "iidl: SensedInfoBaseType">
<sequence>
<element name = "CameraOrientation" type = "siv: OrientationSensorType" minOccurs = "0"/>
<element name = "CameraLocation" type = "siv: GlobalPositionSensorType" minOccurs = "0"/>
<element name = "CameraAltitude" type = "siv: AltitudeSensorType" minOccurs = "0"/>
</ sequence>
<attribute name = "focalLength" type = "float" use = "optional"/>
<attribute name = "aperture" type = "float" use = "optional"/>
<attribute name = "shutterSpeed" type = "float" use = "optional"/>
<attribute name = "filter" type = "mpeg7: termReferenceType" use = "optional"/>
</ extension>
</ complexContent>
</ complexType> CameraSensorType { Number of bits Mnemonic CameraOrientationFlag One bslbf CameraLocationFlag One bslbf CameraAltitudeFlag One bslbf focalLengthFlag One bslbf apertureFlag One bslbf shutterSpeedFlag One bslbf filterFlag One bslbf SensedInfoBaseType See above SensedInfoBaseType if (CameraOrientationFlag) { CameraOrientation See above OrientationSensorType } if (CameraLocationFlag) { CameraLocation See above GlobalPositionSensorType } if (CameraAltitudeFlag) { CameraAltitude See above AltitudeSensorType } if (focalLengthFlag) { focalLength 32 fsbf } if (apertureFlag) { Aperture 32 fsbf } if (shutterSpeedFlag) { shutterSpeed 32 fsbf } if (filterFlag) { Filter 4 bslbf } }

Table 42 shows the semantics of CameraSensorType.

Name Definition Camera sensor type
CameraSensorType Tool for describing sensed information with respect to a camera sensor.
A tool that describes sensor information for camera sensors. Camera location
CameraLocation Describes the location of a camera using the structure defined by GlobalPositionSensorType.

Describe the location of the camera using the structure defined by the global position sensor type (GlobalPositionSensorType). Camera Altitude CameraAltitude Describes the altitude of a camera using the structure defined by AltitudeSensorType.
Describes camera altitude using the structure defined by AltitudeSensorType. Camera Orientation CameraOrientation Describes the orientation of a camera using the structure defined by OrientationSensorType.
Describes the orientation of the camera, using the structure defined by the OrientationSensorType. FocalLength Describes the distance between the lens and the image sensor when the subject is in focus, in terms of millimeters (mm).
Describes the length (in mm) between the lens and the image sensor when the object is in focus. Aperture aperture Describes the diameter of the lens opening. It is expressed as F-stop, eg F2.8. It may also be expressed as f-number notation such as f / 2.8.
Describes the diameter when the lens is open. The aperture is expressed as F-stop as in F2.8, or f-number notation as f / 2.8 Shutter speed shutterSpeed Describes the time the shutter remains open when taking a photograph in terms of seconds.
Describes the time (in seconds) that the shutter stays open when you take a picture Filter Describes kinds of camera filters as a reference to a classification scheme that uses the mpeg7: termReferenceType defined in 7.6 of ISO / IEC 15938-5: 2003. The CS may be used for this purpose, the CameraFilterTypeCS defined in A.2.12.6.

Describe the types of camera filters by referring to the classification scheme term using Mpeg7. The CS used for this purpose is represented by the CameraFilterTypeCS. Camera orientation flag CameraOrientationFlag This field, which is present in the binary representation, signals if the camera orientation is sensed information is available. A value of "1" indicates that the sensed information will be included and 0 indicates that the sensed information will not be included.
This field is signaled when camera direction sensor information is available and has a binary representation. 1, the camera direction sensor information is included, and when it is 0, the camera direction sensor information is not included Camera position flag CameraLocationFlag This field, which is present in the binary representation, A value of "1" indicates that the sensed information will be included and "j" indicates that the sensed information will not be included.

This field is signaled when camera position sensor information is available and has a binary representation.

1, the camera position sensor information is included, and when it is 0, the camera position sensor information is not included Camera Altitude Flag CameraAltitudeFlag This field, which is present in the binary representation, A value of "1" indicates that the sensed information will be included and 0 indicates that the sensed information will not be included.

This field is signaled when camera altitude sensor information is available and has a binary representation.

1, the camera altitude sensor information is included, and when it is 0, the camera altitude sensor information is not included The focal length flag focalLengthFlag This field, which is present in the binary representation, signals the presence of the focal length attribute. A value of "1" means the attribute shall be used and * j "means the attribute shall not be used.

This field signals the presence of a focal length attribute and has a binary representation. If 1, the focal length attribute is used; if 0, the focal length attribute is not used. Aperture flag apertureFlag This field, which is present in the binary representation, signals the presence of the aperture attribute. A value of "1" means the attribute shall be used and 0 means the attribute shall not be bused.

This field signals the presence of an aperture attribute and has a binary representation.

1, the iris property is used; if it is 0, the iris property is not used Shutter speed flag shutterSpeedFlag This field, which is present in the binary representation, signals the presence of the shutter speed attribute. A value of "1" means the attribute shall be used and 0 means the attribute shall not be bused.

This field signals the presence of a shutter speed attribute and has a binary representation.

1, the shutter speed attribute is used; if it is 0, the shutter speed attribute is not used Filter flag filterFlag This field, which is present in the binary representation, signals the presence of the filter attribute. A value of "1" means the attribute shall be used and 0 means the attribute shall not be bused.

This field signals the presence of a filter attribute and has a binary representation.

If 1, the filter attribute is used; if 0, the filter attribute is not used.

The following semantics represent the specification of camera sensing. This specification has an ID of " CSID001 ". The camera sensor then senses a time stamp of 60000 with 100 clock ticks per second. The focal length of the camera sensor is 50 mm, the aperture is F2.8, the shutter speed is 1/250 sec, and a UV filter is used. The location information of the camera sensor is latitude 37.23N and longitude 131.23E. The direction information of this camera sensor has Ox = 2.0 (radian), Oy = 2.0 (radian), and Oz = 1.0 (radian).

Table 43 shows the syntax.

<iidl: InteractionInfo>
<iidl: SensedInfoList>
<iidl: SensedInfo xsi: type = "siv: CameraSensorType" id = "CSID001" activate = "true" focalLength = "50" aperture = "2.8" shutterSpeed = "0.004" filter = "urn: mpeg: mpeg-v: 01 -SI-CameraFilterTypeCS-NS: UV ">
<iidl: TimeStamp xsi: type = "mpegvct: ClockTickTimeType" timeScale = "100" pts = "60000"/>
<siv: CameraOrientation xsi: type = "siv: OrientationSensorType" unit = "radian">
<siv: Orientation>
<mpegvct: X> 2.0 </ mpegvct: X>
<mpegvct: Y> -0.5 </ mpegvct: Y>
<mpegvct: Z> 1.0 </ mpegvct: Z>
</ siv: Orientation>
</ siv: CameraOrientation>
<siv: CameraLocation xsi: type = "siv: GlobalPositionSensorType" longitude = "131.23" latitude = "37.23"/>
</ iidl: SensedInfo>
</ iidl: SensedInfoList>
</ iidl: InteractionInfo>

* Microphone sensor type

The microphone sensor type specifies a device capable of sensing audio information. The sensing attribute of the microphone sensor is specified by the microphone sensor capability type. The application of the microphone sensor type may be included in the navigation system or where there is voice recognition such as home automation (intelligent system), AR application.

Table 44 shows the syntax of the microphone sensor type.

<! - ################################## ->
<! - Definition of microphone sensor type ->
<! - ################################## ->

<complexType name = "MicrophoneSensorType">
<complexContent>
<extension base = "iidl: SensedInfoBaseType">
<sequence>
<element name = "Orientation" type = "siv: OrientationSensorType" minOccurs = "0"/>
<element name = "Altitude" type = "siv: AltitudeSensorType" minOccurs = "0"/>
<element name = "Location" type = "siv: GlobalPositionSensorType" minOccurs = "0"/>
<element name = "AudioData" type = "siv: RawAudioType"/>
</ sequence>
</ extension>
</ complexContent>
</ complexType>

<complexType name = "RawAudioType">
<choice>
< element name = " AudioData 16 " type = " hexBinary &
<element name = "AudioData64" type = "base64Binary"/>
</ choice>
<attribute name = "sample_rate" type = "unsignedint"/>
<attribute name = "byte_order" type = "ByteOrderType"/>
<attribute name = "sign" type = "SignType"/>
<attribute name = "resolution" type = "ResolutionType"/>
</ complexType>

<simpleType name = "ByteOrderType">
<restriction base = "string">
<enumeration value = "LittleEndian"/>
<enumeration value = "BigEndian"/>
</ restriction>
</ simpleType>

<simpleType name = "SignType">
<restriction base = "string">
<enumeration value = "Signed"/>
<enumeration value = "Unsigned"/>
</ restriction>
</ simpleType>

<simpleType name = "ResolutionType">
<restriction base = "mpeg7: unsignedByte">
<enumeration value = "4"/>
<enumeration value = "8"/>
<enumeration value = "12"/>
<enumeration value = "16"/>
<enumeration value = "20"/>
<enumeration value = "24"/>
<enumeration value = "32"/>
<enumeration value = "48"/>
<enumeration value = "64"/>
</ restriction>
</ simpleType> MicrophoneSensorType { Number of bits Mnemonic OrientationFlag One bslbf LocationFlag One bslbf        sampleRateFlag One bslbf        resolutionFlag One bslbf SensedInfoBase SensedInfoBaseType if (OrientationFlag) { Orientation OrientationSensorType         }         If (LocationFlag) { Location GlobalPositionSensorType         } AudioData { If (sampleRateFlag) {                      sample_rate_size vluimsbf5 sample_rate sample_rate_size uimsbf    } byte_order One bslbf sign One bslbf               If (resolutionFlag) { resolution 4 bslbf               }               RawAudioDataSize vluimsbf5               RawAudioData RawAudioDataSize * 8 bslbf } }

Table 45 shows the syntax of the microphone sensor type.

Name Definition Microphone sensor type
MicrophoneSensorType Tool for describing sensed information with respect to a microphone sensor.

A tool that describes sensor information for a microphone sensor. Direction flag OrientationFlag This field, which is present in the binary representation, signals the presence of the Orientation element. A value of "1" means that the orientation element exists in the binary representation and 0 means that the element does not exist in the binary reprentation.
This field signals the presence of an orientation element and has a binary representation. 1, the direction element is present in the binary representation, and if zero, it is not present in the binary representation Altitude flag AltitudeFlag This field, which is present in the binary representation, signals the presence of the Altitude element. A value of "1" means that the Altitude element exists in the binary representation and 0 means that the binary element does not exist in the binary reprentation.

This field signals the presence of an altitude element and has a binary representation.

1, then the altitude element is present in the binary representation, and if it is zero, it is not present in the binary representation Location Flag LocationFlag This field, which is present in the binary representation, signals the presence of the Location element. A value of "1" means that the location element exists in the binary representation and 0 means that the location element does not exist in the binary reprentation.

This field signals the presence of a Location element and has a binary representation.

1, the location element is present in the binary representation, and if zero, it is not present in the binary representation. Sample Rate Flag sampleRateFlag This field, which is present in the binary representation, signals the presence of sampleRate attribute in AudioData. A value of "1" means the attribute will be used and 0 means the attribute sll not be used.

This field signals the presence of a sample rate attribute (sampleRate attribute) in the audio data and has a binary representation.

1, the sample rate attribute is used, if 0, not used Resolution flags resolutionFlag This field, which is present in the binary representation, signals the presence of resolution attribute in AudioData. A value of "1" means the attribute shall be used and 0 means the attribute shall not be bused.

This field signals the presence of a resolution attribute in the audio data and has a binary representation.

If 1, the resolution attribute is used; if 0, it is not used Orientation Describes the orientation of the microphone using the structure defined by OrientationSensorType.

Describe the orientation of the microphone using the structure defined by the OrientationSensorType. Altitude Altitude Describes the altitude of the microphone using the structure defined by AltitudeSensorType.

Explain the altitude of the microphone using the structure defined by the altitude sensor type (AltitudeSensorType). Location Describes the location of the microphone using the structure defined by GlobalPositionSensorType.

Describe the location of the microphone using the structure defined by the global position sensor type (GlobalPositionSensorType). Audio data 16AudioData16 Holds binary audio data encoded as a textual string in base-16 format.

Hold binary audio data encoded in a text string in base 16 format Audio data 64 AudioData64 Holds binary audio data encoded as a textual string in base-64 format.

Hold binary audio data encoded in a text string in base 64 format Sample Rate Size sample_rate_size This field is the binary representation of the binary representation of the binary value.
This field indicates the size of the encoded binary representation of the sample rate attribute value in the bits and has a binary representation
Sample rate sample_rate Sample rate is the number of samples carried out per second, measured in Hz.

The number of samples of audio transmitted per second, measured in Hz. Byte order byte_orderIt tells how the data is stored with the most significant byte on one end or the other. When more than one byte is used to represent a PCM sample, the byte order (big endian vs. little endian) must be known. Due to the widespread use of little-endian Intel CPUs, little-endian PCM tends to be the most common byte orientation.

Explain how data is stored at the end of one or the MSB at the other. If more than one byte is used to represent a PCM sample, the byte order (big endian vs. little endian) must be known. Because of the widespread use of little-endian Intel CPUs, little-endian PCMs tend to be the most common byte orientations.
The following table should be used for the binary representation, and this field should be specified in the binary representation.
Binary representation
(1 bit) ByteOrderType0LittleEndian1BigEndian
The following table is used for binary representations, and this field is for specifying binary representations. If the sample is unsigned, the sample range is 0 ... 255. If the sample is unsigned or unsigned, the sample is unsigned. with a center point of 128. If the sample is signed, the sample range is -128 ... 127 with a center point of 0. If a PCM type is signed, the sign encoding is almost always 2's complement. , signed PCM audio is represented as a series of sign / magnitude coded numbers.

For example, it is not enough to know PCM samples with 8 bit width. It is necessary to understand the range of whether the sample is expressed as a code or not as a code. If the sample is not represented by a sign, the sample represents a range of 0 to 255 with an intermediate point of 128. If the sample is represented by a sign, the sample represents a range of -128 to 127 with an intermediate point of zero. In a very exceptional case, the PCM audio represented by a sign is represented by a sequence of signed / size encoded digits.

This field should be present in binary representation and the following table shall be used for binary representation.
Binary representation
(1 bits) SignType0Signed1Unsigned Resolution This parameter specifies the amount of data used to represent each discrete amplitude sample. The most common values are 8 bits (1 byte), which gives a range of 256 amplitude steps, or 16 bits (2 bytes), which gives a range of 65536 amplitude steps. Other sizes, such as 12, 20, and 24 bits, are occasionally seen. Some king-sized formats even for opt for 32 and 64 bits per sample.

This parameter specifies the amount of data used to represent each discrete amplitude sample. When the maximum common value is 8 bits (1 byte), a range of 256 amplitude steps is provided, and if the maximum common value is 16 bits (2 bytes), a range of 65536 amplitude steps is provided. In some cases, the king size format even selects 32 bits or 64 bits per sample Signed Signed Specifies the raw audio data coming from the microphone sensor as stored as signed numbers.

Specified by the number represented by the raw audio data code derived from the microphone sensor Not represented by sign Unsigned Specifies the raw audio data coming from the microphone sensor as stored as unsigned numbers.
Specifies whether the raw audio data derived from the microphone sensor is stored as numbers that are not represented by signs Big Indian BigEndian It identifies that the audio data is stored in the Big Endian format: the most significant byte of a smallest address and the least significant byte is stored in the largest address.

This specifies whether the audio data is stored in the Big Indian format. The most significant byte (MSB) of the word is stored at the smallest address, and the LSB (least significant byte) of the word is stored at the largest address. Little Indian LittleEndian It identifies the audio data is stored in the Little Endian format: the least significant byte in the smallest address.

This specifies whether the audio data is stored in Little Indian format. The LSB of the word is stored in the smallest address Raw Audio Data Size RawAudioDataSize Describes the size of the RawAudioData in bytes. This field is only present in binary representation.

The size of the raw audio data in bytes is described. This field only has a binary representation. Raw Audio Data RawAudioData Actual data holder for binary raw audio data, only in binary representation. The size of this field is given in the RawAudioDataSize field.

The actual data holder for binary low audio data has a binary representation. The size of this field is given in the raw audio data size field (RawAudioDataSize field).

This example shows the specification of the microphone sensor according to the following semantics. The microphone sensor is in the (-10, 0, 25) position and in the (0.3, 0.6 0 0.2) direction. The audio data format has a sampling rate of 8000 Hz, the byte order is Little Indians, the value has a sign and is represented by 16 bits.

Table 46 shows the syntax of the microphone sensor type.

<cidl: SensorDeviceCapability
xsi: type = "scdv: MircophoneSensorType"
id = "micsens01">
<Orientation> 0.3 0.6 0 0.2 </ Orientation>
<Location> -10 0 25 </ Location>
<AudioData>
<sample_rate> 8000 </ sample_rate>
<byte_order> LittleEndian </ byte_order>
<sign> Signed </ sign>
<resolution> 16 </ resolution>
</ AudioData>
</ cidl: SensorDeviceCapability>

Meanwhile, the method according to the present invention may be embodied as a program that can be executed by a computer, and may be embodied as various recording media such as a magnetic storage medium, an optical reading medium, and a digital storage medium.

Implementations of the various techniques described herein may be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or combinations thereof. Implementations may be implemented in a computer program product, such as an information carrier, e.g., a machine readable storage device, such as a computer readable storage medium, for example, for processing by a data processing apparatus, Apparatus (computer readable medium) or as a computer program tangibly embodied in a propagation signal. A computer program, such as the computer program (s) described above, may be written in any form of programming language, including compiled or interpreted languages, and may be stored as a stand-alone program or in a module, component, subroutine, As other units suitable for use in the present invention. A computer program may be deployed to be processed on one computer or multiple computers at one site or distributed across multiple sites and interconnected by a communications network.

Processors suitable for processing a computer program include, by way of example, both general purpose and special purpose microphones, as well as any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The elements of a computer may include at least one processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer may include one or more mass storage devices for storing data, such as magnetic, magneto-optical disks, or optical disks, or may receive data from them, transmit data to them, . &Lt; / RTI &gt; Information carriers suitable for embodying computer program instructions and data include, for example, semiconductor memory devices, for example, magnetic media such as hard disks, floppy disks and magnetic tape, compact disk read only memory A magneto-optical medium such as a floppy disk, an optical disk such as a DVD (Digital Video Disk), a ROM (Read Only Memory), a RAM , Random Access Memory), a flash memory, an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), and the like. The processor and memory may be supplemented or included by special purpose logic circuitry.

In addition, the computer-readable medium can be any available media that can be accessed by a computer, and can include both computer storage media and transmission media.

While the specification contains a number of specific implementation details, it should be understood that they are not to be construed as limitations on the scope of any invention or claim, but rather on the description of features that may be specific to a particular embodiment of a particular invention Should be understood. Certain features described herein in the context of separate embodiments may be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments, either individually or in any suitable subcombination. Further, although the features may operate in a particular combination and may be initially described as so claimed, one or more features from the claimed combination may in some cases be excluded from the combination, Or a variant of a subcombination.

Likewise, although the operations are depicted in the drawings in a particular order, it should be understood that such operations must be performed in that particular order or sequential order shown to achieve the desired result, or that all illustrated operations should be performed. In certain cases, multitasking and parallel processing may be advantageous. Also, the separation of the various device components of the above-described embodiments should not be understood as requiring such separation in all embodiments, and the described program components and devices will generally be integrated together into a single software product or packaged into multiple software products It should be understood.

It should be noted that the embodiments of the present invention disclosed in the present specification and drawings are only illustrative of specific examples for the purpose of understanding and are not intended to limit the scope of the present invention. It will be apparent to those skilled in the art that other modifications based on the technical idea of the present invention are possible in addition to the embodiments disclosed herein.

Claims (19)

Obtaining first sensor information from a real-world sensor;
Converting the acquired first sensor information into virtual world object characteristics applied to a virtual world or second sensor information applied to a virtual world;
Applying the transformed virtual world object feature or second sensor information to a virtual world
Lt; / RTI &gt;
The real world sensors correspond to the sensor capability specification,
Wherein the global coordinates are set in dependence on the environment of the real world. The method according to claim 1,
The real world sensor includes a camera sensor,
Wherein the camera sensor is a camera sensor capability type. The method according to claim 1,
The camera sensor capability type comprises:
A resolution range, a resolution list, a resolution list type, a resolution type, a width, a height, a focal length range flag, a focal length range, a value range type, an aperture range flag, an aperture range, a shutter speed range flag, ISO Speed Range Flag, ISO Speed Range, Exposure Value Range Flag, Exposure Value Range, Video Flag, Sensor Type, Color Filter Array Flag, Color Filter Array Type, Color Space Flag, Color Space Type, Bit Depth Range Flag, Bit Depth Range, Spectrum A method for processing sensor information between a virtual world and a real world including at least one of a range flag, a spectrum range, a column range flag, a column range, a white balance temperature range flag, a white balance temperature range, a white balance tint flag, and a white balance tint range. The method according to claim 1,
The real world sensor includes a microphone sensor,
Wherein the microphone sensor is a microphone sensor capability type. 5. The method of claim 4,
The microphone sensor capability type comprises:
A method for processing sensor information between a virtual world and a real world including at least one of a microphone type, a transducer array type, a probe type, a polar pattern, a frequency range, a response type flag, a response frequency, and a pickup sensitivity. The method according to claim 1,
The camera sensor is specified according to a camera sensor type (CameraSensorType)
The camera sensor type includes at least one of a camera orientation, a camera position, a camera altitude, a focal length, an aperture, a shutter speed, a filter, a camera direction flag, Wherein the sensor information includes at least one of a camera position flag, a camera height flag, a focal length flag, an aperture flag, a shutter speed flag, and a filter flag. The method according to claim 1,
The microphone sensor is specified according to a microphone sensor type (MicrophoneSensorType)
The microphone sensor type includes a directional flag, an altitude flag, a position flag, a sample rate flag, a resolution flag, a direction, an altitude, a position, a sample rate size, a sample rate, a byte order, A data size, and a raw audio data size. A recording medium readable by an electronic device in which sensor information of a sensor in a real world for applying sensor information to a virtual object in a virtual world is recorded,
In the real world sensor,
The real world sensors correspond to the sensor capability specification,
And global coordinates that depend on the environment of the real world are set. 9. The method of claim 8,
The real world sensor includes a camera sensor,
Wherein the camera sensor capability type is set. 10. The method of claim 9,
The camera sensor capability type comprises:
A resolution range, a resolution list, a resolution list type, a resolution type, a width, a height, a focal length range flag, a focal length range, a value range type, an aperture range flag, an aperture range, a shutter speed range flag, ISO Speed Range Flag, ISO Speed Range, Exposure Value Range Flag, Exposure Value Range, Video Flag, Sensor Type, Color Filter Array Flag, Color Filter Array Type, Color Space Flag, Color Space Type, Bit Depth Range Flag, Bit Depth Range, Spectrum A white balance temperature range flag, a white balance tint flag, and a white balance tint range, wherein the range of the white balance temperature range includes a range flag, a spectrum range, a column range flag, a column range, a white balance temperature range flag, 9. The method of claim 8,
The real world sensor includes a microphone sensor,
Wherein the microphone sensor type is set to a microphone sensor capability type. 12. The method of claim 11,
The microphone sensor capability type comprises:
Wherein the recording medium includes at least one of a microphone type, a transducer array type, a probe type, a polar pattern, a frequency range, a response type flag, a response frequency, and a pickup sensitivity. 10. The method of claim 9,
The camera sensor is specified according to a camera sensor type (CameraSensorType)
The camera sensor type includes at least one of a camera orientation, a camera position, a camera altitude, a focal length, an aperture, a shutter speed, a filter, a camera direction flag, A camera position flag, a camera height flag, a focal length flag, an aperture flag, a shutter speed flag, and a filter flag. 12. The method of claim 11,
The microphone sensor is specified according to a microphone sensor type (MicrophoneSensorType)
The microphone sensor type includes at least one of direction flags, altitude flags, position flags, sample rate flags, resolution flags, direction, altitude, position, sample rate size, sample rate, byte order, sign, resolution, A data size, and a low audio data size. The sensor information processing system
Media processor,
The media processor comprising:
Acquiring first sensor information from a real-world sensor,
Converts the acquired first sensor information into virtual world object characteristics applied to the virtual world or second sensor information applied to the virtual world,
Applying the transformed virtual world object feature or second sensor information to a virtual world,
The real world sensors correspond to the sensor capability specification,
Wherein global coordinates dependent on the environment of the real world are set. 16. The method of claim 15,
The real world sensor includes a camera sensor,
Wherein the camera sensor capability type is set to the camera sensor capability type. 16. The method of claim 15,
The real world sensor includes a microphone sensor,
Wherein the microphone sensor is set to a microphone sensor capability type. 17. The method of claim 16,
Wherein the camera sensor is specified according to a camera sensor type (CameraSensorType). 18. The method of claim 17,
Wherein the microphone sensor is specified according to a microphone sensor type (Microphone Sensor Type).

Images