KR20150098162A - Method and apparatus for controlling operation associated with multimedia data - Google Patents

Abstract

An electronic device according to the present invention reproduces a moving image in a state in which it is coupled to a head wearable accessory device, monitors a motion state of the electronic device using a sensor value measured by an inertial sensor, If it is determined that the stationary state of the electronic device is maintained for a time or longer, the reproduction of the moving picture can be stopped.

Description

TECHNICAL FIELD [0001] The present invention relates to an apparatus and method for controlling multimedia data,

The present disclosure relates to an operation control method and apparatus associated with multimedia data of an electronic device.

2. Description of the Related Art Recently, various services and functions provided by portable devices have been gradually expanded. A variety of applications that are executable on portable devices are also being developed. In order to increase the utility value of the portable device and meet various needs of users, a service for linking with an external accessory device is also being developed.

For example, a portable device such as a smart phone can provide a head mounted display (HMD) function in conjunction with a hair-wearing accessory device. In other words, after the portable apparatus is combined with the head wearable accessory apparatus, when the image contents are reproduced on the portable apparatus, the reproduced image data is displayed on the display of the portable apparatus, and the reproduced audio data is transmitted to the head- Lt; / RTI > Accordingly, the user can enjoy the image content on a large screen and provide the effect.

When the portable device provides the HMD (Head Mounted Display) function in conjunction with the head wearable accessory device, in order to control the motion picture reproduction or to control the operation of the portable device, There is an inconvenience in that the device must be operated directly.

Various embodiments of the present invention provide a method and apparatus by which a user can conveniently control the operation of a portable device.

And various embodiments of the present invention provide a method and apparatus by which a user can conveniently control operations such as reproduction of multimedia data such as moving picture contents.

According to an embodiment of the present invention, there is provided a method for controlling motion picture reproduction of an electronic device, comprising the steps of: reproducing a moving picture in a state of being coupled to a head wearable accessory device; Monitoring a motion state of the electronic device; and stopping the reproduction of the moving image if it is determined that the stop state of the electronic device is maintained over a reference time based on the sensor value.

According to another embodiment, an electronic device includes a display, at least one inertial sensor, and a head wearable accessory device coupled to the accessory device for playing back a moving image, And a playback module for stopping the playback of the moving image if it is determined that the stop state of the electronic device is maintained over a reference time based on the sensor value.

The various embodiments of the present invention allow a user to conveniently control the reproduction of multimedia data such as moving picture contents or to control the reproduction of multimedia data such as moving picture contents when the portable apparatus provides a head mounted display (HMD) And to control the operation.

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a perspective view of an electronic device according to an embodiment of the present invention, Fig.
2 is a view of a headwear accessory device according to an embodiment of the present invention,
FIG. 3 is a view showing a state of attachment of a head wearable accessory device and an electronic device according to an embodiment of the present invention,
FIG. 4 is a view showing a configuration of a head wearable accessory device according to an embodiment of the present invention,
5 illustrates a network environment including an electronic device according to an embodiment of the present invention;
6 is a diagram illustrating a configuration of a motion detection module according to an embodiment of the present invention;
7 and 8 are diagrams illustrating an operation process of an electronic device according to various embodiments of the present invention.
9 is a diagram illustrating a configuration of an electronic device to which a moving picture display control method according to an embodiment of the present invention is applied.

The present disclosure will be described below with reference to the accompanying drawings. The present disclosure is capable of various modifications and various embodiments, and specific embodiments are illustrated in the drawings and detailed description of the invention is set forth. It is to be understood, however, that this disclosure is not intended to be limited to the specific embodiments, but includes all changes and / or equivalents and alternatives falling within the spirit and scope of the disclosure. In connection with the description of the drawings, like reference numerals have been used for like elements.

The use of the terms "comprises" or "comprising" may be used in the present disclosure to indicate the presence of a corresponding function, operation, or element, etc., and does not limit the presence of one or more other features, operations, or components. Also, in this disclosure, the terms "comprises" or "having ", and the like, specify that the presence of stated features, integers, But do not preclude the presence or addition of other features, numbers, steps, operations, components, parts, or combinations thereof.

The "or" in the present disclosure includes any and all combinations of words listed together. For example, "A or B" may comprise A, comprise B, or both A and B.

The expressions "first," " second, "" first," or "second, " and the like in the present disclosure can modify various elements of the disclosure, For example, the representations do not limit the order and / or importance of the components. The representations may be used to distinguish one component from another. For example, both the first user equipment and the second user equipment are user equipment and represent different user equipment. For example, without departing from the scope of the present disclosure, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used in this disclosure is used only to describe a specific embodiment and is not intended to limit the disclosure. The singular expressions include plural expressions unless the context clearly dictates otherwise.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the contextual meaning of the relevant art and are to be interpreted as either ideal or overly formal in the sense of the art unless explicitly defined in this disclosure Do not.

Hereinafter, an electronic device according to various embodiments will be described with reference to the accompanying drawings. The term user as used in various embodiments may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

The electronic device according to the present disclosure may be a head mounted display (HMD) device. Alternatively, the electronic device according to the present disclosure may be an electronic device capable of providing an image reproducing function, such as an HMD device, in cooperation with a hair-wearing accessory device. In other words, after the electronic device is combined with the head wearable accessory device, when the multimedia data corresponding to the moving picture content is reproduced in the electronic device, the reproduced video data is displayed on the display of the electronic device, Can be output through the speaker of the type accessory. Accordingly, when a user wears a hair-wearing accessory device combined with an electronic device on his / her head, he / she can feel the effect of viewing the video content on a large screen or in a stereoscopic image.

For example, the electronic device may be a smartphone, a tablet personal computer, a mobile phone, a videophone, an e-book reader, a laptop personal computer, A head-mounted display device such as a netbook computer, a personal digital assistant (PDA), a portable multimedia player (PMP), an MP3 player, a mobile medical device, a camera, or a wearable device (HMD), electronic garments, electronic bracelets, electronic necklaces, electronic apps, electronic tattoos, or smartwat ch).

An example of an electronic device 100 according to an embodiment of the present disclosure is shown in Fig. 1, a longitudinal direction of the electronic device 100 is referred to as a y-axis direction, a direction perpendicular to the y-axis is referred to as an x-axis direction, and a direction perpendicular to the y- The vertical direction is called the z-axis direction.

An example of a headwear accessory device 200 according to an embodiment of the present disclosure is shown in FIG. 2 is a view showing a state in which the wearer wears the hair accessory device 200 on the head. FIG. 3 shows an example of a state in which the hair-wearing accessory device 200 and the electronic device 100 are combined. Referring to FIG. 3, the electronic device 100 is mounted on the inside of the hair-wearing accessory device 200, and can serve as a display when reproducing image data. In other words, when the user wears the hair-wearing accessory device 200 on which the electronic device 100 is mounted, the electronic device 100 can be positioned in such a structure that the electronic device 100 can be positioned in a portion where the user's field of view is secured. And the head wearable accessory device 200 can be combined.

FIG. 4 is a view showing a configuration of a head wearable accessory device 200 according to an embodiment of the present invention. 4, the head wearable accessory device 200 includes a control unit 210, a sensor module 220, an audio output unit 230, a communication interface 240, and a memory 250 .

The sensor module 220 may include an inertial sensor (e.g., an acceleration sensor, a gyro sensor), a proximity sensor, an image sensor, and the like. The sensor value sensed by the sensor module 220 may be transmitted to the controller 210.

The audio module 230 can convert a sound and an electric signal in both directions by controlling the controller 210. The audio module 230 can be input to the audio module 230 through, for example, a speaker, a receiver, an earphone, Or output sound information.

The communication interface 240 can connect the communication between the head wearable accessory device 200 and an external device (e.g., the electronic device 100). For example, the communication interface 240 can be connected to a network via wireless communication or wired communication to communicate with an external device. The wireless communication may include at least one of, for example, wireless fidelity (Wifi), Bluetooth (BT), or near field communication (NFC) or global positioning system (GPS). The wired communication may include at least one of, for example, a universal serial bus (USB), a high definition multimedia interface (HDMI), a recommended standard 232 (RS-232) or a plain old telephone service (POTS).

The memory 250 may store instructions or instructions received from the control 210 or other components such as the communication interface 240 or the sensor module 220 or generated by the control 210 or other components, Data can be stored. Memory 250 may include, for example, a programming module such as a kernel, a middleware, an application programming interface (API) or an application. Each of the above-described programming modules may be composed of software, firmware, hardware, or a combination of at least two of them.

The control unit 210 processes the audio data received from the electronic device 100 connected through the communication interface 240 so that the sound can be output through the audio module 230. The sensor value sensed by the sensor module 220 may be transmitted to the electronic device 100 through the communication interface 240. [

FIG. 5 illustrates a network environment including an electronic device 100, in accordance with various embodiments. 5, an electronic device 100 includes a bus 110, a processor 120, a memory 130, an input / output interface 140, a display 150, a communication interface 160, a playback module 170, A sensor module 180, and a camera module 190.

The bus 110 may be a circuit that interconnects the components described above and communicates (e.g., control messages) between the components described above.

Processor 120 may be coupled to other components (e.g., memory 130, input / output interface 140, display 150, communication interface 160, playback module 170, or the camera module 190), decrypts the received command, and executes an operation or data processing according to the decrypted command.

The memory 130 may include a processor 120 or other components such as an input and output interface 140, a display 150, a communication interface 160, a playback module 170, a sensor module 180, (E.g., processor 190, etc.) or may store instructions or data generated by processor 120 or other components. The memory 130 may include programming modules such as, for example, a kernel 131, a middleware 132, an application programming interface (API) 133, or an application 134. Each of the above-described programming modules may be composed of software, firmware, hardware, or a combination of at least two of them.

The kernel 131 may be used to store system resources (e. G., Buses (e. G., Buses)) used to execute operations or functions implemented in the rest of the programming modules, e. G., Middleware 132, API 133, 110, processor 120, memory 130, etc.). The kernel 131 may also provide an interface through which the individual components of the electronic device 100 may be accessed and controlled or managed in the middleware 132, the API 133, or the application 134.

The middleware 132 can act as an intermediary for the API 133 or the application 134 to communicate with the kernel 131 to exchange data. The middleware 132 may also be configured to communicate system resources (e.g., bus 110) of the electronic device 100 to at least one application of the application 134, for example in connection with work requests received from the application 134. [ (E.g., scheduling or load balancing) of a work request using a method such as assigning a priority that can be used by a processor (e.g., processor 120, memory 130, etc.).

The API 133 is an interface for the application 134 to control the functions provided by the kernel 131 or the middleware 132 and includes at least one interface for controlling the functions of the application 134 such as file control, Interface or function (e.g., command).

According to various embodiments, application 134 may be a multimedia content playback application, an SMS / MMS application, an email application, a calendar application, an alarm application, a health care application (e.g., Environmental information applications (e.g., applications that provide air pressure, humidity, or temperature information, etc.), and the like. Additionally or alternatively, application 134 may be an application related to the exchange of information between electronic device 100 and an external electronic device (e.g., electronic device 104). An application related to information exchange may include, for example, a notification relay application for communicating specific information to an external electronic device, or a device management application for managing an external electronic device.

For example, the notification delivery application may send notification information from another application (e.g., SMS / MMS application, email application, health care application or environmental information application) of the electronic device 100 to an external electronic device 104). ≪ / RTI > Additionally or alternatively, the notification delivery application may receive notification information from, for example, an external electronic device (e.g., electronic device 104) and provide it to the user. The device management application may be capable of performing functions (e.g., of the external electronic device itself (or some component)) on at least a portion of an external electronic device (e.g., electronic device 104) (Eg, install, delete, or update) a service (eg, call service or message service) provided by an application running on an external electronic device or external electronic device )can do.

According to various embodiments, the application 134 may include an application specified in accordance with the attributes (e.g., the type of electronic device) of the external electronic device (e.g., the electronic device 104). For example, if the external electronic device is an MP3 player, the application 134 may include an application related to music playback. Similarly, if the external electronic device is a mobile medical device, the application 134 may include applications related to health care. According to one embodiment, application 134 may include at least one of an application specified in electronic device 100 or an application received from an external electronic device (e.g., server 106 or electronic device 104).

And the application 134 may include a multimedia content playback application associated with the headwear accessory device 200. [ For example, if the multimedia contents playback application requests multimedia contents playback in a state where the head wearable accessory device 200 and the electronic device 100 are connected, the multimedia contents playback application may access the head wearable accessory device 200 ) And the electronic device 100 so that appropriate types of data can be output. For example, if there is a video content playback request, the playback application may cause the display 150 to display video data corresponding to the video content. At this time, the image data is displayed on two divided screens as shown in FIG. 3, and the image data displayed on each screen may be the same image or a similar image. As another example, if there is a request to reproduce the moving image content, the multimedia content playback application may display the image data corresponding to the moving image content on two divided screens of the display 150 as shown in FIG. Then, the audio signal corresponding to the reproduction of the audio data corresponding to the moving image content is transmitted to the head-worn accessory device 200 so that the sound is output through the audio output module 230 of the head- .

The memory 130 may store multimedia data corresponding to at least one multimedia content according to an embodiment. The multimedia content may include, for example, video content, video content, audio content, text content, and the like. The memory 130 may store reference information for determining the motion state of the electronic device 100 according to an embodiment. The motion state may include, for example, a stop state, a motion state, a non-insertion state, and a wear state. The non-wearing state may be a state in which the head wearable accessory device 200 on which the electronic device 100 is mounted is separated from the human body and placed on the fixed object. The wearing state may be a state in which the hair-wearing accessory device 200 on which the electronic device 100 is mounted is worn on the head of a person. The reference information may be information that becomes a reference for determining the non-insertion state and the wearing state.

The non-sticking reference information serving as a reference of the non-sticking state may include reference time information and a stop state value. The stop state is a state in which the motion of the electronic device 100 is not detected, and the stop state value may be a reference value that determines the stop state. The stop state value may be the sensor value of the inertia sensor. For example, the stop state information may be a constant change amount of the sensor values in the x-, y-, and z-axis directions of the acceleration sensor or the gyro sensor. In other words, if the amount of change of the sensor value in each of the x-, y-, and z-axis directions is a stop state value, the motion state of the electronic device 100 can be determined to be a stop state. The reference time information is a time interval during which the electronic device 100 is held stationary, and the electronic device 100 may be determined to be in an unlocked state when the electronic device 100 remains stationary for more than the reference time.

The wearer reference information, which is a standard for the wearing state, may include a wearing pose sensor value and a motion state value. The motion state is a state in which the motion of the electronic device 100 is detected, and the motion state value is information that is a reference for determining the motion state. The motion state value may be a sensor value of the inertial sensor. For example, the motion state value may be a certain variation amount of sensor values in the x, y, and z axis directions of the acceleration sensor or the gyro sensor. In other words, if the amount of change of the sensor value in each of the x-, y-, and z-axis directions is larger than the motion state value, the motion state of the electronic device 100 can be determined as a motion state. According to an embodiment, the stop state value and the motion state value may be the same. The wear pose sensor value may be a sensor value of the inertial sensor as information indicating a pose of the electronic device 100 when the electronic device 100 and the head wearing accessory device 200 are combined and worn on a person's head. For example, when the electronic device 100 and the hair-wearing accessory device 200 are combined and worn on the head of a person, the wear pose information is stored in the x-, y-, and z-axis directions of the acceleration sensor or the gyro sensor Sensor value.

The memory 130 may store concentration information according to an embodiment.

The input / output interface 140 is connected to the processor 120, the memory 130, and the memory 130 via a bus 110, for example, by a command or data input from a user via an input / output device (e.g., To the communication interface 160, or to the playback module 170. For example, the input / output interface 140 may provide data to the processor 120 about the user's touch input through the touch screen. The input / output interface 140 is connected to the input / output interface 140 via a bus 110. The input / output interface 140 receives commands or data received from the processor 120, the memory 130, the communication interface 160, (For example, a speaker or a display). For example, the input / output interface 140 may output audio data processed through the processor 120 to a user through a speaker.

Display 150 may display various information (e.g., multimedia data or text data) and images to the user.

The communication interface 160 may connect communications between the electronic device 100 and an external device such as the electronic device 104 or the server 106 or the headwear accessory device 200. [ For example, the communication interface 160 may be connected to the network 162 via wireless or wired communication to communicate with external devices. The wireless communication may be, for example, wireless fidelity (WFI), Bluetooth (BT), near field communication (NFC), global positioning system (GPS), or cellular communication (e.g., LTE, LTE-A, CDMA, WCDMA, UMTS, WiBro or GSM, etc.). The wired communication may include at least one of, for example, a universal serial bus (USB), a high definition multimedia interface (HDMI), a recommended standard 232 (RS-232) or a plain old telephone service (POTS).

According to one embodiment, the network 162 may be a telecommunications network. The communication network may include at least one of a computer network, an internet, an internet of things, or a telephone network. (E.g., a transport layer protocol, a data link layer protocol, or a physical layer protocol) for communication between the electronic device 100 and an external device may include an application 134, an application programming interface 133, The kernel 131, or the communication interface 160. In this case,

The sensor module 180 may include at least one inertial sensor. For example, the sensor module 180 may include an acceleration sensor and a gyro sensor. The sensor values sensed by the sensor module 180 may be communicated to the regeneration module 170 or the processor 120 via the bus 110.

An accelerometer measures the acceleration of an object. The inherent acceleration is the change in speed that a person or object receives when an object falls freely. From another point of view, at every point in the air, an object has inertia according to the equivalent principle, and the accelerometer measures the moment of inertia of the moment, which is often referred to as gravitational acceleration. The accelerometer shows about 1 g in the upward direction with respect to the Earth's surface at rest, because everywhere on Earth's surface accelerates upward in the inertial coordinate system.

The acceleration sensor in the electronic device 100 has sensors in the directions of x-axis, y-axis, and z-axis, and the tilt angle of the object can be measured as gravity based on the gravitational acceleration. For example, in a pose in which the front surface of the electronic device 100 lies parallel to the xy plane, the z-axis sensor value is 1G. In a pose in which the front surface of the electronic device 100 is erected perpendicular to the xy plane, the z-axis sensor value becomes 0G. In this way, the inclination of the electronic device 100 due to the pose can be measured using all three axis sensor values.

The camera module 190 is a device capable of capturing a still image and a moving image. According to an embodiment, the camera module 190 may include at least one image sensor (e.g., a front sensor or a rear sensor), a lens (not shown), an image signal processor Or a flash (not shown), such as a LED or xenon lamp. The video data generated through the camera module 190 may be transmitted to the playback module 170 via the bus 110. [

The playback module 170 may be connected to other components (e.g., the processor 120, the memory 130, the input / output interface 140, the communication interface 160, the sensor module 180, the camera module 190, etc.) Process at least some of the acquired information, and provide it to the user in a variety of ways. For example, the playback module 170 may detect a connection to the headwear accessory device 200 using the processor 120 or independently thereof, and if there is a request to play back the multimedia content, The multimedia data can be reproduced and displayed on the display 150. If the requested content is video content, the multimedia data corresponding to the video content is reproduced. Accordingly, the image data is displayed on two divided screens as shown in FIG. 3, and the images displayed on the respective screens are the same video Or similar video. The playback module 170 determines the motion state of the electronic device 100 using the sensor value sensed by the sensor module 180 and controls the operation of the electronic device 100 related to the multimedia data according to the motion state have.

For example, the playback module 170 determines whether the electronic device 100 is moving using the sensor value sensed by the sensor module 180. If the electronic device 100 remains in the stop state for a predetermined time or longer, The reproduction of the data can be stopped. Also, the playback module 170 determines that the electronic device 100 is worn using the sensor value sensed by the sensor module 180 after the movement of the electronic device 100 is detected while the playback of the multi-diary data is stopped Once determined, the multimedia data can be reproduced again.

6 is a diagram illustrating the configuration of a playback module 170 of an electronic device (e.g., electronic device 100), in accordance with various embodiments. Referring to FIG. 6, the playback module 170 may include a playback control unit 171 and a motion determination unit 172.

The reproduction control unit 171 may control the operation of the electronic device 100 related to the multimedia data when the multimedia contents are requested to be reproduced while the connection with the headwear accessory device 200 is detected. For example, if there is a request to reproduce the moving image content, the moving image multimedia data may be reproduced and displayed on the display 150. The reproduction control unit 171 may control related operations related to the moving picture multimedia data according to the motion state of the electronic device 100 determined by the motion determining unit 172. [

The motion determiner 172 may determine the motion state of the electronic device 100 using the sensor values collected by the sensor module 180. [ For example, the motion determiner 172 may determine that the electronic device 100 is in a stopped state when the amount of change of the sensor value of the acceleration sensor or the gyro sensor collected in the sensor module 180 is smaller than the stop state value . When the stop state of the electronic device 100 is maintained at the first reference time interval or more, the motion determiner 172 may determine that the motion state of the electronic device 100 is in the non-stick state.

The motion determiner 172 may determine that the electronic device 100 is in the moving state if the variation of the sensor value of the acceleration sensor or the gyro sensor collected by the sensor module 180 is greater than the motion sensor value.

The motion determiner 172 may determine that the electronic device 100 is worn if the sensor value of the acceleration sensor or the gyro sensor collected by the sensor module 180 matches the wear pose sensor value.

The motion determiner 172 may transmit the motion state of the electronic device 100 to the playback controller 171. [

The reproduction control unit 171 may control the operation of the electronic device 100 related to the multimedia data according to the motion state transmitted by the motion determination unit 172. [ For example, if the motion state of the electronic device 100 is in a non-use state, the playback control unit 171 may stop playback of the moving image multimedia data. If it is determined that the motion state is the wear state in the state where the reproduction is stopped, the reproduction control section 172 can restart the reproduction of the moving image multimedia data. At this time, the reproduction control section 172 may determine the restarting point of the reproduction as the point of time when the reproduction is stopped, or may determine the point of time before the reproduction of the point of time.

According to another embodiment, when the multimedia data is reproduced while the playback control unit 171 is connected to the headwear accessory device 200, the camera module 190 may be driven to allow the user's eyes to be photographed. Then, the reproduction control unit 171 can detect the eyes of the user from the image including the user's eyes, and confirm the user's eyes open and the eyes closed. The reproduction control unit 171 counts the time during which the user closes his or her eyes and counts that the eyes are closed over a predetermined period of time. When the count is counted, it is determined that the user's concentration has decreased and the corresponding point in time is stored in the memory 130 . Thereafter, the playback control unit 171 may start playback of the multimedia data from the point of time when the playback is restarted.

The operation of the electronic device 100 will be described with reference to FIG. 7 is a flowchart illustrating an operation of the electronic device 100 according to an embodiment of the present invention. The embodiment of FIG. 7 is an embodiment showing a case of reproducing moving picture contents. Although FIG. 7 shows a case related to the reproducing operation of the moving picture contents, it can be similarly applied to the reproducing of the picture contents or the audio contents.

The connection with the head wearable accessory device 200 is detected and if there is a request to reproduce the moving image content, the playback module 170 may reproduce the moving image multimedia data in step 301 and display the same on the display 150. [ The playback module 170 may monitor the motion state of the electronic device 100 using the sensor value of the sensor module 180.

The electronic device 100 will continue to move because the user can not maintain a complete stop state while wearing the headwear accessory device 200 on his head. Accordingly, when the head wearable accessory device 200 is worn on the user's head, the motion of the electronic device 100 can be sensed, and the motion state can be determined to be the motion state.

In step 303, the playback module 170 determines whether the motion state of the electronic device 100 is a motion state based on the sensor value of the sensor module 180. If the motion state is not the motion state, that is, The process proceeds to step 305. In step 305, The playback module 170 may check in step 305 whether the electronic device 100 remains in the stopped state for more than the reference time interval.

If the motion state of the electronic device 100 is maintained in the stopped state longer than the reference time interval, then the hair-wearing accessory device 200 may be separated from the user's body and placed on a stationary object (e.g., a table) high.

Accordingly, if the motion state of the electronic device 100 is maintained in the stopped state longer than the reference time interval, the playback module 170 may pause the playback of the multimedia data in step 307. [

Thereafter, in step 309, the playback module 170 may count a predetermined stop time. If the motion of the electronic device 100 is not detected during a predetermined time interval (stop time) after stopping the playback of the moving image multimedia data, the playback module 170 may turn off the display of the electronic device 100 in step 311 . Thereafter, in step 313, the playback module 170 detects the movement of the electronic device 100, and may turn on the display device in step 315.

Based on the sensor value of the sensor module 180, even if the movement of the electronic device 100 is detected before the stop time expires or the movement of the electronic device 100 is detected in step 313 even after the stop time expires The playback module 170 can check whether the electronic device 100 is worn in step 319. [ In other words, it is confirmed whether or not the user wore the hair-wearing accessory device 200 again on the head.

If the motion state of the electronic device 100 is determined to be worn based on the sensor value of the sensor module 180, the playback module 170 may proceed to step 301 to restart the playback of the moving image multimedia data.

In this case, the point at which the reproduction of the moving image multimedia data is restarted may be the image frame in which the reproduction of the moving image multimedia data is paused in step 307. According to another embodiment, the multimedia multimedia data playback may be restarted in the video frames preceding the plurality of video frames, rather than the video frame in which the playback of the moving image multimedia data is suspended.

Next, referring to FIG. 8, the operation process of the electronic device 100 according to another embodiment of the present disclosure will be described. The embodiment of FIG. 8 is an embodiment showing a case of reproducing moving picture contents. Although FIG. 8 shows a case related to the reproducing operation of the moving picture contents, it can be similarly applied to the reproducing of the picture contents or the audio contents.

The connection with the head wearable accessory device 200 is detected and if there is a request to reproduce the moving image content, the playback module 170 can reproduce the moving image multimedia data in step 401 and display the same on the display 150. [ In step 403, the reproduction module 170 may collect the concentration information. In other words, the playback module 170 drives the camera module 190 to capture a user's eyes, detect an eye in an image containing the user's eyes, You can check the status. The playback module 170 counts the time that the user closes his or her eyes and determines that the concentration of the user is less than or equal to the reference value when counted as being close to the eyes over a predetermined time. The playback module 170 may store the video frame corresponding to the playback time of the moving image multimedia data corresponding to the time point at which the user starts to close the eyes, that is, the time point at which the user starts to close the eyes, .

The playback module 170 may monitor the motion state of the electronic device 100 using the sensor value of the sensor module 180. When the head wearable accessory device 200 is worn on the user's head, the motion state of the electronic device 100 may be determined to be in a motion state. However, if the hair-wearing accessory device 200 is separated from the user's body and placed in a specific place, the motion state of the electronic device 100 will be determined to be in a stopped state.

In step 405, the playback module 170 determines whether the motion state of the electronic device 100 is the motion state based on the sensor value of the sensor module 180. If the motion state is not the motion state, that is, The process proceeds to step 407. In step 407, The playback module 170 may check in step 407 whether the electronic device 100 remains stopped for more than the reference time interval.

If the motion state of the electronic device 100 is maintained in the stopped state longer than the reference time interval, the playback module 170 may pause the playback of the moving image multimedia data in step 409.

Thereafter, in step 411, the playback module 170 may count a predetermined stop time. If the motion of the electronic device 100 is not detected during a predetermined time interval (stop time) after stopping the playback of the moving image multimedia data, the playback module 170 may turn off the display of the electronic device 100 in step 413 . Thereafter, in step 415, the playback module 170 may turn on the display device in step 417 if motion of the electronic device 100 is detected.

The movement state of the electronic device 100 is checked in step 419 before the stop time expires or the movement state of the electronic device 100 is checked in step 417 even after the stop time expires based on the sensor value of the sensor module 180 The playback module 170 can check whether the electronic device 100 is worn in step 421. [ In other words, it is confirmed whether or not the user wore the hair-wearing accessory device 200 again on the head.

If the motion state of the electronic device 100 is determined to be worn on the basis of the sensor value of the sensor module 180, the playback module 170 proceeds to step 423 so as to start playback of the moving image multimedia data It is possible to start playing back multimedia multimedia data. Thereafter, in step 403, the above operations may be repeated. According to another embodiment, the concentration information may include sensing information about the focus of the eye. That is, the focus of the user's eye can be sensed through the sensor, and the point of time when the focus of the eye disappears can be determined as the point of time at which the concentration level falls below the reference value. In this case, the restarting point of the moving image multimedia data playback will be the point where the focus of the eyes disappears. The focus of the eye may be sensed by a focus detection sensor, which may be included in the sensor module 180 of the electronic device 100. Or a focus detection sensor may be included in the sensor module 230 of the hair-wearing accessory device 200. When the focus sensor is included in the sensor module 230, the head wearable accessory device 200 may transmit the sensor value of the focus sensor to the electronic device 100.

9 is a diagram illustrating a configuration of an electronic device 800 according to various embodiments. The electronic device 800 may constitute all or part of the electronic device 100 shown in Fig. 5, for example. 9, the electronic device 800 includes at least one application processor (AP) 810, a communication module 820, a subscriber identification module (SIM) card 824, a memory 830, Module 182 includes a module 180, an input device 850, a display 860, an interface 870, an audio module 880, a camera module 190, a power management module 895, a battery 896, an indicator 897, A motor 898, and a regeneration module 170.

The AP 810 may control a plurality of hardware or software components connected to the AP 810 by driving an operating system or an application program, and may perform various data processing and operations including multimedia data. The AP 810 may be implemented as a system on chip (SoC), for example. According to one embodiment, the AP 810 may further include a graphics processing unit (GPU) (not shown).

The communication module 820 (e.g., the communication interface 160) may communicate with other electronic devices (e.g., electronic device 104 or server 106) that are networked with the electronic device 800 (e.g., the electronic device 100) Data transmission / reception can be performed in the communication between the base station and the base station. According to one embodiment, the communication module 820 includes a cellular module 821, a Wifi module 823, a BT module 825, a GPS module 827, an NFC module 828, and a radio frequency (RF) module 829).

The cellular module 821 may provide voice calls, video calls, text services, or Internet services over a communication network (e.g., LTE, LTE-A, CDMA, WCDMA, UMTS, WiBro or GSM). The cellular module 821 may also perform identification and authentication of electronic devices within the communication network using, for example, a subscriber identity module (e.g., SIM card 824). According to one embodiment, the cellular module 821 may perform at least some of the functions that the AP 810 may provide. For example, the cellular module 821 may perform at least some of the multimedia control functions.

According to one embodiment, the cellular module 821 may include a communication processor (CP). In addition, the cellular module 821 may be implemented with an SoC, for example. 8, components such as the cellular module 821 (e.g., communication processor), the memory 830, or the power management module 895 are shown as separate components from the AP 810, According to an embodiment, the AP 810 may be implemented to include at least a portion of the aforementioned components (e.g., cellular module 821).

According to one embodiment, the AP 810 or the cellular module 821 (e.g., a communications processor) may load or read commands or data received from at least one of non-volatile memory or other components connected thereto, load. In addition, the AP 810 or the cellular module 821 may receive data from at least one of the other components or may store data generated by at least one of the other components in the non-volatile memory.

Each of the Wifi module 823, the BT module 825, the GPS module 827, and the NFC module 828 includes a processor for processing data transmitted and received through a corresponding module, for example . Although the cellular module 821, the Wifi module 823, the BT module 825, the GPS module 827, or the NFC module 828 are shown as separate blocks in FIG. 8, according to one embodiment, At least some (e.g., two or more) of the wireless module 821, the Wifi module 823, the BT module 825, the GPS module 827 or the NFC module 828 may be included in one integrated chip (IC) have. (E.g., cellular module 821) of the processors corresponding to each of cellular module 821, Wifi module 823, BT module 825, GPS module 827, or NFC module 828, And a Wifi processor corresponding to the Wifi module 823) can be implemented in one SoC.

The RF module 829 can transmit and receive data, for example, transmit and receive RF signals. The RF module 829 may include, for example, a transceiver, a power amplifier module (PAM), a frequency filter, or a low noise amplifier (LNA). In addition, the RF module 829 may further include a component for transmitting and receiving electromagnetic waves in a free space in a wireless communication, for example, a conductor or a conductor. 8, the cellular module 821, the Wifi module 823, the BT module 825, the GPS module 827, and the NFC module 828 are shown sharing one RF module 829, At least one of the cellular module 821, the Wifi module 823, the BT module 825, the GPS module 827, or the NFC module 828 transmits and receives an RF signal through a separate RF module can do.

The SIM card 824 may be a card including a subscriber identity module and may be inserted into a slot formed at a specific location of the electronic device. The SIM card 824 may include unique identification information (e.g., an integrated circuit card identifier (ICCID)) or subscriber information (e.g., international mobile subscriber identity (IMSI)).

The memory 830 (e.g., the memory 130) may include an internal memory 832 or an external memory 834. The built-in memory 832 may be a nonvolatile memory such as a dynamic RAM (DRAM), a static RAM (SRAM), a synchronous dynamic RAM (SDRAM), or the like, For example, one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, NAND flash memory, And may include at least one.

According to one embodiment, the internal memory 832 may be a solid state drive (SSD). The external memory 834 may be a flash drive such as a compact flash (CF), a secure digital (SD), a micro secure digital (SD), a mini secure digital (SD), an extreme digital A Memory Stick, and the like. The external memory 834 may be functionally connected to the electronic device 800 through various interfaces. According to one embodiment, the electronic device 800 may further include a storage device (or storage medium) such as a hard drive.

The sensor module 180 may measure the physical quantity or sense the operating state of the electronic device 800 and convert the measured or sensed information into electrical signals. The sensor module 180 includes a gesture sensor 840A, a gyro sensor 840B, an air pressure sensor 840C, a magnetic sensor 840D, an acceleration sensor 840E, a grip sensor 840F, A color sensor 840G, a color sensor 840H such as an RGB (red, green, blue) sensor, a biosensor 840I, a temperature / humidity sensor 840J, an illuminance sensor 840K, 840M). ≪ / RTI > Additionally or alternatively, the sensor module 180 may include, for example, an E-nose sensor (not shown), an EMG sensor (not shown), an EEG sensor (not shown) (Not shown), an iris sensor (not shown), or a fingerprint sensor (not shown). The sensor module 180 may further include a control circuit for controlling at least one or more sensors belonging to the sensor module 180.

The input device 850 may include a touch panel 852, a (digital) pen sensor 854, a key 856 or an ultrasonic input device 858 . The touch panel 852 can recognize a touch input in at least one of an electrostatic type, a pressure sensitive type, an infrared type, and an ultrasonic type. In addition, the touch panel 852 may further include a control circuit. In electrostatic mode, physical contact or proximity recognition is possible. The touch panel 852 may further include a tactile layer. In this case, the touch panel 852 may provide a tactile response to the user.

The (digital) pen sensor 854 may be implemented using the same or similar method as receiving the touch input of the user, for example, or using a separate recognition sheet. The key 856 may include, for example, a physical button, an optical key or a keypad. The ultrasonic input device 858 is an apparatus that can confirm data by sensing sound waves from an electronic device 800 to a microphone (e.g., a microphone 888) through an input tool for generating an ultrasonic signal, Recognition is possible. According to one embodiment, the electronic device 800 may use the communication module 820 to receive user input from an external device (e.g., a computer or a server) connected thereto.

The display 860 (e.g., the display 150) may include a panel 862, a hologram device 864, or a projector 866. The panel 862 may be, for example, a liquid-crystal display (LCD) or an active-matrix organic light-emitting diode (AM-OLED). The panel 862 may be embodied, for example, flexible, transparent or wearable. The panel 862 may be formed of a single module with the touch panel 852. The hologram device 864 can display a stereoscopic image in the air using interference of light. The projector 866 can display an image by projecting light onto a screen. The screen may be located, for example, inside or outside the electronic device 800. According to one embodiment, the display 860 may further include control circuitry for controlling the panel 862, the hologram device 864, or the projector 866.

The interface 870 may include a high-definition multimedia interface (HDMI) 872, a universal serial bus (USB) 874, an optical interface 876, or a D- ) ≪ / RTI > The interface 870 may be included in, for example, the communication interface 160 shown in FIG. Additionally or alternatively, the interface 870 may be implemented with a mobile high-definition link (MHL) interface, a secure digital (SD) card / multi-media card (MMC) interface, or an infrared data association . ≪ / RTI >

The audio module 880 can convert sound and electrical signals in both directions. At least some of the components of the audio module 880 may be included, for example, in the input / output interface 140 shown in FIG. The audio module 880 may process sound information input or output through, for example, a speaker 882, a receiver 884, an earphone 886, a microphone 888, or the like.

According to one embodiment, the camera module 190 may include at least one image sensor (e.g., a front sensor or a rear sensor), a lens (not shown), an image signal processor (ISP) (Not shown) or a flash (not shown), such as an LED or xenon lamp.

The power management module 895 may manage the power of the electronic device 800. Although not shown, the power management module 895 may include, for example, a power management integrated circuit (PMIC), a charger integrated circuit (PMIC), or a battery or fuel gauge.

The PMIC can be mounted, for example, in an integrated circuit or a SoC semiconductor. The charging method can be classified into wired and wireless. The charging IC can charge the battery, and can prevent an overvoltage or an overcurrent from the charger. According to one embodiment, the charging IC may comprise a charging IC for at least one of a wired charging scheme or a wireless charging scheme. The wireless charging system may be, for example, a magnetic resonance system, a magnetic induction system or an electromagnetic wave system, and additional circuits for wireless charging may be added, such as a coil loop, a resonant circuit or a rectifier have.

The battery gauge can measure the remaining amount of the battery 896, the voltage during charging, the current or the temperature, for example. The battery 896 may store or generate electricity and may supply power to the electronic device 800 using stored or generated electricity. The battery 896 may include, for example, a rechargeable battery or a solar battery.

The indicator 897 may indicate a specific state of the electronic device 800 or a portion thereof (e.g., the AP 810), such as a boot state, a message state, or a charged state. The motor 898 can convert an electrical signal to mechanical vibration. Although not shown, the electronic device 800 may include a processing unit (e.g., a GPU) for mobile TV support. The processing device for supporting the mobile TV can process media data conforming to standards such as digital multimedia broadcasting (DMB), digital video broadcasting (DVB), or media flow.

Each of the above-described components of the electronic device according to the present disclosure can be composed of one or more components, and the name of the component can be changed according to the type of the electronic device. The electronic device according to the present disclosure may be configured to include at least one of the above-described components, and some components may be omitted or further include other additional components. In addition, some of the components of the electronic device according to the present disclosure may be combined and configured as an entity, so that the functions of the corresponding components before being combined can be performed in the same manner.

The term "module" as used herein may mean a unit comprising, for example, one or a combination of two or more of hardware, software or firmware. A "module" may be interchangeably used with terms such as, for example, unit, logic, logical block, component or circuit. A "module" may be a minimum unit or a portion of an integrally constructed component. A "module" may be a minimum unit or a portion thereof that performs one or more functions. "Modules" may be implemented either mechanically or electronically. For example, a "module" in accordance with the present disclosure may be implemented as an application-specific integrated circuit (ASIC) chip, field-programmable gate arrays (FPGAs) or programmable logic arrays (FPGAs) logic device).

According to various embodiments, at least some of the devices (e.g., modules or functions thereof) or methods (e.g., operations) according to the present disclosure may be stored in a computer readable storage medium -readable storage media). The instructions, when executed by one or more processors (e.g., the processor 210), may cause the one or more processors to perform functions corresponding to the instructions. The computer readable storage medium may be, for example, the memory 220. At least some of the programming modules may be implemented (e.g., executed) by, for example, the processor 210. At least some of the programming modules may include, for example, modules, programs, routines, sets of instructions or processes, etc. to perform one or more functions.

The computer-readable recording medium includes a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, an optical recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD (Digital Versatile Disc) A magneto-optical medium such as a floppy disk, and a program command such as a read only memory (ROM), a random access memory (RAM), a flash memory, Module) that is configured to store and perform the functions described herein. The program instructions may also include machine language code such as those generated by a compiler, as well as high-level language code that may be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of this disclosure, and vice versa.

A module or programming module according to the present disclosure may include at least one or more of the elements described above, some of which may be omitted, or may further include other additional elements. Operations performed by modules, programming modules, or other components in accordance with the present disclosure may be performed in a sequential, parallel, iterative, or heuristic manner. Also, some operations may be performed in a different order, omitted, or other operations may be added.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments. Therefore, the scope of the present invention should not be limited by the described embodiments but should be determined by the equivalents of the claims and the claims.

Claims (24)

A method of controlling an operation of an electronic device associated with multimedia data,
Reproducing multimedia data while being functionally connected to a head wearable accessory device,
Monitoring a motion state of the electronic device using a sensor value measured by an inertial sensor;
And changing an operation of the electronic device related to the multimedia data if it is determined that the stop state of the electronic device is maintained over a reference time based on the sensor value. The method as claimed in claim 1, further comprising: after the operation of the electronic device related to the multimedia data is changed, if the movement of the electronic device is confirmed based on the sensor value,
Further comprising the step of re-modifying the operation of the electronic device associated with the multimedia data when the wear state is determined. 3. The method according to claim 2, wherein if the sensor value is the wearing pose sensor value of the electronic device after the movement of the electronic device is confirmed, then the electronic device is determined to be in the wearing state. 4. The method according to claim 3, wherein the operation of the electronic device related to the multimedia data is an operation related to the reproduction control of the multimedia data. 5. The method according to claim 4, wherein, if it is determined that the electronic device is stopped for more than the reference time, the reproduction of the multimedia data is stopped. The method as claimed in claim 5, wherein the multimedia data related to the multimedia data is restarted when the wear state is determined. 6. The method according to claim 5, wherein the multimedia data is restarted from a point of time when the stop state is detected. 6. The method of claim 5, wherein the multimedia data is restarted from a point of time before the stop state is detected. The method as claimed in claim 5, further comprising the step of turning off the display of the electronic device if the reproduction of the multimedia data is stopped and the movement of the electronic device is not confirmed during the stopping time.  The method as claimed in claim 9, further comprising turning on the display if movement of the upper limb electronic device is confirmed after the display is turned off. 10. The method of claim 9, further comprising: after turning on the display, determining whether the electronic device is in a wear state;
And restarting the playback of the multimedia data when the wear state is determined. 5. The method according to claim 4, further comprising: counting a time when the eye is wrapped using an image including eyes of a user who has played back the multimedia data, and if the counted time is longer than a predetermined time, And storing the start point of the multimedia data as a restart point of the multimedia data playback. In an electronic device,
Display,
At least one inertial sensor,
The method comprising the steps of: reproducing multimedia data while functionally connected to a head wearable accessory device; monitoring a motion state of the electronic device using a sensor value measured by the inertial sensor; and based on the sensor value, And a playback module for, when it is determined that the device is in a stopped state, changing an operation of the electronic device associated with the multimedia data. 14. The method of claim 13, wherein, if the movement of the electronic device is confirmed based on the sensor value after changing the operation of the electronic device associated with the multimedia data, the playback module determines whether the electronic device is worn, And re-changing the operation of the electronic device associated with the multimedia data if the wear state is determined. 15. The motion control apparatus according to claim 14, wherein when the sensor value is the wear pose sensor value of the electronic device after the movement of the electronic device is confirmed, the electronic device is determined to be in the wear state. 16. The apparatus according to claim 15, wherein the operation of the electronic device associated with the multimedia data is an operation related to the reproduction control of the multimedia data. 17. The apparatus according to claim 16, wherein, if it is determined that the electronic device is stopped for more than the reference time, reproduction of the multimedia data is stopped. 18. The apparatus of claim 17, wherein when the wear state is determined, the playback of the multimedia data related to the multimedia data is restarted. 18. The apparatus according to claim 17, wherein the multimedia data is restarted from a point of time when the stop state is detected. 18. The apparatus as claimed in claim 17, wherein the multimedia data is restarted from a point of time before the stop state is detected. 18. The apparatus according to claim 17, wherein the playback module turns off the display of the electronic device if playback of the multimedia data is stopped and movement of the electronic device is not confirmed during a stop time.  22. The motion control apparatus according to claim 21, wherein, when the motion of the upper limb electronic device is confirmed after turning off the display, the playback module turns on the display. 22. The apparatus according to claim 21, wherein the playback module determines whether the electronic device is in a wear state after turning on the display, and restarts playback of the multimedia data when the wear state is determined. 17. The apparatus of claim 16, wherein the playback module counts a time that the eye is wrapped using an image including eyes of the user that has been played since the multimedia data was played back, and if the counted time exceeds a predetermined time, And stores the point of time at which the eye is started to be closed as the restart point of the operation.

Images