KR102446769B1 - Electric device and method for controlling the same - Google Patents

Abstract

The present invention relates to an electronic device capable of generating meeting minutes, a body formed to be rotatable, a camera mounted on the body, a microphone for detecting voice information of a first speaker located in the vicinity of the body, and voice information detected by the microphone determines the position of the first speaker's utterance using and a controller for storing the captured image and the sensed voice information in association with each other, wherein the controller re-captures the first speaker when the captured image is not normally captured.

Description

ELECTRIC DEVICE AND METHOD FOR CONTROLLING THE SAME

The present invention relates to an electronic device capable of generating meeting minutes.

The terminal may be divided into a mobile/portable terminal and a stationary terminal according to whether the terminal is movable. Again, the mobile terminal can be divided into a handheld terminal and a vehicle mounted terminal according to whether the user can carry it directly.

The functions of mobile terminals are diversifying. For example, there are functions for data and voice communication, photography and video recording through a camera, voice recording, music file playback through a speaker system, and outputting an image or video to the display unit. Some terminals add an electronic game play function or perform a multimedia player function. In particular, recent mobile terminals can receive multicast signals that provide broadcast and visual content such as video or television programs.

As such a terminal is diversified in functions, for example, in the form of a multimedia device (Multimedia player) equipped with complex functions such as taking pictures or videos, playing music or video files, playing games, and receiving broadcasts. is being implemented.

In order to support and increase the function of the terminal, it may be considered to improve the structural part and/or the software part of the terminal.

Recently, for reasons such as information accumulation, the use rate of a voice recording function for recording voice is increasing. Accordingly, a vast amount of voice information is accumulated, but it is difficult to search for specific information desired by a user from such a large amount of voice information.

Accordingly, in the present invention, a method for effectively providing additional information related to voice information will be described.

An object of the present invention is to improve the convenience of taking minutes of a meeting in which a plurality of speakers participate.

A mobile terminal according to an embodiment of the present invention uses a main body formed to be rotatable, a camera mounted on the main body, a microphone for detecting voice information of a first speaker located in the vicinity of the main body, and voice information detected by the microphone. to determine the position of the first speaker's utterance, the camera rotates the main body so that the first speaker is photographed, and after the rotation of the main body, the first speaker is photographed through the camera, and the photographed and a controller for storing an image and the sensed voice information in association with each other, wherein the controller re-photographs the first speaker when the captured image is not normally captured.

In an embodiment, the control unit stores the utterance position of the first speaker in a memory, when the voice information of a second speaker other than the first speaker is received before re-photographing of the first speaker, and In order to photograph the second speaker, the main body is rotated.

In an embodiment, the control unit rotates the main body based on the stored utterance position of the first speaker in order to re-photograph the first speaker after photographing the second speaker through the camera. characterized.

In an embodiment, the control unit stores the re-photographed image of the first speaker in association with the voice information of the first speaker.

In an embodiment, the controller determines whether the captured image is normally captured based on a face recognition algorithm.

In one embodiment, the control unit captures an image in real time through the camera during rotation of the main body, and when a visualization ticket is included in the captured image, determines whether a visualization ticket can be photographed, and When shooting is possible, it is characterized in that the main body is rotated to take a visualization sheet.

In one embodiment, the control unit determines that the recording of the visualization sheet is possible when the voice information is not received through the microphone or an image of the speaker corresponding to the voice information received through the microphone has already been captured characterized in that

In one embodiment, the control unit is characterized in that based on the shooting time information of the captured image, to calculate the location information of the visualization sheet.

In one embodiment, the control unit is characterized in that based on the location information of the visualization form, the visualization of the form is photographed at a preset period.

In an embodiment, the control unit determines whether the visualization ticket is the same as a pre-stored visualization ticket, and if the visualization ticket is not the same as the previously stored visualization ticket, at the time of photographing the visualization ticket and the visualization ticket It is characterized in that the image of the speaker who uttered the voice information is photographed together, and if the visualization sheet is the same as the stored visualization sheet, the visualization sheet is updated.

In one embodiment, the control unit is characterized in that the recording time of the visualization sheet, the reception time of the voice information, and the image of the first speaker are stored together.

The mobile terminal according to the present invention can create a meeting record stored by linking the video and audio information of the speaker who spoke during the meeting by using the rotatable main body to capture an image of the speaker in the conference room. Through this, the present invention can provide speaker information and voice information more intuitively.

In addition, the mobile terminal according to the present invention can improve the convenience of generating meeting minutes by determining whether the talker's image is normally captured, and automatically re-capturing when it is not normally captured.

In addition, the present invention can provide more accurate meeting minutes information by generating the meeting minutes using the image of the visualization sheet used in the meeting.

1 is a block diagram illustrating a mobile terminal related to the present invention.
2 is a conceptual view of an electronic device according to the present invention viewed from one side.
FIG. 3 is a conceptual diagram illustrating a state in which the electronic device 100 according to the present invention generates meeting minutes, and FIG. 4 is a flowchart illustrating a method of generating meeting minutes in the electronic device 100 according to the present invention.
5 is a conceptual diagram illustrating an embodiment of outputting the meeting minutes generated by the electronic device 100 according to the present invention.
6 is a flowchart illustrating a method of re-capturing an image of a speaker in an electronic device according to the present invention.
7 is a conceptual diagram illustrating a situation in which a meeting is conducted using a visualization sheet, FIG. 8 is a flowchart illustrating a method of photographing a visualization sheet by an electronic device according to the present invention, and FIG. It is a conceptual diagram showing an example of a user interface to
10 is a flowchart illustrating a method of updating a visualization sheet when an electronic device generates meeting minutes according to the present invention.

Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components will be given the same reference numerals regardless of reference numerals, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" for components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have distinct meanings or roles by themselves. In addition, in describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed herein is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention , should be understood to include equivalents or substitutes.

Terms including ordinal numbers such as first, second, etc. may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The singular expression includes the plural expression unless the context clearly dictates otherwise.

In the present application, terms such as “comprises” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that this does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

The portable electronic device described in this specification includes a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, personal digital assistants (PDA), a portable multimedia player (PMP), a navigation system, and a slate PC. ), tablet PCs, ultrabooks, wearable devices, such as watch-type terminals (smartwatch), glass-type terminals (smart glass), HMD (head mounted display), etc. can

However, it can be easily understood by those skilled in the art that the configuration according to the embodiments described in this specification can be applied to fixed terminals such as digital TVs, desktop computers, digital signage, etc., except when applicable only to portable electronic devices. There will be.

1 is a block diagram illustrating a portable electronic device related to the present invention. Referring to FIG. 1 , the portable electronic device 100 includes a wireless communication unit 110 , an input unit 120 , a sensing unit 140 , an output unit 150 , an interface unit 160 , a memory 170 , and a control unit ( 180) and a power supply unit 190, and the like. The components shown in FIG. 1 are not essential for implementing the portable electronic device, and thus the portable electronic device described herein may have more or fewer components than those listed above.

More specifically, among the above components, the wireless communication unit 110 may be connected between the portable electronic device 100 and the wireless communication system, between the portable electronic device 100 and another portable electronic device 100 , or the portable electronic device 100 . ) and one or more modules that enable wireless communication between an external server. In addition, the wireless communication unit 110 may include one or more modules for connecting the portable electronic device 100 to one or more networks.

The wireless communication unit 110 may include at least one of a broadcast reception module 111 , a mobile communication module 112 , a wireless Internet module 113 , a short-range communication module 114 , and a location information module 115 . .

The input unit 120 includes a camera 121 or an image input unit for inputting an image signal, a microphone 122 or an audio input unit for inputting an audio signal, and a user input unit 123 for receiving information from a user, for example, , a touch key, a push key, etc.). The voice data or image data collected by the input unit 120 may be analyzed and processed as a user's control command.

The sensing unit 140 may include one or more sensors for sensing at least one of information in the portable electronic device, surrounding environment information surrounding the portable electronic device, and user information. For example, the sensing unit 140 may include a proximity sensor 141, an illumination sensor 142, an illumination sensor, a touch sensor, an acceleration sensor, a magnetic sensor, and gravity. Sensor (G-sensor), gyroscope sensor, motion sensor, RGB sensor, infrared sensor (IR sensor: infrared sensor), fingerprint sensor (finger scan sensor), ultrasonic sensor , optical sensors (eg, cameras (see 121)), microphones (see 122), battery gauges, environmental sensors (eg, barometers, hygrometers, thermometers, radiation sensors, It may include at least one of a thermal sensor, a gas sensor, etc.) and a chemical sensor (eg, an electronic nose, a healthcare sensor, a biometric sensor, etc.). Meanwhile, the portable electronic device disclosed in the present specification may combine and utilize information sensed by at least two or more of these sensors.

The output unit 150 is for generating an output related to visual, auditory or tactile sense, and includes at least one of a display unit 151 , a sound output unit 152 , a haptip module 153 , and an optical output unit 154 . can do. The display unit 151 may implement a touch screen by forming a layer structure with the touch sensor or being formed integrally with the touch sensor. Such a touch screen may function as a user input unit 123 providing an input interface between the portable electronic device 100 and a user, and may provide an output interface between the portable electronic device 100 and a user.

The interface unit 160 serves as a passage with various types of external devices connected to the portable electronic device 100 . Such an interface unit 160, a wired / wireless headset port (port), an external charger port (port), a wired / wireless data port (port), a memory card (memory card) port, for connecting a device equipped with an identification module It may include at least one of a port, an audio I/O (Input/Output) port, a video I/O (Input/Output) port, and an earphone port. In response to the connection of the external device to the interface unit 160 , the portable electronic device 100 may perform appropriate control related to the connected external device.

In addition, the memory 170 stores data supporting various functions of the portable electronic device 100 . The memory 170 may store a plurality of application programs (or applications) driven in the portable electronic device 100 , data for operation of the portable electronic device 100 , and commands. At least some of these application programs may be downloaded from an external server through wireless communication. In addition, at least some of these application programs may exist on the portable electronic device 100 from the time of shipment for basic functions (eg, incoming calls, outgoing functions, message reception, and outgoing functions) of the portable electronic device 100 . have. Meanwhile, the application program may be stored in the memory 170 , installed on the portable electronic device 100 , and driven to perform an operation (or function) of the portable electronic device by the controller 180 .

In addition to the operation related to the application program, the controller 180 generally controls the overall operation of the portable electronic device 100 . The controller 180 may provide or process appropriate information or functions to the user by processing signals, data, information, etc. input or output through the above-described components or by driving an application program stored in the memory 170 .

Also, the controller 180 may control at least some of the components discussed with reference to FIG. 1 in order to drive an application program stored in the memory 170 . Furthermore, in order to drive the application program, the controller 180 may operate at least two or more of the components included in the portable electronic device 100 in combination with each other.

The power supply unit 190 receives external power and internal power under the control of the controller 180 to supply power to each component included in the portable electronic device 100 . The power supply unit 190 includes a battery, and the battery may be a built-in battery or a replaceable battery.

At least some of the respective components may operate in cooperation with each other to implement the operation, control, or control method of the portable electronic device according to various embodiments described below. Also, the operation, control, or control method of the portable electronic device may be implemented on the portable electronic device by driving at least one application program stored in the memory 170 .

Hereinafter, before looking at various embodiments implemented through the portable electronic device 100 as described above, the above-listed components will be described in more detail with reference to FIG. 1 .

First, referring to the wireless communication unit 110 , the broadcast reception module 111 of the wireless communication unit 110 receives a broadcast signal and/or broadcast related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Two or more of the broadcast reception modules may be provided to the mobile terminal 100 for simultaneous broadcast reception or broadcast channel switching for at least two broadcast channels.

Mobile communication module 112, the technical standards or communication methods for mobile communication (eg, Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (CDMA2000), EV -DO (Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), WCDMA (Wideband CDMA), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced, etc.) transmits and receives radio signals to and from at least one of a base station, an external terminal, and a server on a mobile communication network constructed according to (Long Term Evolution-Advanced).

The wireless signal may include various types of data according to transmission/reception of a voice call signal, a video call signal, or a text/multimedia message.

The wireless Internet module 113 refers to a module for wireless Internet access, and may be built-in or external to the portable electronic device 100 . The wireless Internet module 113 is configured to transmit and receive wireless signals in a communication network according to wireless Internet technologies.

As wireless Internet technology, for example, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), etc., and the wireless Internet module ( 113) transmits and receives data according to at least one wireless Internet technology within a range including Internet technologies not listed above.

From the point of view that wireless Internet access by WiBro, HSDPA, HSUPA, GSM, CDMA, WCDMA, LTE, LTE-A, etc. is made through a mobile communication network, the wireless Internet module 113 performs wireless Internet access through the mobile communication network. ) may be understood as a type of the mobile communication module 112 .

The short-range communication module 114 is for short-range communication, and includes Bluetooth™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, NFC. Short-distance communication may be supported by using at least one of (Near Field Communication), Wireless-Fidelity (Wi-Fi), Wi-Fi Direct, and Wireless Universal Serial Bus (USB) technologies. The short-distance communication module 114, between the portable electronic device 100 and the wireless communication system, between the portable electronic device 100 and other portable electronic devices 100 through wireless area networks (Wireless Area Networks), or portable It is possible to support wireless communication between the electronic device 100 and a network in which another portable electronic device 100 or an external server is located. The local area network may be local area networks (Wireless Personal Area Networks).

Here, the other portable electronic device 100 is a wearable device capable of exchanging (or interworking) data with the portable electronic device 100 according to the present invention, for example, a smart watch, It may be a smart glass or a head mounted display (HMD). The short-range communication module 114 may detect (or recognize) a wearable device capable of communicating with the portable electronic device 100 in the vicinity of the portable electronic device 100 . Furthermore, when the detected wearable device is a device authenticated to communicate with the portable electronic device 100 according to the present invention, the controller 180 transmits at least a portion of data processed by the portable electronic device 100 to the short-distance communication. It can be transmitted to the wearable device through the module 114 . Accordingly, the user of the wearable device may use data processed by the portable electronic device 100 through the wearable device. For example, according to this, when a call is received in the portable electronic device 100, the user performs a phone call through the wearable device, or when a message is received in the portable electronic device 100, the user uses the wearable device It is possible to check the received message.

The location information module 115 is a module for acquiring a location (or current location) of a portable electronic device, and a representative example thereof includes a Global Positioning System (GPS) module or a Wireless Fidelity (WiFi) module. For example, if the portable electronic device utilizes a GPS module, the location of the portable electronic device may be acquired by using a signal transmitted from a GPS satellite. As another example, if the mobile electronic device utilizes the Wi-Fi module, it can acquire the location of the mobile electronic device based on the information of the Wi-Fi module and the wireless AP (Wireless Access Point) that transmits or receives the wireless signal. have. If necessary, the location information module 115 may perform any function of the other modules of the wireless communication unit 110 to obtain data on the location of the portable electronic device as a substitute or additionally. The location information module 115 is a module used to obtain the location (or current location) of the portable electronic device, and is not limited to a module that directly calculates or obtains the location of the portable electronic device.

Next, the input unit 120 is for input of image information (or signal), audio information (or signal), data, or information input from a user. For input of image information, the portable electronic device 100 is One or a plurality of cameras 121 may be provided. The camera 121 processes an image frame such as a still image or a moving image obtained by an image sensor in a video call mode or a shooting mode. The processed image frame may be displayed on the display unit 151 or stored in the memory 170 . On the other hand, the plurality of cameras 121 provided in the portable electronic device 100 may be arranged to form a matrix structure, and through the cameras 121 forming the matrix structure as described above, the portable electronic device 100 may be provided at various angles or A plurality of image information having a focus may be input. In addition, the plurality of cameras 121 may be arranged in a stereo structure to acquire a left image and a right image for realizing a stereoscopic image.

The microphone 122 processes an external sound signal as electrical voice data. The processed voice data may be variously utilized according to a function (or a running application program) being performed by the portable electronic device 100 . Meanwhile, various noise removal algorithms for removing noise generated in the process of receiving an external sound signal may be implemented in the microphone 122 .

The user input unit 123 is for receiving information from a user. When information is input through the user input unit 123 , the controller 180 may control the operation of the portable electronic device 100 to correspond to the input information. have. The user input unit 123 is a mechanical input means (or a mechanical key, for example, a button positioned on the front, rear or side of the portable electronic device 100 , a dome switch, and a jog wheel). , jog switch, etc.) and a touch input means. As an example, the touch input means consists of a virtual key, a soft key, or a visual key displayed on the touch screen through software processing, or a part other than the touch screen. On the other hand, the virtual key or the visual key may be displayed on the touch screen while having various shapes, for example, a graphic, text ), an icon, a video, or a combination thereof.

Meanwhile, the sensing unit 140 senses at least one of information in the portable electronic device, surrounding environment information surrounding the portable electronic device, and user information, and generates a sensing signal corresponding thereto. The controller 180 may control the driving or operation of the portable electronic device 100 or perform data processing, functions, or operations related to an application program installed in the portable electronic device 100 based on the sensing signal. Representative sensors among various sensors that may be included in the sensing unit 140 will be described in more detail.

First, the proximity sensor 141 refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object existing in the vicinity without mechanical contact using the force of an electromagnetic field or infrared rays. The proximity sensor 141 may be disposed in an inner region of the portable electronic device covered by the touch screen described above or near the touch screen.

Examples of the proximity sensor 141 include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive type proximity sensor, a magnetic type proximity sensor, an infrared proximity sensor, and the like. When the touch screen is capacitive, the proximity sensor 141 may be configured to detect the proximity of an object having conductivity by a change in an electric field according to the proximity of the object. In this case, the touch screen (or touch sensor) itself may be classified as a proximity sensor.

On the other hand, for convenience of description, the act of approaching an object on the touch screen without contacting it so that the object is recognized that it is located on the touch screen is called “proximity touch”, and the touch The act of actually touching an object on the screen is called "contact touch". The position where the object is touched in proximity on the touch screen means a position where the object is perpendicular to the touch screen when the object is touched in proximity. The proximity sensor 141 may detect a proximity touch and a proximity touch pattern (eg, proximity touch distance, proximity touch direction, proximity touch speed, proximity touch time, proximity touch position, proximity touch movement state, etc.) have. Meanwhile, the controller 180 processes data (or information) corresponding to the proximity touch operation and the proximity touch pattern sensed through the proximity sensor 141 as described above, and furthermore, provides visual information corresponding to the processed data. It can be printed on the touch screen. Furthermore, the controller 180 may control the portable electronic device 100 to process different operations or data (or information) depending on whether a touch to the same point on the touch screen is a proximity touch or a contact touch. have.

The touch sensor receives a touch (or touch input) applied to the touch screen (or the display unit 151) using at least one of various touch methods such as a resistive film method, a capacitive method, an infrared method, an ultrasonic method, and a magnetic field method. detect

As an example, the touch sensor may be configured to convert a change in pressure applied to a specific part of the touch screen or a change in capacitance occurring in a specific part of the touch screen into an electrical input signal. The touch sensor may be configured to detect a position, an area, a pressure at the time of touch, capacitance at the time of touch, and the like where a touch object applying a touch on the touch screen is touched on the touch sensor. Here, the touch object is an object that applies a touch to the touch sensor, and may be, for example, a finger, a touch pen, a stylus pen, or a pointer.

As such, when there is a touch input to the touch sensor, a signal(s) corresponding thereto is sent to the touch controller. The touch controller processes the signal(s) and then transmits corresponding data to the controller 180 . Accordingly, the controller 180 can know which area of the display unit 151 has been touched, and the like. Here, the touch controller may be a component separate from the controller 180 , or may be the controller 180 itself.

Meanwhile, the controller 180 may perform different controls or may perform the same control according to the type of the touch object that touches the touch screen (or a touch key provided in addition to the touch screen). Whether to perform different control or the same control according to the type of the touch object may be determined according to the current operating state of the portable electronic device 100 or a running application program.

On the other hand, the above-described touch sensor and proximity sensor independently or in combination, a short (or tap) touch on the touch screen (short touch), long touch (long touch), multi touch (multi touch), drag touch (drag touch) ), flick touch, pinch-in touch, pinch-out touch, swype touch, hovering touch, etc. It can sense touch.

The ultrasonic sensor may recognize location information of a sensing target by using ultrasonic waves. Meanwhile, the controller 180 may calculate the position of the wave source based on information sensed by the optical sensor and the plurality of ultrasonic sensors. The position of the wave source may be calculated using the property that light is much faster than ultrasonic waves, that is, the time at which light arrives at the optical sensor is much faster than the time at which ultrasonic waves reach the ultrasonic sensor. More specifically, the position of the wave source may be calculated by using a time difference from the time that the ultrasonic wave arrives using light as a reference signal.

On the other hand, the camera 121 as seen in the configuration of the input unit 120 includes at least one of a camera sensor (eg, CCD, CMOS, etc.), a photo sensor (or an image sensor), and a laser sensor.

The camera 121 and the laser sensor may be combined with each other to detect a touch of a sensing target for a 3D stereoscopic image. The photo sensor may be stacked on the display device, and the photo sensor is configured to scan the motion of the sensing target close to the touch screen. More specifically, the photo sensor mounts photo diodes and transistors (TRs) in rows/columns and scans the contents placed on the photo sensor using electrical signals that change according to the amount of light applied to the photo diodes. That is, the photo sensor calculates the coordinates of the sensing target according to the amount of change in light, and through this, location information of the sensing target can be obtained.

The display unit 151 displays (outputs) information processed by the portable electronic device 100 . For example, the display unit 151 may display execution screen information of an application program driven in the portable electronic device 100 or UI (User Interface) and GUI (Graphic User Interface) information according to the execution screen information. have.

Also, the display unit 151 may be configured as a stereoscopic display unit for displaying a stereoscopic image.

A three-dimensional display method such as a stereoscopic method (glasses method), an auto stereoscopic method (glassesless method), or a projection method (holographic method) may be applied to the stereoscopic display unit.

The sound output unit 152 may output audio data received from the wireless communication unit 110 or stored in the memory 170 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. The sound output unit 152 also outputs a sound signal related to a function (eg, a call signal reception sound, a message reception sound, etc.) performed by the portable electronic device 100 . The sound output unit 152 may include a receiver, a speaker, a buzzer, and the like.

The haptic module 153 generates various tactile effects that the user can feel. A representative example of the tactile effect generated by the haptic module 153 may be vibration. The intensity and pattern of vibration generated by the haptic module 153 may be controlled by a user's selection or setting of the controller. For example, the haptic module 153 may synthesize and output different vibrations or output them sequentially.

In addition to vibration, the haptic module 153 is configured to respond to stimuli such as a pin arrangement that moves vertically with respect to the contact skin surface, a jet or suction force of air through a nozzle or an inlet, a brush against the skin surface, contact of an electrode, an electrostatic force, etc. Various tactile effects can be generated, such as an effect caused by heat absorption and an effect by reproducing a feeling of cold and heat using an element capable of absorbing heat or generating heat.

The haptic module 153 may not only deliver a tactile effect through direct contact, but may also be implemented so that the user can feel the tactile effect through a muscle sense such as a finger or arm. Two or more haptic modules 153 may be provided according to the configuration of the portable electronic device 100 .

The light output unit 154 outputs a signal for notifying the occurrence of an event by using the light of the light source of the portable electronic device 100 . Examples of the event occurring in the portable electronic device 100 may be message reception, call signal reception, missed call, alarm, schedule notification, email reception, information reception through an application, and the like.

The signal output from the light output unit 154 is realized when the portable electronic device emits light of a single color or a plurality of colors toward the front or back. The signal output may be terminated when the portable electronic device detects the user's event confirmation.

The interface unit 160 serves as a passage with all external devices connected to the portable electronic device 100 . The interface unit 160 receives data from an external device, receives power and transmits it to each component inside the portable electronic device 100 , or allows data inside the portable electronic device 100 to be transmitted to an external device. For example, a wired/wireless headset port, an external charger port, a wired/wireless data port, a memory card port, a port for connecting a device equipped with an identification module (port), audio I/O (Input/Output) port (port), video I/O (Input/Output) port (port), earphone port (port), etc. may be included in the interface unit 160 .

On the other hand, the identification module is a chip storing various information for authenticating the use authority of the portable electronic device 100 , a user identification module (UIM), a subscriber identity module (SIM), a general user It may include a universal subscriber identity module (USIM) and the like. A device equipped with an identification module (hereinafter, 'identification device') may be manufactured in the form of a smart card. Accordingly, the identification device may be connected to the terminal 100 through the interface unit 160 .

In addition, the interface unit 160 is a passage through which power from the cradle is supplied to the portable electronic device 100 when the portable electronic device 100 is connected to an external cradle, or is removed from the cradle by a user. It may be a path through which input various command signals are transmitted to the portable electronic device 100 . The various command signals or the power input from the cradle may be operated as signals for recognizing that the portable electronic device 100 is correctly mounted on the cradle.

The memory 170 may store a program for the operation of the controller 180 and may temporarily store input/output data (eg, a phone book, a message, a still image, a moving picture, etc.). The memory 170 may store data related to vibrations and sounds of various patterns output when a touch input is performed on the touch screen.

Memory 170 is a flash memory type (flash memory type), a hard disk type (hard disk type), an SSD type (Solid State Disk type), an SDD type (Silicon Disk Drive type), a multimedia card micro type ), card-type memory (such as SD or XD memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read (EEPROM) -only memory), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk may include at least one type of storage medium. The portable electronic device 100 may be operated in relation to a web storage that performs a storage function of the memory 170 on the Internet.

Meanwhile, as described above, the controller 180 controls the operation related to the application program and the general operation of the portable electronic device 100 . For example, if the state of the portable electronic device satisfies a set condition, the controller 180 may execute or release a lock state that restricts input of a user's control command to applications.

In addition, the controller 180 performs control and processing related to voice calls, data communication, video calls, etc., or performs pattern recognition processing capable of recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively. can Furthermore, in order to implement various embodiments described below on the portable electronic device 100 according to the present invention, the controller 180 may control any one or a plurality of the components described above by combining them.

The power supply unit 190 receives external power and internal power under the control of the control unit 180 to supply power required for operation of each component. The power supply unit 190 includes a battery, and the battery may be a built-in battery configured to be rechargeable, and may be detachably coupled to the terminal body for charging or the like.

In addition, the power supply unit 190 may include a connection port, and the connection port may be configured as an example of the interface 160 to which an external charger that supplies power for charging the battery is electrically connected.

As another example, the power supply unit 190 may be configured to charge the battery in a wireless manner without using the connection port. In this case, the power supply unit 190 using one or more of an inductive coupling method based on a magnetic induction phenomenon or a resonance coupling method based on an electromagnetic resonance phenomenon from an external wireless power transmitter. power can be transmitted.

Meanwhile, various embodiments below may be implemented in a computer-readable recording medium using, for example, software, hardware, or a combination thereof.

Hereinafter, an electronic device including at least one of the components described with reference to FIG. 1 will be described. 2 is a conceptual view of an electronic device according to the present invention viewed from one side.

Referring to FIG. 2 , the electronic device according to the present invention may include a main body 100 , a camera 121 , a microphone 122 , and a speaker (or a sound output unit, 152 ).

The body 100 may have a 360-degree rotatable shape. For example, the body 100 may have a cylindrical shape. As for the main body 100, the entire main body may be rotated, or only one region of the parts constituting the main body may be rotated. For example, the electronic device 100 may be formed such that only the upper region of the body on which the camera 121 is mounted is rotated. The body 100 may be made of a hard material such as plastic or metal.

The camera 121 may be mounted on one area of the outer surface of the body 100 . The camera 121 is disposed in a rotatable area of the main body 100 , and a photographing range of the camera 121 may be changed by the rotation of the main body 100 .

The microphone 122 and the speaker 152 may be mounted on the outer surface of the main body. The microphone 122 and the speaker 152 may be configured in plurality, and may be disposed on the outer surface of the main body to be spaced apart from each other. Through this, the electronic device 100 may receive voice information generated in all directions of the main body 100 .

In the above, the electronic device 100 according to the present invention has been described. Hereinafter, a method for generating meeting minutes using such an electronic device will be described.

FIG. 3 is a conceptual diagram illustrating a state in which the electronic device 100 according to the present invention generates meeting minutes, and FIG. 4 is a flowchart illustrating a method of generating meeting minutes in the electronic device 100 according to the present invention. 5 is a conceptual diagram illustrating an embodiment of outputting the meeting minutes generated by the electronic device 100 according to the present invention.

The electronic device 100 according to the present invention may execute a meeting minutes writing function to create meeting minutes. The meeting minutes function is a function that converts voice information spoken during a meeting into text information, and creates meeting minutes by linking the text information with the speaker's image information. Such a meeting minutes writing function may be installed as a basic application of the electronic device 100 , or a user may directly install it on the electronic device 100 as a 3rd party app.

The controller 180 may execute a meeting minutes writing function based on a user control command or when a preset condition is satisfied. The preset condition may be a condition to coincide with a pre-stored meeting schedule, a condition for receiving voice information of a plurality of speakers for a predetermined time, a condition for the current location of the electronic device 100 to be a location stored as a meeting place, and the like. In this case, the controller 180 may execute a meeting minutes writing function without a user's control command.

Meanwhile, the electronic device 100 may be located in an area in which a plurality of speakers participating in a meeting can be photographed. For example, as shown in FIG. 3 , the electronic device 100 may be located at the center of the positions of the plurality of speakers.

Referring to FIG. 4 , the electronic device 100 may detect the speaker's voice information ( S410 ). The controller 180 may detect voice information through the microphone 152 while the meeting minutes writing function is being executed. The controller 180 may extract characteristic information for distinguishing a speaker from the sensed voice information. The characteristic information may include wavelength, frequency, period, amplitude, intensity, propagation speed, and the like.

The controller 180 may generate speaker information corresponding to the sensed voice information based on a voice detection algorithm. The voice detection algorithm is an algorithm for discriminating different speakers based on voice characteristic information, and is a technique well known to those skilled in the art, and a detailed description thereof will be omitted herein.

After generating speaker information (eg, first speaker) from the sensed voice information, the controller 180 may convert the voice information into text information. In addition, the speaker information and the voice information may be linked and stored.

Also, the controller 180 may determine the location of the speaker's utterance by using the sensed voice information (S420). The controller 180 may extract information on a time point and a reception direction at which the voice information is received through the plurality of microphones 152 , and calculate the utterance position of the speaker based thereon.

Then, the controller 180 may rotate the main body to photograph the speaker based on the calculated utterance position ( S430 ). The controller 180 may rotate the main body 100 so that the speaker is located within a shooting range of the camera 121 in order to capture an image of the speaker.

After the main body is rotated, the controller 180 may photograph the speaker (S440). In addition, the controller 180 may determine whether the image of the talker is photographed normally (S450).

Whether to photograph normally may be determined by a face extraction algorithm. For example, the controller 180 may determine that the photograph has been taken normally when the photographing is performed to the extent that all feature points of the face can be extracted from the photographed image. Conversely, when the feature points of the face are not extracted from the captured image, the controller 180 may determine that the photograph is abnormal.

Alternatively, whether normal shooting is performed may be determined based on image brightness, image sharpness, image illuminance, and the like. For example, the controller 180 extracts at least one of brightness, sharpness, and illuminance from the captured image, and when the extracted at least one brightness, sharpness, and illuminance satisfies a reference value, it may be determined as normal shooting. . Conversely, when the extracted at least one brightness, sharpness, and illuminance do not satisfy the reference values, the controller 180 may determine that the photographing is abnormal.

When the captured image is normally captured, the controller 180 may store the captured image and audio information in association with each other (S470). The controller 180 may store the captured image and the image obtained by converting audio information into text together in the memory 170 . More specifically, the controller 180 may store the captured image and audio information converted into text in the memory 170 to be output together on the touch screen 151 . Accordingly, the present invention can more intuitively provide speaker information when writing meeting minutes.

Meanwhile, when it is determined that the captured image is abnormally captured, the controller 180 may proceed with re-photographing of the speaker ( S460 ). When the speaker's image is abnormally captured, the controller 180 may perform re-capturing without associating the abnormally captured image with audio information. Accordingly, the present invention can improve user convenience by automatically re-photographing when an image added to the meeting minutes is taken abnormally.

The controller 180 may store, in the memory 170 , meeting information in which the speaker's video and text information converted from the speaker's voice information are linked with each other for each of the plurality of speakers participating in the meeting. In addition, the controller 180 may generate meeting minutes by sequentially arranging the meeting information in the order in which the voice information was received.

Referring to FIG. 5 , the controller 180 may output an image of a talker and voice information of the talker to an area adjacent to each other on the touch screen 151 based on the reception of the request for outputting the generated meeting minutes. have. Through this, the present invention can more intuitively provide speaker information and speaker's utterance information during a meeting.

In the above, the method for generating the meeting minutes of the electronic device according to the present invention has been described. Hereinafter, a method of re-capturing an image of a speaker in the electronic device according to the present invention will be described in more detail.

6 is a flowchart illustrating a method of re-capturing an image of a speaker in an electronic device according to the present invention.

The controller 180 may receive different voice information from a plurality of speakers at different times in a conference room in which a plurality of speakers exist. For example, the controller 180 may receive voice information of the first speaker at a first time point, and receive voice information of the second speaker at a second time point.

As described in FIG. 4 , when voice information of the first speaker is received, the controller 180 may capture an image of the first speaker. And, referring to FIG. 6 , the controller 180 may determine whether an image of the first speaker is normally captured ( S450 ).

When the image of the first speaker is not normally captured, the controller 180 may re-capture the image of the first speaker. In this case, the controller 180 may determine whether to receive the voice information of the second speaker before re-capturing the image of the first speaker ( S610 ). Here, the second speaker is a speaker different from the first speaker. The controller 180 may distinguish the first speaker from the second speaker based on the characteristic information of the voice information received through the microphone.

When the voice information received through the microphone before re-photographing of the first talker is the voice information of the first talker or before the re-capturing of the first talker, the control unit 180 determines that voice information is not received through the microphone, Re-photographing of the first speaker may be performed ( S640 ).

However, before re-photographing of the first speaker, if the voice information received through the microphone is the voice information of the second speaker, the controller 180 stores the location information of the first speaker and then captures the image of the second speaker. can be (S620).

When the voice information of the second speaker is received, the controller 180 does not proceed with re-photographing of the first speaker, but based on the voice information corresponding to the second speaker, the control unit 180 does not proceed with the photographing of the second speaker. Location information can be calculated. Then, the controller 180 may rotate the main body based on the calculated position information. In this case, the controller 180 may store the location information of the first speaker in the memory. In this case, the controller 180 may temporarily store in the memory 170 text information obtained by converting the abnormally captured image of the first speaker and the voice information of the first speaker.

After capturing the image of the second speaker, the controller 180 may re-capture the image of the first speaker ( S630 ). The controller 180 may store, in the memory 170 , text information obtained by converting the image of the second speaker and the voice information of the second speaker. In addition, after capturing an image of the second speaker, the controller 180 may rotate the main body based on pre-stored position information of the first speaker. After the main body is rotated, the controller 180 may re-capture the image of the first speaker. The controller 180 deletes the abnormally photographed image of the first speaker from the memory 170, and the re-photographed image of the first speaker together with text information obtained by converting the voice information of the first speaker together with the memory 170 can be stored in Through this, the present invention can generate meeting minutes providing accurate speaker information.

In the above, a method of re-photographing the speaker for generating meeting minutes has been described. Hereinafter, a method of photographing a visualization sheet used in a meeting will be described.

7 is a conceptual diagram illustrating a situation in which a meeting is conducted using a visualization sheet, FIG. 8 is a flowchart illustrating a method of photographing a visualization sheet by an electronic device according to the present invention, and FIG. It is a conceptual diagram showing an example of a user interface to

As shown in FIG. 7 , a visualization sheet 700 may be used in a meeting. In this case, the electronic device 100 according to the present invention may photograph the visualization form 700 and generate the meeting minutes by using the captured visualization form 700 .

More specifically, referring to FIG. 8 , the controller 180 of the electronic device 100 according to the present invention may capture an image in real time while rotating the main body at a point in time when the talker image is not captured ( S810 ). The controller 180 may determine whether to use the visualization form based on the image captured in real time (S820). For example, the controller 180 may determine whether a visualization sheet exists through image analysis of an image captured in real time.

Alternatively, the controller 180 may record an image in real time while rotating the main body in order to capture an image of the speaker. Similarly, the controller 180 may determine whether a visualization sheet exists from an image captured in real time.

In addition, the controller 180 may calculate the position of the visualization sheet based on the captured image. The controller 180 may store the calculated position of the visualization ticket in the memory 170 and take an image of the visualization ticket at every preset period.

On the other hand, when there is a visualization ticket, the control unit 180 may determine whether it is possible to photograph the visualization ticket (S830). The photographing possible state of the visualization sheet may be a standby state in which the speaker's voice information is not received through the microphone 152 . Alternatively, the photographing possible state of the visualization sheet may be a state in which an image of the speaker corresponding to the voice information currently received through the microphone 152 is pre-stored.

In this case, when it is possible to photograph the visualization form, the controller 180 may proceed with capturing the visualization form (S840). The control unit 180 may rotate the main body based on the calculated position information of the visualization form, and after the rotation of the main body, take a picture of the visualization form.

When the visualization voucher is taken, the controller 180 converts the visualization voucher and the voice information into text based on the time point at which the visualization voucher is photographed and the detection time of the voice information sensed through the microphone, and the image of the speaker can be stored together. In addition, the controller 180 may generate meeting minutes by arranging an image obtained by photographing a visualization sheet, text information converted from voice information into text, and speaker information in the order of receiving the voice information.

The controller 180 may output the generated meeting minutes on the touch screen 151 based on a user request.

As shown in (a) of FIG. 9 , the controller 180 divides the touch screen into a plurality of pieces, and outputs an image 710 obtained by photographing a visualization sheet to a first area, and a speaker image and text to a second area. Information 720 may be output. Although not shown, the controller 180 may perform scrolling based on a drag input to the area where the speaker image and text information 720 are output. In this case, when the talker information and voice information output on the touch screen 151 are changed according to scrolling, the controller 180 may output a visualization sheet linked to the changed talker information and voice information. Therefore, according to the present invention, the visualization sheet can be provided together by linking the speaker information and the voice information.

Alternatively, as shown in (b) of FIG. 9 , the controller 180 may output the visualization form 710 on the full screen, and output speaker information and text information 720 associated with the visualization form in one area. In this case, although not shown, when the visualization form is changed based on a user input, the controller 180 may output speaker information and text information linked to the changed visualization form.

In the above, a method for generating meeting minutes using the visualization table used during the meeting has been described. Hereinafter, a method for updating the visualization table will be described.

10 is a flowchart illustrating a method of updating a visualization sheet when an electronic device generates meeting minutes according to the present invention.

Referring to FIG. 10 , when there is a visualization ticket, the controller 180 may photograph the visualization ticket at every preset period based on the pre-stored location information of the visualization ticket ( S1010 ). The controller 180 may compare the captured visualization form and a pre-stored visualization form after taking an image of the visualization form (S1020). The controller 180 may update the visualization form when the captured visualization form and the pre-stored visualization form are the same (S1030).

When the captured visualization form and the pre-stored visualization form are the same, the controller 180 may update the pre-stored visualization form into a newly captured visualization form.

If the captured visualization voucher is different from the previously stored visualization voucher, the controller 180 may store a new visualization voucher in the memory 170 (S1040). In addition, the voice information received through the microphone may be stored in association with the new visualization form after taking the new visualization form.

On the other hand, when the shooting time of the talker's image and the shooting time of the visualization sheet are set to the same time point, the controller 180 may photograph the speaker's image first and then photograph the visualization sheet.

In the above, a method for updating the visualization table has been described.

The mobile terminal according to the present invention can create a meeting record stored by linking the video and audio information of the speaker who spoke during the meeting by using the rotatable main body to capture an image of the speaker in the conference room. Through this, the present invention can provide speaker information and voice information more intuitively.

In addition, the mobile terminal according to the present invention can improve the convenience of generating meeting minutes by determining whether the talker's image is normally captured, and automatically re-capturing when it is not normally captured.

In addition, the present invention can provide more accurate meeting minutes information by generating the meeting minutes using the image of the visualization sheet used in the meeting.

The present invention described above can be implemented as computer-readable code on a medium in which a program is recorded. The computer readable medium includes all kinds of recording devices in which data readable by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is also a carrier wave (eg, transmission over the Internet) that includes implementation in the form of. In addition, the computer may include the control unit 180 of the terminal. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (12)

a body formed to be rotatable;
a camera mounted on the body;
a microphone for detecting voice information of the first speaker located in the vicinity of the main body;
generating speaker information for the first speaker based on the voice information detected by the microphone, converting the detected voice information into text information, and determining the position of the first speaker's utterance;
Rotating the body so that the camera shoots the first speaker,
After the main body is rotated, the first speaker is photographed through the camera,
and a control unit for storing the captured image and the image obtained by converting the sensed voice information into text in association with each other,
The control unit is
If the photographed image is not photographed normally, the first speaker is photographed again,
When the image captured in real time through the camera during the rotation of the body includes a visualization ticket, it is determined whether the visualization ticket can be photographed, and when the visualization ticket can be photographed, the body is rotated to photograph the visualization ticket Electronic device, characterized in that. According to claim 1,
the control unit
Before re-photographing of the first speaker, when voice information of a second speaker other than the first speaker is received, the location of the first speaker's utterance is stored in a memory;
The electronic device of claim 1, wherein the main body is rotated to take a picture of the second speaker. 3. The method of claim 2,
the control unit
The electronic device of claim 1, wherein after the second speaker is photographed through the camera, the main body is rotated based on the stored utterance position of the first speaker in order to re-photograph the first speaker. 4. The method of claim 3,
the control unit
and storing the re-photographed image of the first speaker in association with the voice information of the first speaker. delete delete According to claim 1,
the control unit
The electronic device of claim 1, wherein when the voice information is not received through the microphone or an image of the speaker corresponding to the voice information received through the microphone has already been captured, it is determined that the visualization of the document can be captured. According to claim 1,
the control unit
The electronic device, characterized in that for calculating the location information of the visualization sheet, based on the photographing time information of the photographed image. delete delete According to claim 1,
the control unit
The electronic device of claim 1, wherein the capturing time of the visualization sheet, the receiving time of the voice information, and the image of the first speaker are stored together. a body formed to be rotatable;
a camera mounted on the body;
a microphone for detecting voice information of the first speaker located in the vicinity of the main body;
determining the position of the first speaker's utterance by using the voice information sensed by the microphone,
Rotating the body so that the camera shoots the first speaker,
After the main body is rotated, the first speaker is photographed through the camera,
A control unit for storing the captured image and the sensed voice information in association with each other,
the control unit
If the photographed image is not normally captured, the first speaker is re-photographed, but when voice information of a second speaker other than the first speaker is received before the re-photography of the first speaker, the first speaker The electronic device of claim 1, wherein the utterance position is stored in a memory and the main body is rotated to capture the second speaker.

Images