JP2000293279A - Multimedia data acquisition device and recording medium where program thereof is recorded - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve operation problems such as the need to manually operate a pointing device, etc., and to improve the operator convenience by controlling a connection between a voice device and a voice data storing process part or a voice recognizing part according to a data process event. SOLUTION: A voice recognizing process part 5 recognizes voice data obtained by a voice acquisition part 3 and sends an event to an event process part 10, etc. The event process part 10 performs a process corresponding to the event, etc., sent from a pointing device part 14 and the voice recognizing process part 5. A voice device connection control process part 6 controls connections on a software-interface between the voice acquisition part 3 and one of a voice data storing process part 4, a stored voice data reproducing process part 9 and the voice recognizing process part 5 through a software interface 106.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

TECHNICAL FIELD The present invention relates to infrastructure, production,
Multimedia data acquisition device that uses various multimedia information devices such as video, audio, and data input / output that are used by maintenance workers and outdoor workers related to distribution or various services, and records recording the programs thereof. It is about media.

[0002]

2. Description of the Related Art Multimedia for supporting work using various multimedia information devices such as video, audio and data input / output used by maintenance workers and outdoor workers related to infrastructure, production, distribution or various services. As a conventional technique related to data acquisition, for example, image data input via a camera or an image processing device, and process data input via a process controller described in JP-A-9-16365 are separately input. And a method of synchronizing and outputting the data to which the text data thus obtained is added with the voice data obtained through a speaker or a voice processing device.

[0003]

However, in such a method, if the user wants to acquire data even while working with both hands occupied, he or she must operate a pointing device or the like of the multimedia data acquiring apparatus. In some cases, it is necessary to temporarily stop the original work and manually operate the multimedia data acquisition device, which poses a problem in operability.

[0004] Even if a speech recognition processing application is added to such a multimedia data acquisition device, since the recording and reproduction are performed at any time by the multimedia data acquisition device, competition occurs in the audio device, and eventually the contention occurs. There is an operational problem that the pointing device or the like of the multimedia data acquisition device must be manually operated in some cases.

SUMMARY OF THE INVENTION In view of the above-mentioned conventional problems, the present invention has a problem of operability such as a need to manually operate a pointing device or the like of a multimedia data acquisition apparatus when acquiring a series of multimedia data during operation. And to improve the convenience of the workers.

[0006]

The present invention is achieved by employing the following novel characteristic means. That is,
In a computer system, an image acquisition unit, an image data display storage processing unit that displays and saves image data acquired by the image acquisition unit, an audio acquisition unit, and audio data that saves audio data acquired by the audio acquisition unit. A storage processing unit, a voice recognition processing unit that recognizes voice data acquired by the voice acquisition unit and generates a data processing event, and a connection or voice recognition between a voice device including the voice acquisition unit and the voice data storage processing unit. An audio device connection control processing unit that controls connection with the processing unit based on a data processing event is provided.

According to the present invention, since the above-described means are taken, even if a speech recognition processing application is added to the multimedia data acquisition device, a conflict occurs in the speech device, and eventually the pointing device of the multimedia data acquisition device. It is possible to solve an operational problem that an operation such as the above has to be used in some cases.

[0008] Further, since a related file creation processing section for creating an association file between the stored image data and audio data is provided, the subsequent editing process and the like for the acquired data can be made more efficient, and the operability can be improved. .

A stored image data display processing section for displaying the stored image data, a stored audio data reproduction processing section for reproducing the corresponding stored audio data, an audio device and an audio data storage processing section. By providing an audio device connection control processing unit that controls and processes the connection with the connected or stored audio data reproduction processing unit or the connection with the voice recognition processing unit based on the data processing event, it is possible to solve the operational problem, Thereby, the operability such as the subsequent confirmation work on the acquired data can be improved.

[0010] In addition, a computer having a data processing unit, a pointing device, a display unit, a voice input / output unit, and a power supply unit, which can be worn by an operator, is used.
It is necessary to provide a detachable image acquisition unit for clothes worn by workers, such as helmets and work clothes, so that they can work outdoors or use both hands to operate devices other than the multimedia data acquisition device. The operability can be improved even in the case where there is any.

As described above, when acquiring a series of multimedia data during a work, there is a problem in operability that a manual operation of a pointing device or the like of the multimedia data acquiring apparatus is sometimes required. There is a difference from the prior art that can solve the problem and improve the convenience for the operator.

[0012]

Embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a configuration diagram showing an embodiment of a media acquisition function unit in a multimedia acquisition device of the present invention.

A is a computer to which the present invention is applied. B is a data processing unit. Reference numeral 1 denotes an image acquisition unit having a software interface 100.

Reference numeral 2 denotes an image data display storage processing unit,
Software interface 101 connectable to software interface 100, software interface
A software interface 107 that can be connected to the software interface 106, and a software interface 111 that can be connected to the software interface 112.
Is displayed on the display unit 12 via the software interface 111 or stored in the storage unit 11.

Reference numeral 3 denotes a voice acquisition unit, which has a software interface 102 shared with the voice output unit 13.

Reference numeral 4 denotes an audio data storage processing unit, which is a software interface 103 connectable to the software interface 102, a software interface 106
A software interface 108 that can be connected to the software interface 108, and stores the audio data acquired by the audio acquisition unit 3 in response to an event input to the software interface 108. The connection between the interfaces 102 and 103 is controlled by an event input to the software interface 108.

A voice recognition processing unit 5 includes a software interface 105 connectable to the software interface 102 and a software interface 110 connectable to the software interface 106, and recognizes voice data acquired by the voice acquisition unit 3. Process and transmit the event to the event processing unit 10 and the like. Further, the connection between the interfaces 102 and 105 is controlled by an event input to the software interface 110.

An event processing unit 10 includes an audio device connection control processing unit 6, a file creation processing unit 7, a stored image data display processing unit 8, and a software interface 106, and has a pointing device unit 14 and a voice recognition processing unit. The processing corresponding to the event or the like transmitted from the unit 5 is performed.

The voice device connection control processing unit 6 is provided on a software interface between the voice acquisition unit 3, the voice data storage processing unit 4, the stored voice data reproduction processing unit 9, and the voice recognition processing unit 5 via the software interface 102. Is controlled through the software interface 106 based on the data processing event.
The file creation processing unit 7 creates an association file between the stored image data and audio data. The stored image data display processing unit 8 displays the stored image data.

Reference numeral 9 denotes a stored audio data reproduction processing unit, which is a software interface 104 connectable to the software interface 102 and a software interface 109 connectable to the software interface 106.
And reproduces the stored audio data in response to an event input to the software interface 109. Further, the connection between the interfaces 102 and 104 is controlled by an event input to the software interface 109.

Reference numeral 11 denotes a storage unit, and 12 denotes a display unit. An audio output unit 13 includes a software interface 102 shared with the audio acquisition unit 3.

Reference numeral 14 denotes a pointing device unit, and 15 denotes a power supply unit.

The connection state of the software interface in the initial state after startup or in the standby state where no processing is performed is as follows.

The software interface 106 is a multi software interface 107, 108,
It is connected to 109 and 110 and thereafter connected as it is.

The software interface 100 is directly connected to the software interface 101 at the time of start-up, and is thereafter connected as it is.

The software interface 112 is directly connected to the software interface 111 at the time of start-up, and is thereafter connected as it is.

The software interface 102 between the voice acquisition unit and the voice output unit is connected to the software interface 105 to the voice recognition processing unit at the time of startup, and thereafter, the connection destination is the software interface 103, 104 according to the processing status for voice. Alternatively, when the voice processing is completed and the apparatus enters a standby state, the apparatus is connected to the software interface 105 and is in a state of waiting for an operation instruction by voice.

The device-compatible software interfaces 100, 102, and 112 are interfaces provided by a Windows OS commercially available from Microsoft Corporation.

The acquisition of image and audio data is performed in the following procedure. The operation of the entire software of the multimedia data acquisition apparatus is operated based on an event transmitted from the pointing device unit 14 or the voice recognition processing unit 5. Here, first, details will be described with reference to an example in which the speech recognition processing unit 5 is connected to the speech acquisition unit 3 via the software interface 102 and the software interface 105.

(Procedure 1) When the operator speaks to obtain multimedia data, the voice recognition processing unit 5 converts the voice data obtained from the voice obtaining unit 3 through the software interface 102 and the software interface 105 into a voice. After processing, it recognizes it as a multimedia acquisition instruction and transmits a multimedia data acquisition start event to the event processing unit 10.

(Procedure 2) Next, the event processing unit 10 transmits an event to the image data display / storage processing unit 2 and causes the storage unit 11 to store the image. Thereafter, after a screen for prompting the user to determine whether or not to acquire the voice data is displayed, when a voice corresponding to the voice data acquisition instruction is made, the voice data from the voice acquisition unit 3 is processed, and the voice recognition processing is performed. The unit 5 recognizes the instruction to acquire audio data, and transmits an audio data acquisition start event to the event processing unit 10.

(Procedure 3) Next, the audio device connection control processing unit 6 in the event processing unit 10
An event for disconnecting the software interface 102 and the software interface 105 is transmitted to the voice recognition processing unit 5 via the software interface 106.

(Procedure 4) Next, the audio device connection control processing section 6 in the event processing section 10 waits for the disconnection between the software interface 102 and the software interface 105 of the audio recognition processing section 5 and then stores the audio data. An event for prompting the processing section 4 to connect the software interface 102 and the software interface 103 is transmitted via the software interface 106. According to this event (Procedure 5) Next, the audio data storage processing unit 4 stores the audio data acquired by the audio acquisition unit 3 according to a previously stored or set time.

(Procedure 6) Next, the audio device connection control processing section 6 in the event processing section 10 waits until the audio data storage processing section 4 releases the connection between the software interface 102 and the software interface 103. Software interface for the voice recognition processing unit 5
An event prompting connection between the software interface 102 and the software interface 105 is transmitted via the software interface 106. At this time, the file creation processing unit 7 in the event processing unit 10 creates an association file between the stored image data and audio data.

With this series of operations, it is possible to solve the operational problem that it is necessary to manually operate the pointing device and the like of the multimedia data acquisition device when acquiring a series of multimedia data during work.

Display and reproduction of image and audio data are performed in the following procedure.

(Step 7) When the operator makes an utterance corresponding to the display of the stored data, the voice acquisition unit 3 sends the software interface 102 and the software interface
The voice recognition processing unit 5 processes the voice data acquired through the step 105, and recognizes the stored data display instruction as an instruction, and transmits a stored data display start event to the event processing unit 10.

(Procedure 8) Next, the stored image data display processing section 8 in the event processing section 10 reads out the corresponding data from the storage section 11 and displays it. At the same time, a screen for prompting the user to determine whether or not to reproduce the audio data is displayed. At this time, when an utterance corresponding to the instruction to acquire the audio data is made, the audio data from the audio acquisition unit 3 is processed, and the audio recognition processing is performed. The unit 5 recognizes the instruction as a sound data reproduction instruction and transmits a sound data reproduction start event to the event processing unit 10.

(Procedure 9) Next, the audio device connection control processing section 6 in the event processing section 10 sends an event that causes the voice recognition processing section 5 to release the connection between the software interface 102 and the software interface 105. 5 via the software interface 106.

(Procedure 10) Next, the audio device connection control processing section 6 in the event processing section 10 waits for the disconnection between the software interface 102 and the software interface 105 of the voice recognition processing section 5, and then stores the stored audio data. An event for prompting the data reproduction processing unit 9 to connect the software interface 102 and the software interface 104 is transmitted via the software interface 106.

(Procedure 11) Next, in accordance with this event, the stored audio data reproduction processing section 9 reproduces the corresponding audio data.

(Procedure 12) Next, the audio device connection control processing unit 6 in the event processing unit 10 waits for the disconnection between the software interface 102 and the software interface 104, An event prompting connection between the software interface 102 and the software interface 105 is transmitted via the software interface 106.

By this series of operations, it is necessary to manually operate a pointing device or the like of the multimedia data acquisition apparatus when reproducing a series of multimedia data during operation. Solvable.

FIG. 2 shows an example of a hardware configuration of a system to which the multimedia acquisition apparatus of the present invention is applied. The system of this embodiment provides a comfortable work support environment utilizing multimedia and wearable PC (wearable personal computer) technology to field workers who are active in fields such as communication infrastructure construction, manufacturing, logistics management, service, and medical support. System. The system of the present embodiment is shown as being constructed assuming operations such as telephone line maintenance and failure recovery.

The components include a wearable PC worn by a field worker and an in-vehicle server mounted on a work vehicle.
It comprises a printer, a wireless LAN for communicating between the wearable PC and the in-vehicle server, and a GPS for acquiring position information.

The wearable PC is characterized in that it is easy to browse and search an electronic manual, and that the operator enjoys a hands-free environment. Due to this feature, research and development has been actively conducted in recent years to apply it to work that requires a large amount of paper-based manuals, such as maintenance of airplanes and ships, and that requires hands-free operation to improve work efficiency. Is becoming

In the system of this embodiment, a wearable PC is equipped with a CCD camera for photographing an image, a bone conduction microphone for voice input, a PCMCIA video capture card connected to the CCD camera, and a PCMCIA wireless LAN slave unit. I have. It is possible to give a voice command to a wearable PC using a bone conduction microphone by voice recognition technology, and almost no peripheral devices such as a keyboard and a mouse are required for input work.

Information is transmitted and received between the wearable PC and the in-vehicle server via a wireless LAN. The in-vehicle server has a database of equipment data, construction records, work manuals, etc., and by detecting position information by GPS and linking it to the equipment data, it is necessary for infrastructure maintenance such as attached equipment at the current position and past construction records. Information can be searched. The field worker receives the information on the wearable PC via the wireless LAN by voice command, and can refer to the information on the head mounted display worn by the wearer.

It is very important to record work in outdoor work such as infrastructure maintenance in order to refer to the construction history and check incidental facilities. In the system of the present embodiment, as described above, the CC
The camera is equipped with a D-camera, captures images of the work site by voice commands, and records them as image files on a hard disk on a wearable PC. After the image acquisition, a voice memo corresponding to the captured image is also recorded as a voice memo file on hardware on the wearable PC by the bone conduction microphone.

File names are automatically given to the acquired image files and audio memo files. At this time, the image file name and the audio file name are added to the image / audio correspondence file indicating which audio file corresponds to which image.

In the image / sound correspondence file, the file names of the corresponding image data and sound data are written out in pairs on one line. If audio data was not added to the acquired image, the audio file name should be written where
A symbol that does not exist is written out.

Each image file, audio file, and image / audio file are transferred to the on-board server via the wireless LAN. These files transferred from the wearable PC are stored in a directory of the vehicle server.

The configuration shown in FIG. 1 is mounted in the wearable PC shown in FIG. That is, a wearable PC
Has a work support information search and provision function unit in addition to the media acquisition function unit shown in FIG. The work support information search and provision function unit performs a configuration, which is not clearly shown in FIG. 1 but is described with reference to FIG. Provides a free environment.

Although the multimedia data acquiring apparatus has been described above, (i) Procedures 1 to 6 and (ii) for acquiring image and audio data and display of image and audio data described with reference to FIG. In steps 7 to 12 for reproduction, a program in which the data processing device describes the corresponding processing can be stored on a recording medium, and read and used. Therefore, the present invention includes in its technical scope a recording medium on which such a program is recorded.

[0055]

As described above, according to the present invention, it is necessary to manually operate the pointing device and the like of the multimedia data acquiring apparatus when acquiring a series of multimedia data during operation. Thus, a multimedia data acquisition device that can solve the above operational problem and improves the convenience of the operator is realized.

[Brief description of the drawings]

FIG. 1 is a configuration diagram showing an embodiment of a media acquisition function unit in a multimedia acquisition device of the present invention.

FIG. 2 shows a hardware configuration example of a system to which the multimedia acquisition device of the present invention is applied.

[Explanation of symbols]

A computer B data processing unit 1 image acquisition unit 2 image data display storage processing unit 3 audio acquisition unit 4 audio data storage processing unit 5 audio recognition processing unit 6 audio device connection control processing unit 7 file creation processing unit 8 stored image data display Processing unit 9 Saved audio data reproduction processing unit 10 Event processing unit 11 Storage unit 12 Display unit 13 Audio output unit 14 Pointing device unit 15 Power supply unit 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 112
Software interface

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5D015 KK01 5E501 AA30 AC15 AC25 BA05 CA02 CA08 CB20 DA16 EA21 FA13 FA14 FA32

Claims (6)

[Claims] An image acquisition unit, an image data display / save processing unit that displays and saves image data acquired by the image acquisition unit, an audio acquisition unit, and audio data acquired by the audio acquisition unit in the computer system. A voice data storage processing unit that stores the voice data, and performs a recognition process on the voice data acquired by the voice acquisition unit.
A voice recognition processing unit that generates a data processing event, and a voice device connection control process that controls connection between a voice device including a voice acquisition unit and a voice data storage processing unit or connection with the voice recognition processing unit based on the data processing event A multimedia data acquisition device, comprising: 2. The multimedia data acquisition device according to claim 1, further comprising an associated file creation processing unit that creates an association file between the stored image data and audio data. 3. A stored image data display processing unit for displaying stored image data, a stored audio data reproduction processing unit for reproducing corresponding stored audio data, and an audio device according to claim 1 or 2. An audio device connection control processing unit that controls and processes the connection with the audio data storage processing unit, the connection with the stored audio data reproduction processing unit, or the connection with the voice recognition processing unit based on the data processing event. Multimedia data acquisition device. 4. A computer having a data processing unit, a pointing device, a display unit, a voice input / output unit and / or an image acquisition unit, a storage unit, and a power supply unit on a wearable personal computer which can be worn and detached by an operator. The multimedia data acquisition device according to claim 1, 2 or 3, wherein 5. A processing step of recognizing audio data instructing to acquire multimedia data and transmitting a multimedia data acquisition start event to an event processing unit, and acquiring audio data in response to the event. Display a screen prompting the user to determine whether or not the voice data acquisition instruction is received and recognized by the recognition processing unit. Based on the result, the voice data acquisition start event is transmitted to the event processing unit. A transmitting step; a processing step for switching a supply destination of audio data to be subsequently emitted from the audio device to store the audio data in the storage means; And a processing step of returning the supply destination of the audio data from the device to the original recognition processing unit side in order. A recording medium on which a data acquisition program is recorded. 6. A processing step of recognizing audio data instructing display of multimedia data and transmitting a stored data display start event to an event processing unit, reading the relevant data from a storage unit and displaying the data, At the same time, a screen is displayed that prompts the user to determine whether or not to reproduce the audio data, and the utterance corresponding to the instruction to acquire the audio data is recognized by the recognition processing unit. A processing step of transmitting the stored audio data to the audio device so as to supply the stored audio data to the audio device, and causing the audio device to reproduce the stored audio data. Processing to return the supply destination of the audio data from the audio device to the recognition processing unit after the reproduction of the already processed audio data is completed A multimedia data acquisition program for enabling the steps to be repeatedly executed in order.