JP2018004795A - Behavior observation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible for an observer to record and reproduce behaviors of a participant by a simple operation, as observing the behavior of the participant.SOLUTION: A participant on a scene of an observation object is displayed by an icon 24, and the movement of the participant is displayed by a trajectory 28. Record images 50 obtained by photographing noteworthy behaviors of the participant, and thumbnails are displayed by being aligned in time series. When moving the icon 24 according to the movement of the participant, the trajectory 28 is stored in a storage device 32 together with time data 52. A mark 48 is displayed at an arbitrary attention point 46 in an observation screen 20 or the record image 50 to obtain an input of attribute data 56 including a meaning of the mark 48. In the attribute data 56, weighted data 60 indicative of a degree of importance of the mark 48, character data 62 indicative of the meaning of the mark 48 or voice data 64 is included.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a behavior observation apparatus that allows an observer to record and reproduce important portions with a simple operation while observing the behavior of a participant.

  It is often done to observe what a person who is present in a scene takes and to get something from the result and to learn. For example, such a purpose can be achieved by analyzing a moving image that has been taken with a photographing device. For example, a technique for analyzing whether or not a nurse can take an appropriate action in a medical field has been introduced (Patent Document 1).

Japanese Patent Laying-Open No. 2015-219247

  If an expert's comment is added to the image recorded and reproduced, the behavior of the participant to be observed can be used as a learning material. However, there was a problem that it took a lot of time to create such teaching materials. The present invention has been made paying attention to the above points, and the observer can record while observing the behavior of the participant with a simple operation, and can reproduce the important part with various information added thereto. A behavior observation apparatus is provided.

  The following configurations are means for solving the above-described problems.

<Configuration 1>
An observation screen that displays the participants on the scene to be observed with icons and displays the movement of the participants by a trajectory,
An image display unit that displays recorded images of the behavior of the participants or thumbnails thereof in chronological order,
In the recording mode, when the icon of the corresponding participant is moved according to the movement of the participant, a locus recording means for storing the locus in a storage device together with time data;
A data editing means for displaying a mark at an arbitrary point of interest in the observation screen or in the recorded image, obtaining input of attribute data including the meaning of the mark, and editing the attribute data and storing it in a storage device; A behavior observation apparatus comprising:

<Configuration 2>
2. The behavior observation apparatus according to Configuration 1, wherein the attribute data of the mark is the transparency of the mark indicating the importance level of interest.

<Configuration 3>
The behavior observation apparatus according to Configuration 1, wherein the attribute data of the mark is coloring of a mark indicating the degree of importance of attention.

<Configuration 4>
The behavior observation apparatus according to Configuration 1, wherein the attribute data of the mark is a character string or a voice indicating a noticed meaning.

<Configuration 5>
The behavior observation apparatus according to Configuration 1, wherein the attribute data of the mark is voice generated by a participant at a mark display location on the observation screen.

<Configuration 6>
In the playback mode, an observation image playback means is provided for reading the participant's icon trajectory and time data stored in the storage device and playing back the motion of the corresponding participant on the observation screen as time passes. The behavior observation apparatus according to Configuration 1, characterized by:

<Configuration 7>
In the reproduction mode, when the observation image reproduction means moves the participant's icon displayed on the observation screen on the trajectory displayed on the observation screen, the state of all icons and symbols on the observation screen is displayed. The behavior observation apparatus according to Configuration 6, wherein the behavior is reproduced and displayed in a corresponding time state.

<Configuration 8>
A behavior observation program for causing a computer to function as each means described in Configuration 1 or 6.

<Configuration 9>
A computer-readable recording medium on which the behavior observation program according to Configuration 8 is recorded.

<Effect of Configuration 1>
The behavior of the participant displayed with the icon is expressed by moving the icon. Participants' important behavior is to take a picture and save a recorded image. And if the mark which can show weighting is displayed on an observation screen or a recorded image, the main point of action can be recorded. Furthermore, if the attribute data that expresses the meaning of the mark with characters or voices is stored, it is possible to extract, analyze, and learn the key points of the participant's series of actions. Moreover, these data can be easily created with a tablet or the like.
<Effect of Configuration 2>
If you specify the transparency of the mark, you can easily express the importance of the place or action you noticed with special visual effects.
<Effect of Configuration 3>
It is also possible to express the importance of attention with the color of the mark.
<Effect of Configuration 4>
As the attribute data of the mark, the intention, meaning, caution, etc. of the mark made by the observer can be expressed in text or voice and stored in association with the mark.
<Effect of Configuration 5>
If the participant utters the voice along with the action, the intention of the participant's action can be recorded at the same time if the place where the sound is generated is marked and recorded.
<Effect of Configuration 6>
Focusing on the behavior of a certain participant and reproducing the video of the corresponding icon as time elapses, it becomes easy to objectively observe the behavior.
<Effect of Configuration 7>
By dragging an icon of a participant on the observation screen and controlling the observation screen to automatically change to the corresponding time state, the behavior analysis of all participants can be performed in detail.

It is explanatory drawing which shows the specific example of the action observation apparatus 12 of this invention. It is explanatory drawing of the scene which uses the action observation apparatus. This is the screen immediately after starting the application. This is a screen for creating a new workshop. This is the workshop loading screen. It is an explanatory screen of the icon of a workshop screen. It is an example of a recording mode workshop screen. It is an example of a workshop screen in browsing mode. It is an example of a workshop screen in a recording mode for operating a recorded image. It is an example of a screen of a recorded image when marking is performed. It is a screen of a voice interface.

FIG. 2 is an explanatory view showing a utilization form of the present invention.
Although the specific description of this figure will be given later, the apparatus of the present invention is generally used as follows. The present invention realizes the behavior observation apparatus 12 by an application that can express the metacognition of the participant 16 in the workshop and enhance the educational effect. Meta means “higher” and supports the recognition of what you know at a higher level. Participant 16 provides an overview of himself / herself, understands that he / she recognizes it, and assists in deepening his / her recognition.

  For example, while the observer 14 uses the tablet 18 to observe the behavior of the participant 16, the behavior trajectory is recorded, and the point of interest of the observer 14 and the degree of importance thereof are visualized. The point that the observer 14 is interested in is recorded using a photograph, a memo, voice, a flow line, and the like. It is expressed and recorded by intuitive weighting which part of the scene the observer 14 pays attention to, and how important it is.

  Playing the record makes it easier to remember the details of the day. It becomes easy to look back on the behavior at that time, and it becomes easier for the observer 14 and the participant 16 to share the points. If the movement of the participant 16 is recorded by dragging with a finger, for example, and an annotation is added thereto, the deformed movement can be recorded unlike the recording of the movie.

  As a result, the observer 14 presents the result of the action to the participant 16 and looks back on the action of each other, or the reproduction is performed only by the participants 16 for analysis. By looking back on what was obtained and what should have been done in a scene aimed at some educational effect, it can be used for education for dealing with various scenes by accurately telling it. Hereinafter, embodiments of the present invention will be described in detail for each example.

(Structure of behavior observation device)
FIG. 1 is an explanatory view showing a specific example of the behavior observation apparatus 12 of the present invention.
The behavior observation apparatus 12 of the present invention is used for an observer to record the behavior while observing the participant. This behavior observation device 12 can be realized by a known tablet 18, for example. FIG. 1 shows a block diagram of the display 19 of the tablet 18 and the arithmetic processing device 30 and the storage device 32 accommodated therein.

  The display 19 of the tablet 18 is divided into an observation screen 20 and an image display unit 22. The scene to be observed is displayed on the observation screen 20 with an icon 24 and a symbol 26. One icon 24 is prepared for each participant. For example, in the case of group action, all the groups may be indicated by one icon 24. In the following description, this icon is called a group icon. The design should be distinguishable from the individual.

  If furniture and various tools are arranged in the scene to be observed, they are displayed as symbols 26 on the observation screen 20. When the participant moves, the observer drags the icon 24 of the corresponding participant with a finger. On the observation screen 20, the movement of the participant is displayed as a locus 28 as shown in the figure.

  In addition, the observer captures a notable state of the participant's behavior with a camera provided on the tablet 18. The captured recorded images 50 and their thumbnails are displayed in time series in the image display unit 22. Furthermore, when the participant performs a notable action while moving, the mark 48 can be recorded at a corresponding place on the locus 28. The menu button 23 is a button for displaying a menu for performing various operations, and the reproduction button 51 is for reproducing the recording with time.

  The arithmetic processing unit 30 of the behavior observation apparatus 12 is provided with a trajectory recording unit 34, a data editing unit 36, an observation image reproducing unit 38, a weighting button 40, a character input button 42, and a voice input button 44. These are computer programs that implement the functions described below. The storage device 32 stores a mark 48, a recorded image 50, and trajectory data 54.

  The mark 48 is provided with attribute data 56. The attribute data 56 includes weighting data 60, character data 62, and voice data 64. Time data 52 indicating the time when the data is acquired is added to the recorded image 50 and the trajectory data 54, respectively. In addition, the image data of the icon 24 and the symbol 26 is stored. However, since these data structures are known, illustration is omitted.

  The trajectory recording means 34 has a function of storing the trajectory 28 together with the time data 52 in the storage device 32 when the icon 24 is moved according to the movement of the participant. The data editing means 36 displays a mark 48 at an arbitrary point of interest in the observation screen 20 or the recorded image 50, asks for input of attribute data 56 including the meaning of the mark 48, and edits the attribute data 56. And has a function of storing in the storage device 32.

  The weighting button 40 has a function of selecting the display size and coloring of the mark indicating the importance level of interest and adding the weighting data 60 to the attribute data 56. The character input button 42 has a function of adding character data 62 to the attribute data 56 by inputting characters of the intention, meaning, cautions, etc. marked by the observer. The voice input button 44 has a function of recording voices such as intentions, meanings, and caution points marked by the observer, and adding voice data 64 to the attribute data 56.

  When the participant utters a voice together with an action at the mark 48 display location on the observation screen 20, when the voice input button 44 is operated, the generated voice can be added to the attribute data 56 of the mark 48. This is useful for recording participants' intentions for action.

  The observation image reproduction means 38 has a function of reading the trajectory data 54 and the time data 52 of the participant's icon 24 stored in the storage device 32 and reproducing the behavior of the corresponding participant on the observation screen 20 over time. Have. The observation image reproducing means 38 moves all the icons and symbols on the observation screen 20 at the corresponding time when the participant icon 24 displayed on the observation screen 20 is moved on the locus 28. It also has a function to reproduce and display the state.

(Simulation using behavior observation device)
FIG. 2 is an explanatory diagram of a scene in which the behavior observation apparatus 12 is used. This will be described with reference to FIG.
This FIG. 2 simulates the behavior of the mother 66 and the nurse 68 and the doctor 70 rushed by the nurse call when the child 74 laid down in the hospital bed 72 falls from the bed 72 to learn. Is for. The child 74 may be a dummy, and the mother 66 and the nurse 68 are new nurses and the like.

  In this scene, the observer 14 observes and records on the tablet 18 the actions of the participant of the mother 66, the participant of the nurse 68, and the participant of the doctor 70. The observer 14 uses the tablet 18 to photograph a scene to be noticed by the action of the participant 16. The recorded image 50 or its thumbnail is displayed in time series on the image display unit 22.

  A scene to be observed is displayed as an observation screen 20 on the tablet 18. Each participant 16 is displayed with an icon 24 that can be distinguished. If there is furniture, equipment, etc., the icon 24 and various symbols 26 are displayed. When the participant 16 moves, the observer 14 drags the corresponding icon 24 on the observation screen 20. A linear locus 28 is displayed on the trace where the icon 24 has moved.

  The locus recording means 34 stores this locus 28 together with the time data 52 in the storage device 32. Thereby, the observation image reproduction means 38 can reproduce the motion of the corresponding participant 16 on the observation screen 20 according to the passage of time.

  When the observer 14 is interested in the action of the participant 16, the observer 14 performs an operation for displaying the mark 48 on the observation screen 20. Attribute data 56 such as importance and annotation is added to the mark 48. When an operation for displaying the mark 48 is performed, the size and color of the mark 48 can be selected. For example, the colors are red, yellow, and green in order of importance. The shape of the mark 48 is a circle, and the lower the transparency, the higher the importance.

  Furthermore, the mark 48 can be designated and a comment by characters can be input. Moreover, the comment by the voice of the observer 14 can be input. The attention data editing means 36 stores the edited data in the storage device 32 as attention data.

  Note that the sound generated when the participant 16 takes some action may be used as attribute data 56 of the mark 48 attached to the icon 24 of the participant 16 at that time as attention data.

  In this way, both the observer 14 and the participant 16 can record what they are interested in. As a result, it is possible to look back on the behavior and point out what should be done at what point in time and increase the educational effect.

  If the behavior observation apparatus 12 according to the present invention is used for “nursing simulation education” as in this embodiment, it is possible to learn “corresponding to a bed 72 fall accident in a child ward 74” by simulation.

  In this scene, the mother 66 runs up to the child 74 who has fallen from the bed 72, holds it up, and makes a nurse call. If there is something that was thought on the spot or something that was actually desired, the voice data 64 is recorded together with the mark 48 as attention data. If you declare it verbally, you do not need to act.

  For example, the nurse 68 who rushed to the scene may move to the next action by saying “Measure blood pressure here”. In response to the report of the mother 66, the child 74 is returned to the bed 72. At this time, the mother 66 is requested to lower the fence of the bed 72 and further instructed to press the wound with a towel or the like. In this state, confirm the state of consciousness of the child 74 who is a patient and confirm the response to stimulation, eye movement, nausea and vomiting, confirmation of the patient's injury status, laceration due to bruise, wound size, depth, blood loss, etc. To do.

  These confirmations are reported when the doctor 70 arrives. This series of behavioral simulations is conducted to analyze the behaviors in detail and learn.

  An attention point 46 is displayed by a mark 48, and further, clicking on the mark 48, the attention point by a comment or voice can be reproduced, and effective learning can be performed. Moreover, since the icons 24 of the participants 16 on the observation screen 20 can be operated to easily record various situations together with the marks 48, it is easy to produce records, and messages from the observer 14 to the participants 16 can be easily recorded. Can be conveyed efficiently.

(Example tablet screen)
FIG. 3 and subsequent figures illustrate photographs of the operation screen of the tablet. FIG. 3 shows a screen immediately after starting the application. The processing when each button is pressed is explained on the right side. Selecting the third button from the top will open the new workshop creation screen. Also, when the 4th to 6th buttons from the top are selected, it is possible to move to the workshop browsing screen.

  Fig. 4 shows the screen for creating a new workshop. This is a screen that opens when a button labeled (NEW) is pressed on the screen of FIG. Enter the workshop name and memo and press the button at the bottom to create the workshop data and move to the editing or browsing screen.

  FIG. 5 is a workshop loading screen. You can load past workshops. You can delete this screen or edit the workshop information by long-pressing the circular button. Also, by selecting a button, the screen moves to a workshop screen for recording and browsing the data of the corresponding workshop.

  FIG. 6 is a screen for explaining icons on the workshop screen. This screen is a recording mode for recording. The upper left is a button that displays the application menu. The first row in the upper right part displays the workshop name, and the second row displays information on sections separating the workshops. Sections can be created and moved by pressing the second arrow.

  Of the three figures displayed in the center of the screen, the left object is an installation object representing a chair or desk in the workshop, and is called a symbol in the above embodiment. The size can be changed freely. The right object is an object representing the participant 16, and a face photo and a name of the participant 16 can be set. In the above embodiment, it is called a participant icon.

  The triangle at the top of the icon indicates the direction in which the participant is facing, and the orientation can be set by rotating. An object without a triangle is a group object representing a group participant. These objects can be added by pressing the add button on the left side of the screen. Press the square button on the bar at the bottom of the screen to end recording and switch to viewing mode.

  FIG. 7 is an example of a workshop screen in the recording mode. If the participant object or group object is moved during recording, the footprint (trajectory) can be recorded. In addition, a circular mark can be placed at a point of concern during the movement of both objects. The size, color, and transparency of the mark can be changed, and the mark can be used for weighting the meaning of the mark.

  FIG. 8 is an example of a workshop screen in the browsing mode. This is the state of the playback mode for browsing workshop information. Information on participant objects and group objects can be viewed. The movement when the participant object and the group object are moved in the recording mode can be reproduced by the reproduction button at the lower left of the screen. You can also see the movement by operating the slide bar at the bottom of the screen. At this time, a change in the direction of the participant object can also be seen. You can switch to the recording mode by pressing the button at the bottom left of the screen.

FIG. 9 is an example of a workshop screen in a recording mode for operating a recorded image. By touching the participant object or the group object, it is possible to view an image taken in the workshop related to the object selected from the lower part of the screen. When the button on the right side of the screen is pressed, the mark is set in order from the top, the camera is started for taking a recorded image, and the associated recorded image is read.

FIG. 10 is a screen example of a recorded image when marking is performed. Press the button on the top right of the screen to return to the workshop screen. When the image is long pressed, circular marking can be performed in the recorded image. The size, color, and transparency of this mark can be changed. A text memo and a voice memo can be recorded for the mark. By pressing the red button on the right when selecting a mark, you can record a voice memo for the mark.

FIG. 11 shows a voice interface screen. When a voice memo is recorded at the mark, a button with a new speaker is displayed when the mark is selected. By selecting this button, an interface for reproducing a voice memo is displayed.

  The above apparatus can be widely used not only for simulation but also for various purposes such as objectively observing human behavior, recording and evaluating annotations, and using them for teaching materials and research materials. In addition, not only humans but also the arrangement and movement of objects can be observed to make a comprehensive evaluation. In the above example, an example has been described in which an observer uses a tablet to record, but at a place away from the observation site, the person's behavior is observed while viewing the video taken with the observation camera, and the computer The above-described recording can be performed using a finger or a mouse.

  As mentioned above, the position, orientation, and movement of the person participating in the simulation are displayed with icons, and the information indicating the comment and importance of the observer is marked by marking the position of interest or the state of interest. Can record. It is possible to record what should be noted and what should be done in the position and state by voice. It is possible to record the voice of the participant and convey the contents of the structure more clearly than when taking a video. At the same time, if you leave an image of a noteworthy state, you can understand the state more specifically.

  Compared with the case where the entire video is taken and commented, the attention points are highlighted, and if the action is confirmed by moving the icon, the participant can observe the entire object more objectively than the image taken of the scene. . Even when used as a teaching material, it becomes easy to analyze and explain by focusing on the position, orientation, movement, and surrounding objects of the participants individually.

DESCRIPTION OF SYMBOLS 12 Action observation apparatus 14 Observer 16 Participant 18 Tablet 19 Display 20 Observation screen 22 Image display part 23 Menu button 24 Icon 26 Symbol 28 Trajectory 30 Arithmetic processing apparatus 32 Storage apparatus 34 Trajectory recording means 36 Data editing means 38 Observation image reproduction means 40 Weighting button 42 Character input button 44 Voice input button 48 Mark 50 Recorded image 51 Playback button 52 Time data 54 Trajectory data 56 Attribute data 60 Weight data 62 Character data 64 Voice data 66 Mother 68 Nurse 70 Doctor 72 Bed 74 Child

Claims (9)

An observation screen that displays the participants on the scene to be observed with icons and displays the movement of the participants by a trajectory,
An image display unit that displays recorded images of the behavior of the participants or thumbnails thereof in chronological order,
In the recording mode, when the icon of the corresponding participant is moved according to the movement of the participant, a locus recording means for storing the locus in a storage device together with time data;
A data editing means for displaying a mark at an arbitrary point of interest in the observation screen or in the recorded image, obtaining input of attribute data including the meaning of the mark, and editing the attribute data and storing it in a storage device; A behavior observation apparatus comprising:   The behavior observation apparatus according to claim 1, wherein the attribute data of the mark is a transparency of the mark indicating the importance level of interest.   The behavior observation apparatus according to claim 1, wherein the attribute data of the mark is coloring of a mark indicating the degree of importance of attention.   2. The behavior observation apparatus according to claim 1, wherein the attribute data of the mark is a character string or a voice indicating a noticed meaning.   The behavior observation apparatus according to claim 1, wherein the attribute data of the mark is a voice generated by a participant at a mark display location on the observation screen.   In the playback mode, an observation image playback means is provided for reading the participant's icon trajectory and time data stored in the storage device and playing back the motion of the corresponding participant on the observation screen as time passes. The behavior observation apparatus according to claim 1.   In the reproduction mode, when the observation image reproduction means moves the participant's icon displayed on the observation screen on the trajectory displayed on the observation screen, the state of all icons and symbols on the observation screen is displayed. The behavior observation apparatus according to claim 6, wherein the behavior is reproduced and displayed in a corresponding time state.   A behavior observation program for causing a computer to function as each means according to claim 1 or 6.   A computer-readable recording medium on which the behavior observation program according to claim 8 is recorded.