JP2007048232A - Information processing device, information processing method, and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device and method efficiently carrying out posture analysis and activity analysis based upon a photographed image of a person. <P>SOLUTION: On the basis of a person image included in a photographed image frame of a camera, a frame corresponding posture determination parameter is acquired, comparison with registered parameters corresponding to a plurality of registered postures stored in a storing part is carried out, a registered posture having a registered parameter similar to the frame corresponding posture determination parameter is categorized as a posture of a person included in the photographed image frame, and analysis information is generated on the basis of the categorized registered posture. By such a composition, posture analysis processing efficiency is improved, and reduction of stored data amount is realized. Evaluation information of a posture, information coordinating a posture with its temporal occupation ratio information, and information coordinating an activity and its temporal occupation ratio information, or the like can be acquired and provided as analysis information. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an information processing apparatus, an information processing method, and a computer program for inputting an image or sound of a user and performing user posture detection, user posture and behavior analysis processing. More specifically, for example, a user's posture is determined from a captured image of the user by a camera, and processing for providing determination information, a user's posture and a behavior summary are generated based on a captured image and acquired sound for a certain period. The present invention relates to an information processing apparatus, an information processing method, and a computer program that can be provided.

  Research and development of a system for photographing a person with a camera and analyzing a user's movement and posture based on the photographed image are actively conducted. For example, Patent Document 1 discloses a system that acquires image data including a person's face and analyzes the action content of the person based on the face image area and other body image areas.

  In data input processing for information processing devices such as PCs, gestures such as user gestures and hand gestures are analyzed without using a physical device such as a mouse or keyboard, and data input is performed based on the analysis results. A method of performing is proposed. For example, Non-Patent Documents 1 to 3 disclose a technique for identifying a gesture such as a user's gesture or hand gesture and inputting a command.

  In this method, first, a person or a part of a person, for example, a hand is photographed with a video camera, and a body region is cut out from the background by image processing. Thereafter, it is specified which part of the body the part of the body region corresponds to. This requires matching with a body model.

  Next, the meaning of gesture (gesture) is estimated based on the continuous change of the body shape. This estimation is performed based on a dictionary (gesture dictionary) that associates time-series changes in body shape and their meanings.

Furthermore, as a prior art disclosing the operation identification processing based on other images, for example, there is a technique described in Patent Document 2. Japanese Patent Application Laid-Open No. 2004-228688 discloses a technique for analyzing a three-dimensional movement of a subject by photographing the subject from different directions with a plurality of cameras and analyzing the plurality of images. Japanese Patent Application Laid-Open No. 2004-228561 discloses a configuration in which the movement of the head and the direction of the line of sight are detected based on a photographed image of the person's head to detect the person's nodding motion with high accuracy.
JP 2005-78257 A JP 10-334270 A JP 2000-163196 A S. Marcel, “Gestures for multi-modal interfaces: A review”, Technical Report IDIAP-RR02-34, 2002. V. I. Pavlovic, R.M. Sharma & T. S. Huang, “Visual interpretation of hand gestures for human-computer interaction: a review”, IEEE Trans, on PAMI, 19 (7), 677-695, 1997. M.M. Turk, “Gesture recognition”, In K. Stanney (Ed.), Handbook of Virtual Environment Technology.

  However, the above-described user motion detection and posture detection processing shown in the prior art requires analysis by an extremely complicated algorithm based on the captured image data, and cannot be said to be efficient processing. There are problems such as necessity of construction of a dedicated system, high cost, and large processing load. That is, the calculation amount is enormous and the recognition accuracy is low. In addition, in order to cut out and track a person or a part of a person from a video image, the conventional method requires learning by machine learning, and there is a problem that this learning takes time.

  The present invention has been made in view of these problems, and an object of the present invention is to improve efficiency from a photographed image of a user by a camera without wearing any physical device and without tuning by a large amount of machine learning. An information processing apparatus capable of automatically determining a user's posture, providing determination information, and generating and providing a user's posture and action summary based on a captured image and acquired sound for a certain period, and An information processing method and a computer program are provided.

  According to a first aspect of the present invention, a photographing unit that photographs a target region, a posture determination processing unit that acquires posture determination information based on a person image included in a captured image of the photographing unit, and a plurality of registered postures A storage unit that stores corresponding posture determination information as registration information, the posture determination information, and a plurality of registration information stored in the storage unit, and a registered posture having registration information similar to the posture determination information Are classified as postures of persons included in the captured image, an analysis information generation unit that generates analysis information based on registered postures classified by the posture classification processing unit, and the analysis information generation unit And an information output unit that outputs the generated analysis information.

  According to this configuration, when performing posture analysis by inputting a large number of captured frame images, the person images included in the captured images are classified into a specific limited number of registered postures, and analysis based on the classification registered postures is performed. Therefore, the processing efficiency is improved and the amount of stored data is also reduced. Also, the generation of analysis information based on the registered posture and the provision of analysis information to the user can be efficiently performed with a reduced processing burden.

  Furthermore, in one embodiment of the information processing apparatus of the present invention, the storage unit has evaluation determination information for each registered posture as registered information corresponding to the plurality of registered postures, and the analysis information generation unit includes the posture classification The configuration is characterized in that evaluation information about the posture of a person included in the photographed image is generated based on the evaluation determination information set corresponding to the registered posture classified in the processing unit.

  According to this configuration, posture evaluation information can be provided as feedback information to the user, and the user can correct the posture based on the evaluation information.

  Furthermore, in an embodiment of the information processing apparatus according to the present invention, the information output unit is configured to execute an indicator display process for the evaluation information generated by the analysis information generation unit.

  According to this configuration, it is possible to check the state of one's posture with an indicator, and correct posture can be correctly executed while checking the indicator.

  Furthermore, in an embodiment of the information processing apparatus of the present invention, the information output unit is configured to execute a process of displaying the inclination information of the person's head and torso included in the captured image frame together with the person image. It is characterized by being.

  According to this configuration, it is possible to check the state of one's posture with the person image and the tilt information of the head and torso, and correct the posture can be correctly executed while checking the image.

  Furthermore, in one embodiment of the information processing apparatus of the present invention, the information output unit is configured to determine a reference line indicating a reference for the generated posture angle and a line indicating the inclination of the posture in the posture determination processing unit. It is the structure which performs the process which superimposes and displays on the image shown.

  According to this configuration, it is possible to confirm the state of one's posture with a reference line indicating the reference of the person image and the angle of the posture, and a line indicating the inclination of the posture, and correcting the posture while checking the image It can be executed correctly.

Furthermore, in one embodiment of the information processing apparatus of the present invention, the analysis information generation unit includes:
Classification registration corresponding to each photographed image classified by the posture classification processing unit based on the photographed image acquired during the measurement period, and generating posture analysis information of a person within a predetermined measurement period Based on the posture, information that associates the registered posture with temporal occupancy information of the registered posture within the measurement period is generated as analysis information.

  According to this configuration, it is possible to acquire information associating the posture with the temporal occupation rate information as analysis information, and it is possible to apply the information to, for example, health management or business management.

  Furthermore, in an embodiment of the information processing apparatus of the present invention, the analysis information generation unit is configured to generate human behavior analysis information within a predetermined measurement period, and the imaging acquired within the measurement period Based on the classification registration posture of each captured image classified by the posture classification processing unit based on the image, the behavior mode associated with the registration posture is associated with the temporal occupation rate of the behavior within the measurement period. The configuration is characterized in that information is generated as analysis information.

  According to this configuration, it is possible to acquire information in which an action is associated with its temporal occupation rate information as analysis information, and the information can be applied to, for example, productivity management or business management.

  Furthermore, in one embodiment of the information processing apparatus of the present invention, the storage unit includes, as recorded data, registration parameters corresponding to a plurality of registered postures and action information corresponding to the registered postures, and the analysis information generating unit Acquires action information corresponding to the registered posture from the storage unit based on the classification registration posture of each frame classified by the posture classification processing unit based on the frame image acquired within the measurement period, and the acquired behavior information On the basis of the behavior, it is a configuration that generates temporal occupancy information of the behavior within the measurement period.

  According to this configuration, since the behavior information is stored in association with the posture information registered in the storage unit in advance, even if there is a user-specific behavior, the behavior information is registered in association with the posture. This makes it possible to perform a unique behavior analysis corresponding to the user.

  Furthermore, in an embodiment of the information processing apparatus of the present invention, the posture determination processing unit is configured as a polygon set based on the outline of a person or a person configuration as posture determination information of a person image included in the captured image. It is a configuration for acquiring at least one of an ellipse, a center of gravity position, and an angle associated with a part, and the posture classification processing unit includes the posture determination information and a plurality of registration information stored in the storage unit. The comparison is characterized in that the comparison is executed by applying at least one of polygon information set based on the outline of the person, or an ellipse, centroid position, and angle associated with the constituent part of the person.

  According to this configuration, instead of comparing image data situations, the comparison processing is performed using polygons set based on the outline of the person, or ellipses, centroid positions, angles, etc. associated with the constituent parts of the person as parameters. As a result, the amount of data is reduced, and the post-processing columns can be increased.

  Furthermore, in one embodiment of the information processing apparatus of the present invention, the posture determination processing unit selects a part of frames from the continuous frames photographed by the photographing unit, and converts the selected frame into a human image included in the photographed image of the selected frame. Based on this, the posture determination information is obtained.

  According to this configuration, since the frame is processed intermittently, the processing load is reduced and efficient processing becomes possible.

  Furthermore, in an embodiment of the information processing apparatus of the present invention, the photographing unit is provided adjacent to a display screen on which the information output unit outputs information, and performs photographing with the upper body of the photographing subject set as a photographing range. The posture determination processing unit is configured to execute posture determination processing by dividing the head and torso of the subject to be photographed based on a person image included in a photographed image of the photographing unit. It is characterized by.

  According to this configuration, since the posture determination process is executed by dividing the head and torso of the person to be imaged, more accurate posture determination is realized.

  Furthermore, in one embodiment of the information processing apparatus of the present invention, the posture determination processing unit obtains an ellipse corresponding to each of the head and torso of the subject and a major axis and a minor axis of the ellipse, The configuration is characterized in that the process of determining the posture is executed based on the inclination of the long axis of the ellipse with respect to the vertical axis.

  According to this configuration, an ellipse corresponding to each of the subject's head and torso and the major and minor axes of the ellipse are obtained, and the posture is determined based on the inclination of the major axis to the vertical axis of each ellipse. Since the configuration is adopted, posture determination based on execution of a simplified processing algorithm is realized.

  Furthermore, in an embodiment of the information processing apparatus of the present invention, the information processing apparatus further includes a voice input unit, and the analysis information generation unit performs analysis based on voice information input from the voice input unit. It is the structure which produces | generates information, It is characterized by the above-mentioned.

  According to this configuration, since the analysis information is generated based on the voice information input from the voice input unit, it is possible to analyze an action such as during a telephone call or during a conversation.

  Furthermore, in one embodiment of the information processing apparatus of the present invention, the analysis information generation unit executes a process of generating health management information or productivity management information based on the registered postures classified by the posture classification processing unit. The information output unit is configured to output the health management information or the productivity management information generated by the analysis information generation unit.

  According to this configuration, since the health management information or the productivity management information based on the registered posture is generated and provided to the user, the user performs his / her own health management and productivity management as the information utilization range. be able to.

  Furthermore, the second aspect of the present invention is a shooting step of shooting a target area, a posture determination processing step of acquiring posture determination information based on a person image included in the shot image shot in the shooting step, The posture determination information is compared with registration information as posture determination information corresponding to a plurality of registered postures stored in the storage unit, and a registered posture having registration information similar to the posture determination information is included in the captured image. A posture classification processing step for classifying as a posture of a person to be classified, an analysis information generation step for generating analysis information based on the registered posture classified in the posture classification processing step, and output of the analysis information generated in the analysis information generation step And an information output method.

  According to this configuration, when performing posture analysis by inputting a large number of captured frame images, the person images included in the frame image are classified into a specific limited number of registered postures, and analysis based on the classification registered posture is performed. Therefore, the processing efficiency is improved and the amount of stored data is also reduced. Also, the generation of analysis information based on the registered posture and the provision of analysis information to the user can be efficiently performed with a reduced processing burden.

  Furthermore, in one embodiment of the information processing method of the present invention, the analysis information generation step includes adding the captured image to the captured image based on the evaluation determination information set corresponding to the registered posture classified in the posture classification processing step. This is a step of generating evaluation information about the posture of the person included.

  According to this configuration, posture evaluation information can be provided as feedback information to the user, and the user can correct the posture based on the evaluation information.

  Furthermore, in an embodiment of the information processing method of the present invention, the analysis information generation step is a step of generating posture analysis information of a person within a predetermined measurement period, and the imaging acquired within the measurement period Based on the classification registration posture of each captured image classified in the posture classification processing step based on the image, the information that associates the registration posture with the temporal occupancy information of the registration posture within the measurement period is analyzed information It is characterized by producing | generating as.

  According to this configuration, it is possible to acquire information associating the posture with the temporal occupation rate information as analysis information, and it is possible to apply the information to, for example, health management or business management.

  Furthermore, in an embodiment of the information processing method of the present invention, the analysis information generation step is a step of generating human behavior analysis information within a predetermined measurement period, and the imaging acquired within the measurement period Based on the classification registration posture of each captured image classified in the posture classification processing step based on the image, the behavior mode associated with the registration posture is associated with the temporal occupation rate of the behavior within the measurement period. The generated information is generated as analysis information.

  According to this configuration, it is possible to acquire information in which an action is associated with its temporal occupation rate information as analysis information, and the information can be applied to, for example, productivity management or business management.

Furthermore, in one embodiment of the information processing method of the present invention, the posture determination processing step includes:
As the posture determination information of the human image included in the photographed image, as the posture determination parameter of the human image, at least the polygon set based on the contour of the person, or the ellipse associated with the constituent part of the person, the barycentric position, and the angle A step of acquiring any information, wherein the posture classification processing step is a polygon set based on a contour of a person by comparing the posture determination information with a plurality of registered information stored in the storage unit Or a step executed by applying information on at least one of an ellipse, a gravity center position, and an angle associated with a constituent part of a person.

  According to this configuration, instead of comparing image data situations, the comparison processing is performed using polygons set based on the outline of the person, or ellipses, centroid positions, angles, etc. associated with the constituent parts of the person as parameters. As a result, the amount of data is reduced, and the post-processing columns can be increased.

  Furthermore, a third aspect of the present invention is a computer program that causes an information processing apparatus to execute posture analysis processing, and includes a photographing step for photographing a target region, and a person image included in the photographed image photographed in the photographing step. Based on the posture determination processing step for acquiring posture determination information, the posture determination information, and registration information as posture determination information corresponding to a plurality of registered postures stored in the storage unit, the posture determination information A posture classification processing step for classifying a registered posture having registration information similar to that as a posture of a person included in the photographed image, and analysis information for generating analysis information based on the registered posture classified in the posture classification processing step A generation step, and an information output step for outputting the analysis information generated in the analysis information generation step. There is a computer program which is characterized the door.

  According to this configuration, when performing posture analysis by inputting a large number of frame images, the person images included in the frame image are classified into a specific limited number of registered postures, and analysis based on the classified registered postures is performed. Since it is configured to execute, the processing efficiency is improved and the amount of stored data is also reduced. Also, the generation of analysis information based on the registered posture and the provision of analysis information to the user can be efficiently performed with a reduced processing burden.

  Note that the computer program of the present invention is a recording medium provided in a computer-readable format for a computer system capable of executing various program codes, for example, a recording medium such as a CD, FD, or MO. A computer program that can be provided by a medium or a communication medium such as a network. By providing such a program in a computer-readable format, processing corresponding to the program is realized on the computer system.

  Other objects, features, and advantages of the present invention will become apparent from a more detailed description based on embodiments of the present invention described later and the accompanying drawings. In this specification, the system is a logical set configuration of a plurality of devices, and is not limited to one in which the devices of each configuration are in the same casing.

  According to the configuration of the present invention, a frame-corresponding posture determination parameter is acquired based on a person image included in a captured image frame of a camera, and compared with registered parameters corresponding to a plurality of registered postures stored in the storage unit. The registered posture having the registration parameter similar to the frame-corresponding posture determination parameter is classified as the posture of the person included in the captured image frame, and the analysis information is generated based on the classified registered posture. With this configuration, posture analysis processing efficiency is improved, and a reduction in the amount of accumulated data is realized. Furthermore, posture evaluation information, information that associates posture with its temporal occupancy information, information that associates behavior with its temporal occupancy information, and the like can be acquired and provided as analysis information. The provided information can be applied to, for example, health management, productivity management, and business management.

  Details of the information processing apparatus, information processing method, and computer program of the present invention will be described below with reference to the drawings.

  First, an overview of processing executed by the information processing apparatus of the present invention will be described with reference to FIG. As illustrated in FIG. 1, the information processing apparatus 100 includes a video camera 101 and a microphone 102 and inputs image data obtained by capturing the user 10 and surrounding audio data. The information processing apparatus 100 inputs image data captured by the video camera 101 and audio data acquired by the microphone 102 and analyzes the posture and behavior of the user.

  Background image information and a plurality of user posture information are registered in a storage unit such as a hard disk of the information processing apparatus 100. Based on the registered posture information and a user posture analysis result based on an analysis of an input image. And perform processing such as alarm output. The posture information of the user by image analysis is displayed in the posture information display area 105 set on the display of the information processing apparatus 100. Posture information display includes various displays such as, for example, a display in which a line capable of determining a predetermined posture is superimposed on a captured image of the user, and an indicator display that can determine whether the user's posture is good or bad. Is displayed. Details of these will be described later.

  The information processing apparatus 100 further executes a user posture analysis based on image data captured by the video camera 101, and generates a summary of the user posture during a certain period of time, for example, one day working time from morning to evening. . That is, the transition of the posture while the user is working for one day in the office is accumulated, and a summary such as what posture is often taken is generated. Further, the information processing apparatus 100 further performs user behavior analysis by analyzing image data captured by the video camera 101 and audio data acquired by the microphone 102. For example, a series of user actions such as a time for inputting information to the PC and a time for making a phone call are analyzed.

  Details of the configuration and processing of the information processing apparatus of the present invention will be described with reference to FIG. FIG. 2 is a block diagram showing the overall configuration of the information processing apparatus of the present invention. The data processing executed by the information processing apparatus is executed by applying the hardware of the information processing apparatus and a computer program. The diagram shown in FIG. It is the block diagram shown as another block. A specific example of the hardware configuration of the information processing apparatus will be described later.

  The camera 101 captures a user image. The microphone 102 acquires the user's voice and other surrounding voices. The input unit 201 is an input unit such as a keyboard or a mouse. The input information analysis unit 202 performs user posture analysis based on information input via the camera 101 and the microphone 102. Further, processing such as registration of a plurality of postures by the user and registration of alarm conditions, which are executed as initial setting processing, are executed. The storage unit 203 stores result data analyzed by the input information analysis unit 202 based on the registration information, background image information, information input via the camera 101 and the microphone 102, and the like.

The input information analysis unit 202 performs data analysis based on image information and audio information input via the camera 101 and the microphone 102, and executes, for example, the following processing.
(1) Based on the analysis result of the user based on the analysis of the input image, the alarm output and the attitude information output to the output means (display) 204 are performed.
(2) A user posture analysis based on the image data is executed, and a summary of the user posture over a certain period is generated.
(3) A user behavior analysis is executed by analyzing image data and audio data.
Details of these processes will be described later.

  The output means (display) 204 outputs the posture information generated by the input information analysis unit 202.

  FIG. 3 shows a flowchart for explaining the overall sequence of processing executed by the information processing apparatus of the present invention. First, initial setting is performed in step S11. The initial setting processing is processing such as acquisition of a background image without a user, alarm condition setting processing, and setting of a period for performing analysis such as posture summary and behavior analysis. The background image without the user is used when extracting only the user from the captured image of the user.

  In step S12, a plurality of posture registration processes are performed. This is a process of registering a plurality of postures in advance in the storage means 203 as a database. This posture registration is performed when the camera 102 captures various postures of the user and the user inputs a registration command via the input unit 201 when the posture is a predetermined posture.

After completion of these processes, user monitoring is started in step S13.
(1) Based on the user's posture analysis result based on the analysis of the input image, alarm output, posture information output to the output means (display) 204,
(2) Execute user posture analysis based on image data and generate a summary of user posture over a certain period of time,
(3) Analysis of user behavior by analysis of image data and audio data,
These processes are executed.

  A specific example of the posture registration process in step S12 will be described with reference to FIG. For example, assume that the user takes various different poses as shown in FIGS. 4A to 4D and sets these postures as registered posture information. 4A to 4D show images taken by the camera 101. FIG. Based on these captured images, the input information analysis unit 202 registers parameters corresponding to each posture in the storage unit.

An example of registered parameters is shown in FIG. As shown in FIG. 5, the following data is registered corresponding to each of the registered postures (a) to (d) by the user.
* Head vertical length (xh)
* Vertical fuselage length (xt)
* Overall vertical length (xa)
* Head center of gravity (x1, y1)
* Body center of gravity (x2, y2)
* Overall center of gravity (x3, Y3)
* Head tilt from the vertical axis (α degrees)
* Body tilt from the vertical axis (β degrees)
* Overall inclination from vertical axis (δ degrees)
* Inclination of shoulder from vertical axis (γ degree)
* Polygon vertex position ((px1, py1), (px2, py2) ..., (pxn, pyn))
* Evaluation judgment information (Good, Bad, etc.)
These data are stored in the storage unit 203 as user registration attitude information.

  FIG. 6 shows a specific example of each of the registration information. The head center-of-gravity position (x1, y1) is the center of gravity 221 shown in FIG. 6, the trunk center-of-gravity position (x2, y2,) is the center of gravity 222 shown in FIG. 6, and the inclination of the head from the vertical axis (α degrees) is The angle [α] formed between the vertical axis 230 shown in FIG. 6 and the line 231 generated based on the head inclination, and the trunk inclination (β degrees) from the vertical axis are the same as the vertical axis 230 shown in FIG. The angle [β] formed with the line 232 generated based on the tilt of the fuselage and the overall tilt (δ degrees) from the vertical axis are generated based on the vertical tilt shown in FIG. 6 and the overall tilt. The angle [δ] formed by the line 234 and the inclination (γ degree) of the shoulder from the vertical axis are the angle [γ] formed by the vertical axis 230 shown in FIG. 6 and the line 233 generated based on the inclination of the shoulder. It corresponds to. Details of these data calculation methods will be described in the column of data analysis processing.

  In addition to these parameters, evaluation determination information indicating whether the posture is good (Good) or bad (Bad) is registered. This can be input based on user judgment. Alternatively, data corresponding to a good model posture having a parameter determined to be a good posture in advance is stored as recorded data in the information processing apparatus, and this model posture data and posture data newly registered by the user are stored. As a setting for performing comparison and collation, registration data having parameters close to the model posture data is registered as good posture (Good) registration information, and registration data having parameters significantly different from the model posture data is registered as bad posture (Bad) registration information. Also good.

  Next, a specific configuration and processing of the information analysis unit 202 illustrated in FIG. 2 will be described with reference to FIG. FIG. 7 is a block diagram showing a specific configuration of the input information analysis unit 202.

  The captured image data including the user input from the camera 101 is input to the background image information deletion processing unit 251 and background image information deletion processing is executed. In the initial setting process in step S11 described above with reference to FIG. 3, the background image without a user that has been captured and stored in the storage unit 203 is compared with the captured image data including the user input from the camera 101. Thus, the background image information deletion process is executed by deleting the matching image portion of both images.

  Further, for example, color filtering may be applied in order to extract a human skin (skin) region that is a target recognition target object. As the skin detection algorithm, for example, the algorithm described in the literature [A. Jaimes. Conceptual Structures and Computational Methods for Indexing and Organization of Visual Information, Ph.D. Thesis, Department of Electrical Engineering, Columbia University, February 2003] is used. It can be based.

The skin area extraction algorithm will be described. First, the pixel value data of each input image frame [f _i ] is expanded into HSV color space coordinates. The HSV color space coordinates are three-dimensional coordinates of hue (H), saturation (S), and luminance (V). In the HSV space, a specific area corresponds to a skin color area.

Among the pixel value data of each frame [f _i ], a pixel corresponding to the color area of the skin (skin) in the HSV color space coordinates is determined as a skin image area, and other than the color area of the skin (skin) in the HSV color space coordinates. The pixel data to which it belongs is determined to be outside the skin area.

The extraction of the skin region by the color filtering described above is executed for each frame [f _i ] input from the camera 101. However, even with this color filtering process, a region other than a skin (skin) region such as a wall or a table having a pixel value similar to a human skin (skin) may be determined as a skin (skin) region.

In order to avoid this, skin areas extracted by color filtering are grouped, a set of adjacent pixels determined as the skin area is set as a group, and a bounding box S _bb as a boundary area in which the boundary is set is set. To detect. Note that areas smaller than a certain size are excluded. Then, set the active bounding box A _bb comprising a contiguous area of the bounding box S _bb and bounding boxes S _bb set to the same center, or motion between frames is detected within this active bounding box A _bb not Determine whether. The presence / absence of motion can be determined based on difference information between the current frame and a frame that is several frames back in the past. By this processing, it is determined that the region with movement is a skin region, that is, a person.

  The image data that has undergone the background image information deletion processing in the background image information deletion processing unit 251 is input to the image processing unit 252, and processing for clarifying a pixel set indicating a person (user) image is performed. Specifically, smoothing processing, closing processing, pixel grouping processing, and the like are executed. This is a grouping process that compensates for the constituent pixels of a person accidentally deleted in the process of background image deletion processing, smoothing processing for smoothing the outline, and skipping pixels to form one person. This process is executed as By this processing, image data in which the outline of at least a person (user) is clear is acquired.

  Next, the person (user) image data acquired by the image processing is input to the posture determination processing unit 253. In the posture determination processing unit 253, posture determination processing based on a person image is executed.

  A detailed sequence of the posture determination process based on the person image executed in the posture determination processing unit 253 will be described with reference to the flowchart shown in FIG. First, the posture determination processing unit 253 executes gravity center extraction processing in step S101, and sets a radial line from the gravity center in step S102. These processes will be described with reference to FIG.

  FIG. 9A is a person (user) image obtained from a captured image. Based on this image data, the center of gravity 261 of the person shown in FIG. 9B is calculated. The barycentric point 241 is calculated based on the pixel distribution occupied by the person area. Next, a plurality of radial lines are set from the barycentric point 241. The example shown in FIG. 9B is an example in which horizontal and vertical lines and lines having an angle of 45 degrees from these lines are set, and four lines are set.

  Next, in step S103 of the flow shown in FIG. 8, the point of the object short portion of the radial line is detected, and in step S104, a polygon (polygon) connecting the end portions is generated. This is the polygon (polygon) 262 shown in FIG.

  Next, in step S105, a histogram based on the number of constituent pixels of the human image data is generated for each of the horizontal (X) axis and the vertical (Y) axis. FIG. 10C shows an example of a histogram. Histograms 263 and 264 based on the number of pixels are generated for the horizontal (X) axis and the vertical (Y) axis, respectively.

  Next, in step S106, the minimum value point of the vertical (Y) axis histogram is detected. In the example shown in FIG. In step S107, the minimum part of the vertical axis histogram is determined to be the neck, and the head and torso parts are identified. That is, in the example shown in FIG. 10C, it is determined that the position of the point 265 is a point constituting the person's neck, and the upper part is determined as the head and the lower part is determined as the torso.

  Next, in step S108, an ellipse is associated with each of the head and torso, the major and minor axes of the associated ellipse are calculated, the center of gravity of the head and torso is obtained, and each ellipse is further calculated. Based on the above, the inclination of each of the head and torso and the inclination of the shoulder (inclination from the vertical axis or the horizontal axis) is obtained.

  This process will be described with reference to FIG. First, as shown in FIG. 11D, ellipses 271 and 272 are associated with the head and the trunk, respectively. This process is performed, for example, by setting the ellipse 271 corresponding to the head to be inscribed in the head outline or circumscribed. Also for the trunk, the trunk corresponding ellipse 272 is set by setting an ellipse that is inscribed or circumscribed to the contour of the trunk.

  Next, the major axis and the minor axis of the ellipses 271 and 272 associated with the head and torso are set, and the intersection of these is obtained as the center of gravity 273 and 274 of the head and torso. Further, based on each ellipse 271, 272, the inclination of each of the head and torso (inclination from the vertical axis or horizontal axis) is obtained, and further, based on a line 275 indicating the inclination of the shoulder based on the contour of the person. Find the shoulder tilt (tilt from the vertical or horizontal axis).

Each of these parameters will be described with reference to FIG. FIG. 12E shows the center of gravity 273 and 274 of the head and torso as intersections of the major and minor axes of the ellipses 271 and 272 associated with the head and torso, respectively, and further, with respect to the vertical axis 290. The head tilt [α], the trunk tilt [β], and the shoulder tilt [γ] are shown. The line 291 is a line extending in the major axis direction of the ellipse associated with the head. The line 292 is a line extending in the major axis direction of the ellipse associated with the trunk. A line 293 is a line indicating the shoulder inclination obtained based on the outline of the person.
Similarly, an ellipse 276 and a center of gravity 277 associated with both the head and the torso are calculated, and a line 294 extending in the major axis direction of the ellipse 276 is obtained, and an overall inclination [δ] with respect to the vertical axis 290 is obtained. .

In this way
* Head vertical length (xh)
* Vertical fuselage length (xt)
* Overall vertical length (xa)
* Head center of gravity (x1, y1)
* Body center of gravity (x2, y2,)
* Overall center of gravity (x3, Y3)
* Head tilt from the vertical axis (α degrees)
* Body tilt from the vertical axis (β degrees)
* Overall inclination from vertical axis (δ degrees)
* Inclination of shoulder from vertical axis (γ degree)
* Polygon vertex position ((px1, py1), (px2, py2) ..., (pxn, pyn))
These data are acquired as parameters corresponding to the user's posture.

  As described above with reference to FIGS. 5 and 6, these parameters are the same as the parameters of the registered posture information. In the posture registration process in step S <b> 12 of the flowchart shown in FIG. 3, The same processing as described above is executed, and parameters corresponding to each registered posture are acquired and stored in the storage unit 203. Note that the registered attitude information includes evaluation determination information such as [Good] and [Bad] as described with reference to FIG. 5 in addition to these parameters.

  In the configuration of the input information analysis unit 202 illustrated in FIG. 7, the data registration unit 254 receives the parameters for each registered posture from the posture determination processing unit 253, and has been described above based on the input of the registration command from the input unit. Each parameter and evaluation determination information input from the input unit 201 are recorded in the storage unit 203 as registration information of each registered posture information.

Also in the information input and analysis processing in step S13 of the flowchart shown in FIG. 3, for the user's captured image, the background image information deletion processing in the background image information deletion processing portion 251 shown in FIG. 7, and the person (user) in the image processing portion 252 Posture determination parameters calculated based on image data acquisition and posture determination processing in the posture determination processing unit 253, that is,
* Head vertical length (xh)
* Vertical fuselage length (xt)
* Overall vertical length (xa)
* Head center of gravity (x1, y1)
* Body center of gravity (x2, y2,)
* Overall center of gravity (x3, Y3)
* Head tilt from the vertical axis (α degrees)
* Body tilt from the vertical axis (β degrees)
* Overall inclination from vertical axis (δ degrees)
* Inclination of shoulder from vertical axis (γ degree)
* Polygon vertex position ((px1, py1), (px2, py2) ..., (pxn, pyn))
Acquisition of these posture determination parameters is executed.

  Note that the posture determination unit 253 of the input information analysis unit 202 illustrated in FIG. 7 selects a frame at a predetermined interval for the frame image input from the camera 101, and performs posture determination parameter calculation processing based on the above-described posture determination. Execute. The calculated posture determination parameter is output to the posture classification processing unit 255.

  The posture classification processing unit 255 receives a frame-based posture determination parameter calculated at a predetermined frame interval. That is, posture determination parameters calculated based on the posture determination processing in the posture determination processing unit 253 are input at predetermined frame intervals.

  The posture classification processing unit 255 compares the input posture determination parameter corresponding to the frame with the parameter corresponding to the registered posture information that has been registered, that is, the parameter corresponding to the registered posture information described above with reference to FIG. Execute. In this parameter comparison process, it is determined that the registered posture information having the parameter value closest to the input posture determination parameter corresponding to the frame is the posture of the user in the frame.

  Further, the posture classification information of the posture classification processing unit 255 is output to the feedback posture analysis information generation unit 256. The feedback posture analysis information generation unit 256 sets the evaluation determination information set corresponding to the registered posture information determined to have similar parameters as a good posture [Good] or as a bad posture [Bad]. The evaluation determination information is output to the output means (display) 204 via the information output unit 258.

  FIG. 13 shows an output example of the evaluation determination information. FIG. 13 shows a display of an information processing apparatus used by a user, and a posture indicator 301 is displayed in a partial area of the display. The posture indicator 301 is displayed with a slider 302 and is set to indicate whether the current posture of the user being monitored is good [Good] or bad [Bad]. The central area of the posture indicator 301 indicates a good [Good] posture, and both sides of the posture indicator 301 indicate a bad [Bad] posture. The feedback posture analysis information generation unit 256 sets the evaluation determination information set corresponding to the registered posture information determined to have similar parameters as a good posture [Good] or as a bad posture [Bad]. Based on the evaluation determination information, the slider 302 is moved left and right for display.

  The user can confirm in real time whether his / her posture is good or bad by observing the display of the indicator. Moreover, it is good also as a structure which performs the alarm by the warning sound through the speaker of information processing apparatus, the alarm display with respect to a display, etc. according to a predetermined alarm setting condition.

  In the initial setting in step S11 in the flow of FIG. 3, the user first sets and stores an alarm condition such as outputting an alarm when a posture determined to be bad continues for a predetermined time or longer, for example. The registration information is stored in the unit 203. When the feedback posture analysis information generation unit 256 compares the alarm condition with the posture detection information based on the user's image analysis to determine whether or not the alarm output condition is satisfied, The alarm output is executed via the information output unit 258.

  With reference to FIG. 14, a processing sequence centering on processing executed by the posture classification processing unit 255 and the feedback posture analysis information generation unit 256 will be described. In step S301, a user detection process based on the monitor image is performed. In step S302, a posture determination parameter based on the user detection information is obtained based on the selected frame. These processes are executed as processes of the posture determination processing unit 253.

  At step S303, the posture classification processing unit 255 has received the frame-related posture determination parameters input from the posture determination processing unit 253 and the parameters corresponding to the registered posture information that has already been registered, that is, described above with reference to FIG. Comparison with the parameter corresponding to the registered posture information is executed.

  In this parameter comparison process, it is determined that the registered posture information having the parameter value closest to the input posture determination parameter corresponding to the frame is the posture of the user in the frame.

  In step S304, the feedback posture analysis information generation unit 256 sets the evaluation determination information set corresponding to the registered posture information determined to have similar parameters as a good posture [Good] or a bad posture [Bad ], And the slider 302 of the posture indicator 301 (see FIG. 13) presented on the output means (display) 204 is moved to the left and right based on the evaluation determination information.

  In step S305, the feedback posture analysis information generation unit 255 compares the alarm condition preset by the user with the posture detection information based on the user image analysis, and determines whether the alarm output condition is satisfied. If it is determined that the alarm output condition is satisfied, alarm output is executed via the information output unit 258 in step S306.

  In this way, the user can confirm in real time whether his / her posture is good or not by observing the display of this indicator, and is notified that the posture is bad by an alarm and corrects his / her posture. It becomes possible.

  Note that the display output to the output means (display) 204 is not limited to the indicator described with reference to FIG. 13, for example, a line indicating the inclination of the trunk, head, and shoulder described above with reference to FIG. 12 and the like. It is good also as a structure which superimposes and displays on a user's image. For example, as shown in FIG. 15, the posture information display unit 312 displays a line indicating the tilt of the trunk, head, and shoulder described above with reference to FIG. The user can clearly know the current state of his / her posture here when viewing the display data.

  Next, processing of the posture analysis information generation unit 257 in the input information analysis unit 202 illustrated in FIG. 7 will be described. The posture analysis information generation unit 257 executes posture analysis information generation processing and behavior analysis information generation processing. First, posture analysis information generation processing will be described.

  An example of posture analysis information generated by the posture analysis information generation unit 257 is shown in FIG. The posture analysis information shown in FIG. 16 is posture analysis information that shows a table of the types of postures of a certain user during the working time of one day and the time taken for the posture. In other words, the measurement time is set to AM10: 0 to PM5: 30 of 2005, 8, and 12, and the user image data acquired through the camera 101 during that time is processed at a predetermined frame interval, and the posture determination parameter of the selected frame Is compared with the registered parameters of the registered posture information, the registered posture having the most similar parameter is selected, and it is determined that the registered posture is held in the measurement frame interval. Get the history associated with the transition. By analyzing this information, the daily posture distribution of the user is calculated, and a posture analysis summary as shown in FIG. 16 is generated.

  In the data shown in FIG. 16, the user shows that the posture shown in (a) is 40%, the posture shown in (b) is 20%, and (c) in the period of AM10: 0 to PM5: 30 of 2005, 8, and 12. This is information indicating that the posture shown is 25% and the posture shown in (d) is 15%. By viewing these posture information summaries, the user can easily grasp the posture status of the day.

  With reference to the flowchart shown in FIG. 17, a processing sequence centering on the posture analysis processing executed by the posture analysis information generation unit 257 will be described. In step S401, a user detection process based on the monitor image is performed. In step S402, an attitude determination parameter based on the user detection information is obtained based on the selected frame. These processes are executed as processes of the posture determination processing unit 253.

  Next, in step S403, the posture classification processing unit 255 refers to the frame corresponding posture determination parameters input from the posture determination processing unit 253 and the parameters corresponding to the registered registered posture information, that is, with reference to FIG. Comparison with the parameter corresponding to the registered posture information described above is executed, and the registered posture information having the parameter value closest to the input posture determination parameter corresponding to the frame is determined to be the posture of the user in the frame.

  Step S404 and subsequent steps are processing of the posture analysis information generation unit 257. First, in step S404, the registered posture information selected by the posture classification processing unit 255 is registered as posture analysis information (summary) generation data. For example, if the frame time interval for acquiring the posture determination parameter is 1 minute, data is registered as the posture corresponding to the registered posture information selected in each frame is continued for 1 minute.

  Next, in step S405, it is determined whether or not a predetermined measurement time T1 of posture analysis information has elapsed. For example, a measurement time such as AM 10:00 to PM: 5:00 is set. If an elapsed time T1 (for example, 7 hours) has elapsed since the start of measurement, the measurement is terminated and the process proceeds to step S407. If the measurement time remains, it is determined in step S406 whether the measurement interval [t] has elapsed. This corresponds to a frame time interval for acquiring the posture determination parameter. The measurement time T1 of posture analysis information and the frame time interval t for obtaining posture determination parameters are set in the initial setting by the user.

  If the measurement interval [t] has elapsed in step S406, the processing in steps S401 to S404 is executed based on the frame at that time, and registered as posture analysis information (summary) generation data. If it is determined in step S405 that the predetermined posture analysis information measurement time T1 has elapsed, the measurement ends, and the process proceeds to step S407.

  In step S407, posture analysis information (summary) is generated and stored in the storage unit 203. The posture analysis information (summary) is the analysis information described with reference to FIG. If it is determined in step S408 that there has been a posture analysis information output request from the user via the input unit 201, the analysis information is output to the output unit (display) 204 via the information output unit 258 shown in FIG. (For example, analysis information shown in FIG. 16) is output.

  Next, behavior analysis processing executed by the posture analysis information generation unit 257 will be described. The behavior analysis process executed by the posture analysis information generation unit 257 is executed based on the user image captured by the camera 101 and the audio information acquired by the microphone 102.

  For example, as the registered posture information, as shown in FIG. 18, action information is further registered in the registered posture information described with reference to FIG. The posture shown in (a) is an action state in which a PC operation such as a keyboard operation is performed, and the posture state shown in (b) is a resting state.

  The user's work is determined based on the registered posture information of the user having the parameter most similar to the posture determination parameter acquired by the posture analysis process based on the user image captured by the camera 101. By this determination processing, for example, user behavior analysis information for one day is obtained as in the above-described posture analysis information. In this behavior analysis, based on the sound pressure information acquired by the microphone 102, information such as that the user is making a phone call or operating a keyboard is acquired and used as auxiliary information for analysis information. be able to.

  An example of the behavior analysis information generated by the posture analysis information generation unit 257 is shown in FIG. The behavior analysis information shown in FIG. 19 is behavior analysis information that shows, as a table, the type of behavior of a certain user during the working time of the day and the time during which the user is taking the behavior. In other words, the measurement time is set to AM10: 0 to PM5: 30 of 2005, 8, and 12, and the user image data acquired through the camera 101 during that time is processed at a predetermined frame interval, and the posture determination parameter of the selected frame The registered posture information is compared with the registered parameters, the registered posture having the most similar parameter is selected, the registered posture is held at the measurement frame interval, and the action corresponding to the posture is executed. It is determined that there is an action history with time. The information obtained from the microphone is added to this information and the analysis is executed to calculate the daily user behavior distribution, and a behavior analysis summary as shown in FIG. 19 is generated.

  In the data shown in FIG. 19, the user has (a) PC operation 40%, (b) telephone 20%, and (c) reading 25% during the period of AM10: 0 to PM5: 30 of 2005, 8, 12 (D) Information indicating that the break was 15%. The user can easily grasp the daily behavior situation by viewing these behavior information summaries.

  With reference to the flowchart shown in FIG. 20, the process sequence centering on the action analysis process which the attitude | position analysis information generation part 257 performs is demonstrated. In step S501, a user detection process based on the monitor image is performed. In step S502, a posture determination parameter based on the user detection information is obtained based on the selected frame. These processes are executed as processes of the posture determination processing unit 253.

  Next, in step S503, the posture classification processing unit 255 receives the frame-related posture determination parameters input from the posture determination processing unit 253 and parameters corresponding to the registered posture information registered, that is, the registered posture information illustrated in FIG. Is compared with the parameter corresponding to the frame, and it is determined that the posture and behavior corresponding to the registered posture information having the parameter value closest to the input frame corresponding posture determination parameter are the posture and behavior of the user in the frame. .

  Step S504 and subsequent steps are processing of the posture analysis information generation unit 257. First, in step S504, the registered posture information selected by the posture classification evaluation unit 255 is registered as behavior analysis information (summary) generation data. For example, if the frame time interval for acquiring the posture determination parameter is 1 minute, data is registered as an action corresponding to the registered posture information selected in each frame being continued for 1 minute.

  Next, in step S505, it is determined whether or not a predetermined measurement time T1 of behavior analysis information has elapsed. For example, a measurement time such as AM 10:00 to PM: 5:00 is set. If an elapsed time T1 (for example, 7 hours) has elapsed since the start of measurement, the measurement is terminated and the process proceeds to step S507. If the measurement time remains, it is determined in step S506 whether the measurement interval [t] has elapsed. This corresponds to a frame time interval for acquiring the posture determination parameter. The measurement time T1 of posture analysis information and the frame time interval t for obtaining posture determination parameters are set in the initial setting by the user.

  If the measurement interval [t] has elapsed in step S506, the processing of steps S501 to S504 is executed based on the frame at that time, and registered as behavior analysis information (summary) generation data. If it is determined in step S505 that the predetermined posture analysis information measurement time T1 has elapsed, the measurement ends and the process proceeds to step S507.

  In step S507, behavior analysis information (summary) is generated and stored in the storage unit 203. The behavior analysis information (summary) is, for example, the analysis information described with reference to FIG. If it is determined in step S508 that there has been a request for behavior analysis information output from the user via the input unit 201, in step S509 analysis information is output to the output unit (display) 204 via the information output unit 258 shown in FIG. (For example, analysis information shown in FIG. 19) is output.

  In the flow shown in FIG. 20, the behavior analysis process using the information acquired from the microphone is not performed. However, as described above, the process of analyzing the behavior pattern may be performed based on the information acquired from the microphone. Good.

As described above, in the configuration of the present invention, the posture of the user is classified by comparing the registered posture information registered in advance with the posture determination parameter of the user image input from the camera 101, and the following processing is performed based on the classification information: That is,
(1) Based on the user's posture analysis result, alarm output, posture information output to the output means (display) 204,
(2) Generate and provide user posture analysis information for a certain period,
(3) Generate and provide user behavior analysis information for a certain period,
It was set as the structure which performs these processes.

  Basically, the posture analysis of the user based on the registered posture information is performed in real time, and the analysis result based on the input frame becomes unnecessary after being associated with the registered posture information, and a large amount of information is accumulated. This eliminates the need for efficient processing.

In the above-described embodiment, the posture classification processing unit 255 converts the posture corresponding to the input frame to one of the registered postures by the similarity determination process based on the comparison between the posture determination parameter corresponding to the input frame and the parameter of the registered posture information. A process for classifying information into information is executed, and the comparison parameters are the parameters shown in FIG.
* Head vertical length (xh)
* Vertical fuselage length (xt)
* Overall vertical length (xa)
* Head center of gravity (x1, y1)
* Body center of gravity (x2, y2,)
* Overall center of gravity (x3, Y3)
* Head tilt from the vertical axis (α degrees)
* Body tilt from the vertical axis (β degrees)
* Overall inclination from vertical axis (δ degrees)
* Inclination of shoulder from vertical axis (γ degree)
* Polygon vertex position ((px1, py1), (px2, py2) ..., (pxn, pyn))
Although an example of performing comparison processing of these parameters has been described, in addition to these parameters, the feature information of this polygon is held as a registration parameter, and a polygon (polygon) generated based on the input frame image is stored. It is good also as a structure which performs a comparison with and performs similarity determination. Furthermore, the configuration may be such that the ellipse information of the head and torso described with reference to FIGS. 11 and 12 is held as a parameter, and the similarity determination is executed by comparing these ellipse information.

  Various types of analysis information obtained by the information processing apparatus of the present invention can be applied to, for example, health management and productivity management.

  For example, an example of processing when applied as health management information will be described. As described above with reference to FIG. 16, the posture analysis information generation unit 257 can acquire a temporal distribution of postures of a user during a certain period as posture analysis information. For example, when a user generates symptoms such as low back pain and stiff shoulders, the posture analysis information shown in FIG. 16 is presented to the doctor to explain his / her symptoms, thereby giving advice on a good posture to improve the symptoms. Can receive.

  As described with reference to FIG. 5, the user registers a plurality of postures as registered posture information in advance, and evaluation determinations such as good posture [Good] and bad posture [Bad] as the evaluation determination information in the registered posture information. Information can be included. By registering a good posture for improving the symptoms advised by the doctor as a good posture [Good] in the registered posture information, posture evaluation information evaluated by this good posture is shown in FIG. It will be indicated by the described posture indicator 301, and the user can hold a good posture designated by the doctor based on the information indicated by the indicator 301.

  An example applied to productivity management will be described. As described earlier with reference to FIG. 19, the posture analysis information generation unit 257 can acquire the temporal distribution of the behavior of the user for a certain period as the behavior analysis information. Further, as described above, the feedback posture analysis information generating unit 256 can output an alarm when, for example, a certain same posture continues for a predetermined time based on a certain alarm condition.

  A productivity management configuration to which such a configuration is applied will be described. For example, if the user has determined that the keyboard operation is continued for one hour or longer, the alarm is set to be output, or the alarm is set to be output when the telephone call is continued for one hour or longer. Thus, it is possible to warn the user that the same work has been continuously performed for a long time. Thus, by applying the configuration of the present invention, it is possible to prevent a reduction in work efficiency due to the continuation of the same work.

  Also, the behavior analysis information can be used for evaluating the user's work. For example, the behavior generated by the posture analysis information generation unit 257 is applied when monitoring the behavior of a user who is working as a sales person and applying an evaluation measure that the more eager the business is performed, the longer the phone call is made. Based on the analysis information, the user's telephone time can be determined, and behavior evaluation and productivity management by behavior analysis are realized.

  Finally, with reference to FIG. 21, a hardware configuration example of the information processing apparatus of the present invention will be described.

  A CPU (Central Processing Unit) 501 describes an execution sequence of each process such as an OS (Operating System), image processing, posture determination processing, posture classification, evaluation processing, posture analysis information generation processing described in the above-described embodiments. It is a control part which performs the process according to a computer program.

  A ROM (Read Only Memory) 502 stores programs used by the CPU 501, calculation parameters, and the like. A RAM (Random Access Memory) 503 stores programs used in the execution of the CPU 501, parameters that change as appropriate during the execution, and the like. These are connected to each other by a host bus 504 including a CPU bus.

  The host bus 504 is connected to an external bus 506 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 505.

  A keyboard 508 and a pointing device 509 are input devices operated by the user. The display 510 includes a liquid crystal display device, a CRT (Cathode Ray Tube), or the like, and displays various types of information as text and images.

  An HDD (Hard Disk Drive) 511 includes a hard disk, drives the hard disk, and records or reproduces a program executed by the CPU 501 and information. The hard disk, for example, functions as a storage unit for various setting information such as registered attitude information and alarm condition setting information described with reference to FIGS. 5 and 18 and stores various computer programs such as various data processing programs. Used as an area.

  The drive 512 reads data or a program recorded on a removable recording medium 521 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and the data or program is read out from the interface 507 and the external bus 506. , And supplied to the RAM 503 connected via the bridge 505 and the host bus 504.

  The connection port 514 is a port for connecting the external connection device 522 and has a connection unit such as USB or IEEE1394. The connection port 514 is connected to the CPU 501 and the like via the interface 507, the external bus 506, the bridge 505, the host bus 504, and the like. A communication unit 515 is connected to a network and executes transmission of data supplied from the CPU 501 or the HDD 511 and data reception. The camera 531 captures user image information, and the microphone 532 acquires and inputs audio data.

  Note that the hardware configuration example of the information processing apparatus illustrated in FIG. 21 is an example of an apparatus configured by applying a PC, and the information processing apparatus of the present invention is not limited to the configuration illustrated in FIG. Any configuration can be used as long as the processing described in the above item can be executed.

  The present invention has been described in detail above with reference to specific embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiments without departing from the gist of the present invention. In other words, the present invention has been disclosed in the form of exemplification, and should not be interpreted in a limited manner. In order to determine the gist of the present invention, the claims section described at the beginning should be considered.

  The series of processes described in the specification can be executed by hardware, software, or a combined configuration of both. When executing processing by software, the program recording the processing sequence is installed in a memory in a computer incorporated in dedicated hardware and executed, or the program is executed on a general-purpose computer capable of executing various processing. It can be installed and run.

  For example, the program can be recorded in advance on a hard disk or ROM (Read Only Memory) as a recording medium. Alternatively, the program is temporarily or permanently stored on a removable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, or a semiconductor memory. It can be stored (recorded). Such a removable recording medium can be provided as so-called package software.

  The program is installed on the computer from the removable recording medium as described above, or is wirelessly transferred from the download site to the computer, or is wired to the computer via a network such as a LAN (Local Area Network) or the Internet. The computer can receive the program transferred in this manner and install it on a recording medium such as a built-in hard disk.

  Note that the various processes described in the specification are not only executed in time series according to the description, but may be executed in parallel or individually according to the processing capability of the apparatus that executes the processes or as necessary. Further, in this specification, the system is a logical set configuration of a plurality of devices, and the devices of each configuration are not limited to being in the same casing.

  As described above, according to the configuration of the present invention, the frame-corresponding posture determination parameter is acquired based on the person image included in the captured image frame of the camera, and a plurality of registered postures stored in the storage unit are supported. The registered posture having a registered parameter similar to the frame-corresponding posture determination parameter is classified as the posture of the person included in the captured image frame, and the analysis information is based on the classified registered posture. It was set as the structure which produces | generates. With this configuration, posture analysis processing efficiency is improved, and a reduction in the amount of accumulated data is realized. Furthermore, posture evaluation information, information that associates posture with its temporal occupancy information, information that associates behavior with its temporal occupancy information, and the like can be acquired and provided as analysis information. The provided information can be applied to, for example, health management, productivity management, and business management.

It is a figure explaining the outline | summary of a process of the information processing apparatus of this invention. It is a figure explaining the structure of the information processing apparatus of this invention. It is a figure which shows the flowchart explaining the whole process sequence which the information processing apparatus of this invention performs. It is a figure explaining the example of the registration attitude | position information registered in the information processing apparatus of this invention. It is a figure which shows the data example of the registration information containing the attitude | position determination parameter of the registration attitude | position information registered in the information processing apparatus of this invention. It is a figure explaining the attitude | position determination parameter of the registration attitude | position information registered in the information processing apparatus of this invention. It is a block diagram explaining the structure of the input information analysis part of the information processing apparatus of this invention. It is a figure which shows the flowchart explaining the process sequence which the attitude | position determination process part in the input information analysis part of the information processing apparatus of this invention performs. It is a figure explaining the specific example of the process which the attitude | position determination process part in the input information analysis part of the information processing apparatus of this invention performs. It is a figure explaining the specific example of the process which the attitude | position determination process part in the input information analysis part of the information processing apparatus of this invention performs. It is a figure explaining the specific example of the process which the attitude | position determination process part in the input information analysis part of the information processing apparatus of this invention performs. It is a figure explaining the specific example of the process which the attitude | position determination process part in the input information analysis part of the information processing apparatus of this invention performs. It is a figure explaining the example of a display of the attitude | position indicator which the information processing apparatus of this invention performs. It is a figure which shows the flowchart explaining the processing sequence which the attitude | position classification process part in the input information analysis part of the information processing apparatus of this invention and a feedback attitude | position analysis information generation part perform. It is a figure explaining the example of a display of the attitude | position information which the information processing apparatus of this invention performs. It is a figure explaining the specific example of the attitude | position analysis information which the attitude | position analysis information generation part in the input information analysis part of the information processing apparatus of this invention produces | generates. It is a figure which shows the flowchart explaining the production | generation processing sequence of the attitude | position analysis information of the attitude | position analysis information generation part in the input information analysis part of the information processing apparatus of this invention. It is a figure which shows the data example of the registration information containing the attitude | position determination parameter of the registration attitude | position information registered in the information processing apparatus of this invention. It is a figure explaining the specific example of the action analysis information which the attitude | position analysis information generation part in the input information analysis part of the information processing apparatus of this invention produces | generates. It is a figure which shows the flowchart explaining the production | generation processing sequence of the action analysis information of the attitude | position analysis information generation part in the input information analysis part of the information processing apparatus of this invention. It is a figure which shows the hardware structural example of the information processing apparatus of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 User 100 Information processing apparatus 101 Camera 102 Microphone 105 Posture information display area 201 Input unit 202 Input information analysis unit 203 Storage unit 204 Output unit 251 Background image information deletion processing unit 252 Image processing unit 253 Posture determination processing unit 254 Data registration unit 255 Posture classification processing unit 256 Feedback posture analysis information generation unit 257 Posture analysis information generation unit 258 Information output unit 301 Posture indicator 302 Slider 312 Posture information display unit 501 CPU (Central processing Unit)
502 ROM (Read-Only-Memory)
503 RAM (Random Access Memory)
504 Host bus 505 Bridge 506 External bus 507 Interface 508 Keyboard 509 Pointing device 510 Display 511 HDD (Hard Disk Drive)
512 Drive 514 Connection port 515 Communication unit 521 Removable recording medium 522 Externally connected device 531 Camera 532 Microphone

Claims (20)

An imaging unit for imaging the target area;
A posture determination processing unit that acquires posture determination information based on a person image included in a photographed image of the photographing unit;
A storage unit storing posture determination information corresponding to a plurality of registered postures as registration information;
A posture that compares the posture determination information with a plurality of registration information stored in the storage unit, and classifies a registered posture having registration information similar to the posture determination information as a posture of a person included in the captured image. A classification processing unit;
An analysis information generating unit that generates analysis information based on the registered postures classified in the posture classification processing unit;
An information output unit for outputting the analysis information generated by the analysis information generation unit;
An information processing apparatus comprising: The storage unit
As registration information corresponding to the plurality of registered postures, there is evaluation determination information for each registered posture,
The analysis information generation unit
The configuration is such that evaluation information about a posture of a person included in the photographed image is generated based on evaluation determination information set corresponding to the registered posture classified by the posture classification processing unit. Item 4. The information processing apparatus according to Item 1. The information output unit includes:
The information processing apparatus according to claim 2, wherein an indicator display process is performed on the evaluation information generated by the analysis information generation unit. The information output unit includes:
The information processing apparatus according to claim 1, wherein the information processing apparatus is configured to execute a process of displaying tilt information of a person's head and torso included in the photographed image together with the person image. The information output unit includes:
The posture determination processing unit is configured to execute a process of superimposing and displaying a reference line indicating a generated posture angle reference and a line indicating a posture inclination on an image indicating the posture. The information processing apparatus according to claim 1. The analysis information generation unit
A configuration for generating posture analysis information of a person within a predetermined measurement period,
Based on the classification registration posture corresponding to each of the photographed images classified by the posture classification processing unit based on the photographed image acquired within the measurement period, the registered posture and the time of the registered posture within the measurement period The information processing apparatus according to claim 1, wherein the information that associates the general occupation rate information is generated as analysis information. The analysis information generation unit
It is a configuration for generating human behavior analysis information within a predetermined measurement period,
Based on the classification registration posture of each photographed image classified by the posture classification processing unit based on the photographed image acquired within the measurement period, an action mode associated with the registration posture, and the behavior in the measurement period The information processing apparatus according to claim 1, wherein the information is associated with a temporal occupancy rate of an action and is generated as analysis information. The storage unit
It has registration parameters corresponding to multiple registered postures and action information corresponding to each registered posture as recorded data,
The analysis information generation unit
Based on the classification registration posture of each frame classified by the posture classification processing unit based on the frame image acquired within the measurement period, the behavior information corresponding to the registered posture is acquired from the storage unit, and based on the acquired behavior information The information processing apparatus according to claim 7, wherein the information processing apparatus is configured to generate a behavior mode and temporal occupancy information of the behavior within the measurement period. The posture determination processing unit
Configuration for acquiring at least one of polygons set based on a person's outline or ellipse, centroid position, and angle associated with a constituent part of the person as posture determination information of the person image included in the captured image And
The posture classification processing unit
A comparison between the posture determination information and a plurality of registration information stored in the storage unit is a polygon set based on the outline of a person, or an ellipse associated with a constituent part of the person, a barycentric position, and an angle. The information processing apparatus according to claim 1, wherein the information processing apparatus is configured to execute by applying any information. The posture determination processing unit
2. The configuration according to claim 1, wherein a part of frames is selected from continuous frames photographed by the photographing unit, and posture determination information is acquired based on a person image included in a photographed image of the selected frame. The information processing apparatus described. The photographing unit
The information output unit is provided adjacent to a display screen for outputting information, and is configured to perform shooting with the upper body of the shooting target set as a shooting range,
The posture determination processing unit
The information processing apparatus according to claim 1, wherein the posture determination process is executed by dividing the head and the body of the person to be photographed based on a person image included in a photographed image of the photographing unit. . The posture determination processing unit
A configuration in which an ellipse corresponding to each of the head and torso of the subject to be photographed, a major axis and a minor axis of the ellipse, and a process of determining a posture based on an inclination of the major axis with respect to a vertical axis in each ellipse The information processing apparatus according to claim 11, wherein the information processing apparatus is an information processing apparatus. The information processing apparatus further includes:
Has a voice input unit,
The analysis information generation unit
The information processing apparatus according to claim 1, wherein the analysis information is generated based on voice information input from the voice input unit. The analysis information generation unit
Based on the registered posture classified in the posture classification processing unit, it is configured to execute processing to generate health management information or productivity management information,
The information output unit includes:
The information processing apparatus according to claim 1, wherein the information processing apparatus is configured to output health management information or productivity management information generated by the analysis information generation unit. A shooting step for shooting the target area;
A posture determination processing step for acquiring posture determination information based on a person image included in the photographed image photographed in the photographing step;
The posture determination information is compared with registration information as posture determination information corresponding to a plurality of registered postures stored in the storage unit, and a registered posture having registration information similar to the posture determination information is added to the captured image. A posture classification processing step for classifying the posture of the included person,
An analysis information generating step for generating analysis information based on the registered posture classified in the posture classification processing step;
An information output step for outputting the analysis information generated in the analysis information generation step;
An information processing method characterized by comprising: The analysis information generation step includes
The step of generating evaluation information about the posture of a person included in the photographed image based on the evaluation determination information set corresponding to the registered posture classified in the posture classification processing step. Item 16. The information processing method according to Item 15. The analysis information generation step includes
A step of generating posture analysis information of a person within a predetermined measurement period,
Based on the registered registration of each captured image classified in the orientation classification processing step based on the captured image acquired within the measurement period, the registered attitude and the temporal occupation of the registered attitude within the measurement period The information processing method according to claim 15, wherein information associated with rate information is generated as analysis information. The analysis information generation step includes
A step of generating human behavior analysis information within a predetermined measurement period,
Based on the classification registration posture of each photographed image classified in the posture classification processing step based on the photographed image acquired within the measurement period, an action mode associated with the registered posture, The information processing method according to claim 15, wherein information associated with the temporal occupancy rate of the action is generated as analysis information. The posture determination processing step includes:
As the posture determination information of the human image included in the photographed image, as the posture determination parameter of the human image, at least the polygon set based on the contour of the person, or the ellipse associated with the constituent part of the person, the barycentric position, and the angle It is a step to acquire any information,
The posture classification processing step includes:
A comparison between the posture determination information and a plurality of registration information stored in the storage unit is a polygon set based on the outline of a person, or an ellipse associated with a constituent part of the person, a barycentric position, and an angle. The information processing method according to claim 15, wherein the information processing method is a step executed by applying any of the information. A computer program that causes an information processing apparatus to execute posture analysis processing;
A shooting step for shooting the target area;
A posture determination processing step for acquiring posture determination information based on a person image included in the photographed image photographed in the photographing step;
The posture determination information is compared with registration information as posture determination information corresponding to a plurality of registered postures stored in the storage unit, and a registered posture having registration information similar to the posture determination information is added to the captured image. A posture classification processing step for classifying the posture of the included person,
An analysis information generating step for generating analysis information based on the registered posture classified in the posture classification processing step;
An information output step for outputting the analysis information generated in the analysis information generation step;
A computer program characterized by comprising:

Images