WO2014207991A1 - 教師データ生成装置、方法、プログラム、および群衆状態認識装置、方法、プログラム - Google Patents
教師データ生成装置、方法、プログラム、および群衆状態認識装置、方法、プログラム Download PDFInfo
- Publication number
- WO2014207991A1 WO2014207991A1 PCT/JP2014/002670 JP2014002670W WO2014207991A1 WO 2014207991 A1 WO2014207991 A1 WO 2014207991A1 JP 2014002670 W JP2014002670 W JP 2014002670W WO 2014207991 A1 WO2014207991 A1 WO 2014207991A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- person
- state
- crowd
- image
- control instruction
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion
Definitions
- the present invention relates to a teacher data generation device, a teacher data generation method, a teacher data generation program, and a crowd state recognition device, a crowd state recognition method, and a crowd state recognition program for recognizing a crowd state in an image. .
- the human behavior determination device described in Patent Document 1 extracts a change area where a difference is generated from a video image by using a background difference or the like, and calculates a feature amount from the change area. And this person action determination apparatus determines whether the change area
- the number counting device described in Patent Document 2 measures the number of people from an image of crowded people.
- This person counting device extracts a human head included in an image based on a head model. And this person counting device uses a feature quantity such as position information and color distribution to connect head positions determined to be the same person between frames, and from the connection result (person tracking result), Measure the number of people.
- the system described in Patent Document 3 detects a state such as steady state (for example, a mainstream flow of a person) / unsteady state (for example, reverse to the mainstream flow) from a crowd image.
- This system aggregates optical flow attributes for a determination block that is a unit of determination, and calculates an evaluation value for evaluating the stationary degree of the optical flow. Then, this system determines the state of the determination block from the evaluation value.
- JP2011-100195A (paragraphs 0028-0030) JP 2010-198566 A (paragraphs 0046-0051) JP 2012-22370 A (paragraph 0009)
- Patent Documents 1 to 3 use each frame of video, and the performance of state determination depends on the frame interval.
- a person region is associated between frames to obtain a person locus.
- head positions are connected between frames, and the result is used as a person tracking result.
- the frame rate is low, the amount of movement of the person increases, and accordingly, changes in the person area and head position and changes in shape (posture) increase.
- the influence of disturbances such as illumination increases. For this reason, it is difficult to associate the person region and the head position between the frames.
- a method using a discriminator based on a learned dictionary can be considered.
- the dictionary is learned with teacher data such as images showing the crowd state.
- teacher data learning data
- the arrangement of people (how people overlap and how people are biased), the direction of the people, and the density (number of people per unit area) are determined in various states. It is necessary to collect a large amount of images with various changes in the background, lighting, clothes and posture of the person. By performing machine learning using these images, a classifier dictionary can be obtained.
- the work load of teacher data collection increases.
- the present invention provides a teacher data generation device, a teacher data generation method, and a teacher data generation program capable of easily generating a lot of teacher data used when machine learning a classifier dictionary for recognizing a crowd state
- the purpose is to provide.
- a teacher data generation device selects a background image from a plurality of background images prepared in advance, extracts a region in the background image, and expands or reduces an image corresponding to the extracted region to an image of a predetermined size.
- a background extracting means for reducing, a plurality of person state control instructions that are instruction information of person states related to a plurality of persons, and an individual person state control instruction that is instruction information of the state of individual persons among the plurality of persons;
- the image of the person corresponding to the person state determined by the person state determination means is combined with the image of the predetermined size obtained by the person state determination means for determining the person state of the crowd and the background extraction means.
- a crowd state image synthesizing unit for generating a crowd state image, specifying a teacher label for the crowd state image, and outputting a set of the crowd state image and the teacher label. And butterflies.
- the crowd state recognition apparatus includes a rectangular region group storage unit that stores a rectangular region group indicating a recognition target location of a crowd state on an image, and a predetermined size image that represents the crowd state.
- a plurality of sets of a crowd state image which is an image including a person whose reference part is represented with a size similar to the size of the reference part of the person defined for, and a teacher label for the crowd state image are used.
- the crowd state recognition dictionary storage means for storing the classifier dictionary obtained by performing machine learning and the regions indicated by the rectangular area groups stored in the rectangular area group storage means are extracted from the given images.
- a crowd state recognizing means for recognizing a crowd state reflected in the extracted image based on the dictionary.
- the teacher data generation method selects a background image from a plurality of background images prepared in advance, extracts a region in the background image, and converts an image corresponding to the extracted region into an image of a predetermined size.
- a background extraction step for enlarging or reducing, a multi-person state control instruction that is instruction information about the state of a person related to a plurality of persons, and an individual person state control that is instruction information about the state of an individual person among the plurality of persons
- the person state determination step for determining the person state of the crowd and the image of the person corresponding to the person state determined in the person state determination step are synthesized with the image of the predetermined size obtained in the background extraction step.
- a crowd state image is generated by generating a crowd state image as an image, specifying a teacher label for the crowd state image, and outputting a set of the crowd state image and the teacher label. Characterized in that it comprises a flop.
- the rectangular region group storage unit stores a rectangular region group indicating the recognition target location of the crowd state on the image
- the crowd state recognition dictionary storage unit stores the predetermined state indicating the crowd state.
- a crowd state image that is an image of a size and includes a person whose reference part is represented in a size similar to the size of the reference part of a person defined for a predetermined size
- a dictionary of classifiers obtained by performing machine learning using a plurality of pairs with teacher labels is stored, and each region indicated by a rectangular region group stored in a rectangular region group storage unit is stored from a given image. It includes a crowd state recognition step of extracting and recognizing the crowd state in the extracted image based on the dictionary.
- the teacher data generation program selects a background image from a plurality of background images prepared in advance in a computer, extracts an area in the background image, and selects an image corresponding to the extracted area as a predetermined size.
- Background extraction processing for enlarging or reducing images, multiple person state control instructions that are instruction information of person states related to a plurality of persons, and individual persons that are instruction information of individual person states in the plurality of persons
- a crowd state recognition program includes a rectangular region group storage unit that stores a rectangular region group indicating a recognition target location of a crowd state on an image, and an image of a predetermined size that represents the crowd state.
- a plurality of sets of a crowd state image which is an image including a person whose reference part is represented with a size similar to the size of the reference part of the person defined for, and a teacher label for the crowd state image are used.
- a rectangular area group stored in the rectangular area group storage means from a given image in a computer having a crowd state recognition dictionary storage means for storing a classifier dictionary obtained by performing machine learning Each region is extracted, and a crowd state recognition process for recognizing the crowd state in the extracted image is executed based on the dictionary.
- the teacher data generation device the teacher data generation method, and the teacher data generation program of the present invention, it is possible to easily generate a lot of teacher data used when machine learning is performed on a classifier dictionary for recognizing a crowd state. it can.
- the crowd state in the image can be recognized well without depending on the frame rate.
- FIG. 1 is a block diagram showing a configuration example of a teacher data generation apparatus according to the present invention.
- the teacher data generation apparatus 10 of the present invention generates teacher data for machine learning of the crowd state in an image.
- the teacher data generation apparatus 10 creates a plurality of sets of local images in a crowded state and teacher labels corresponding to the local images.
- “local” means an area smaller than an area of an image (an image acquired by an image acquisition device 3 (see FIG. 7) described later) that is a recognition target of the crowd state.
- the local image of the crowd state is an image representing a set of reference parts (hereinafter referred to as reference parts), which are parts of persons constituting the crowd in such a region. .
- the case where the head is used as the reference part will be described as an example, but a part other than the head may be used as the reference part.
- a local image of the crowd state is referred to as a crowd patch.
- a part of a person other than the reference part may be represented in the crowd patch.
- the teacher data generation device 10 includes a data processing device 1 that operates under program control, and a storage device 2 that stores information.
- the storage device 2 includes a background image storage unit 21, a learning local image information storage unit 22, a crowd state control instruction storage unit 23, a person state control instruction storage unit 24, a person image storage unit 25, and a person area image.
- Storage means 26 includes a background image storage unit 21, a learning local image information storage unit 22, a crowd state control instruction storage unit 23, a person state control instruction storage unit 24, a person image storage unit 25, and a person area image.
- the background image storage means 21 stores a plurality of background images (background image group) used as a background in the crowd patch. This background image does not include a person. You may use the local image which image
- CG Computer Graphics
- the local image information storage unit 22 for learning stores the size of a crowd patch (a local image in a crowd state used for machine learning) and the size of a reference portion of a person with respect to the crowd patch.
- the size of the crowd patch is determined to be vertical h pixels and horizontal w pixels.
- the vertical size of the reference portion (head in this example) of the person constituting the crowd shown in the crowd patch is 1 / ⁇ times the vertical size of the crowd patch, that is, h / ⁇ pixels.
- the height h pixel and the width w pixel are stored in the learning local image information storage unit 22 as the size of the crowd patch.
- the vertical h / ⁇ pixel is stored in the learning local image information storage unit 22 as the size of the reference portion of the person.
- the case where the vertical size is stored as the size of the reference portion is taken as an example, but the size of the reference portion to be stored is not limited to the vertical size.
- the horizontal size of the reference region of the person is determined to be 1 / ⁇ times the horizontal size of the crowd patch, that is, w / ⁇ pixels.
- the learning local image information storage means 22 stores the vertical h pixels and the horizontal w pixels as crowd patch sizes in the learning local image information storage means 22, and the learning local image information storage means 22 uses horizontal w / ⁇ pixels as the size of the person's reference region.
- the size of the reference portion of the person may be determined by using either the vertical size or the horizontal size.
- a diagonal size or the like may be used.
- the size of the reference portion of a person is a size that allows the person to be recognized as a person if the reference portion of the person appears in the crowd patch at the same size as that size. For example, if the reference part of a person appears extremely large in the crowd patch, or conversely appears extremely small, the person is considered a mere background, even though it is a person that makes up the crowd. .
- the crowd state control instruction storage means 23 stores person state instruction information related to a plurality of persons (hereinafter referred to as a “multiple person state control instruction”) when a plurality of person images are combined in a crowd patch. .
- the multiple person state control instruction is determined in advance by the operator of the teacher data generation apparatus 10 and stored in the crowd state control instruction storage unit 23.
- the multi-person state control instruction includes items “personal arrangement” relating to the arrangement relationship of persons such as the degree of person overlap when combining multiple person images, and the item “personal direction” relating to the orientation of persons. ”And the item“ number of persons ”relating to the number of persons and density. Items for which a multiple person state control instruction is determined are not limited to these items.
- FIG. 2 is a schematic diagram illustrating an example of information stored in the crowd state control instruction storage unit 23.
- FIG. 2 exemplifies a multi-person state control instruction defined for the items “personal arrangement”, “person direction”, and “number of people”.
- the mode of the multiple person state control instruction includes “predetermined state”, “random”, “predetermined rule”, and the like.
- the “predetermined state” is an instruction mode for instructing a specific state for the corresponding item.
- “three people” defined for the item “number of people” corresponds to the “predetermined state”.
- “3 people” is specifically designated as “number of people”.
- Another example of the “predetermined state” includes, for example, instructing “right direction for all members” with respect to the item “direction of person”.
- Random means that the state can be arbitrarily determined for the corresponding item.
- a “multi-person state control instruction” “random” is defined for “person arrangement” and “person direction”.
- the “predetermined rule” is an instruction mode for instructing that the state of the corresponding item may be determined within a range that satisfies the rule specified by the operator. For example, regarding the item “placement of persons”, when the rule “place 50% of persons in a superimposed manner” is defined, the state of the person is determined so as to satisfy at least the rule regarding the placement of persons. It is instructing that. Further, for example, with respect to “the direction of the person”, a rule is defined that “a person arranged on the right side of the center of the crowd patch faces rightward, and a person arranged on the left side of the center faces leftward”. In the case of the person, it is instructed to determine the state of the person so that at least the rule is satisfied with respect to the direction of the person.
- the crowd state control instruction storage means 23 stores the presence / absence of designation of a teacher label for each item.
- “ ⁇ ” represents information indicating that a teacher label has been specified
- “x” represents information indicating that a teacher label has not been specified. This also applies to FIG. 3 described later.
- the operator selects one or more items for designating the teacher label among the items for which the plural person state control instruction is determined. Further, the operator determines a plural person state control instruction for each item regardless of whether or not it is an item for designating a teacher label.
- a multi-person state control instruction in this example, a random instruction
- the operator sets the mode of the plural person state control instruction to “predetermined state” for the item for designating the teacher label.
- a specific state of three persons is instructed regarding “number of persons”, which is an item for which a teacher label is designated.
- the crowd state control instruction storage means 23 stores a plurality of person state control instructions determined by the operator for each item and whether or not a teacher label is designated.
- the crowd state control instruction storage means 23 includes a plurality of person state control instructions determined by the operator and a teacher label for at least the items “personal arrangement”, “person direction”, and “number of persons”. An example in which the presence / absence of the designation is stored will be described.
- the person state control instruction storage unit 24 stores information (hereinafter referred to as individual person state control instructions) that indicates the state of each person when a plurality of person images are combined in the crowd patch.
- the “multi-person state control instruction” indicates the state of a person related to a plurality of persons, whereas the “individual person state control instruction” indicates the state of individual persons belonging to the plurality of person groups. To do.
- the individual person state control instruction is determined in advance by the operator of the teacher data generation apparatus 10 and stored in the person state control instruction storage unit 24.
- FIG. 3 is a schematic diagram illustrating an example of information stored in the person state control instruction storage unit 24.
- FIG. 3 exemplifies individual person state control instructions defined for the items “photographing angle of person”, “lighting to person”, and “posture of person”.
- the mode of the individual person state control instruction includes “predetermined state”, “random”, “predetermined rule”, etc., as in the case of the multiple person state control instruction.
- the “predetermined state” is an instruction mode for instructing a specific state with respect to the corresponding item, similarly to the case described with the plural person state control instruction.
- “walking” defined for the item “person's posture” corresponds to a “predetermined state”.
- “person's posture” is specifically designated as a walking posture.
- Random means that the state may be arbitrarily determined with respect to the corresponding item, similarly to the case described in the plural person state control instruction.
- an individual person state control instruction “random” is defined for “lighting a person”.
- the “predetermined rule” is an instruction mode for instructing to determine the state of the corresponding item within a range that satisfies the rule specified by the operator, as in the case of the description of the plural person state control instruction.
- a predetermined rule is defined for “person's shooting angle”.
- an instruction based on the camera parameters is used to calculate the shooting angle of the person from the person layout at the time of composition, and the instruction is given to determine the state of the person according to the shooting angle.
- the person size at the time of combining is set. If the rule “determined” is defined, it means that at least the size of the person is determined so as to satisfy the rule.
- the person state control instruction storage unit 24 also stores the designation of the teacher label for each item.
- the operator may select one or a plurality of items for designating the teacher label not only for items for which a plurality of person state control instructions are set but also for items for which individual person state control instructions are set. Even in this case, the operator determines an individual person state control instruction for each item regardless of whether or not it is an item for designating a teacher label.
- individual person state control instructions are also set for items “person's shooting angle” and “lighting a person” without designation of a teacher label. However, the operator sets the mode of the individual person state control instruction to “predetermined state” for the item for specifying the teacher label.
- FIG. 3 the operator may select one or a plurality of items for designating the teacher label not only for items for which a plurality of person state control instructions are set but also for items for which individual person state control instructions are set. Even in this case, the operator determines an individual person state control instruction for each item regardless of whether or not it is an item for designating a teacher label.
- individual person state control instructions are also set for items “person's
- the person state control instruction storage means 24 stores an individual person state control instruction determined by the user for each item and whether or not a teacher label is designated.
- the operator may not specify the teacher label for all items for which the individual person state control instruction is determined. However, as described above, with respect to the items for which the plural person state control instruction is determined, the operator determines one or more items as items for specifying the teacher label.
- the person state control instruction storage unit 24 includes at least a “person's shooting angle”, “lighting a person”, “person's posture”, “person's clothes”, “person's body shape”, “person” ”Hairstyle” and “Human size when combined with crowd patch”, an example in which an individual person state control instruction determined by the operator and whether or not a teacher label is specified is stored as an example will be described. .
- the content of the plural person state control instruction determined for the item for which the teacher label is designated becomes the teacher label corresponding to the crowd patch generated according to the information stored in the crowd state control instruction storage means 23.
- the content of the individual person state control instruction determined for the item for which the teacher label is designated becomes the teacher label corresponding to the crowd patch generated according to the information stored in the person state control instruction storage unit 24.
- the teacher label based on the plural person state control instruction is the main teacher label
- the teacher label based on the individual person state control instruction is a supplementary teacher label for the teacher label.
- the data processing apparatus 1 (see FIG. 1) is configured to execute the plural person state control instruction of each item stored in the crowd state control instruction storage unit 23 and each of the person state control instruction storage unit 24 stored in the person state control instruction storage unit 24.
- the state of the person is determined, and a crowd patch is generated by synthesizing those persons.
- the data processing apparatus 1 determines, as the teacher label, the contents of the plural person state control instruction and the individual person state control instruction determined for the item for which the teacher label is designated for the crowd patch.
- the data processing apparatus 1 generates a crowd patch in response to each of the plural person state control instruction and the individual person state control instruction illustrated in FIGS.
- the crowd patch shows a state where three people are walking.
- the data processing apparatus 1 determines a teacher label “3 people walking” as the teacher label of the crowd patch.
- the item stored in the person state control instruction storage means 24 includes “person size when combined with a crowd patch”.
- person size when synthesizing a person to be recognized as a person in the crowd patch for example, it is stored in the local image information storage means 22 for learning as an individual person state control instruction of “person size when synthesizing to a crowd patch”.
- the size of the reference part of the person may be designated, or random may be designated. As a result of designating random, if the person's state is tentatively determined with a reference part size greatly different from the person's reference part size stored in the local image information storage means 22 for learning, Just make a tentative decision.
- a person to be a background is synthesized in the crowd patch, for example, it is stored in the local image information storage means 22 for learning as an individual person state control instruction of “person size when synthesized into a crowd patch”.
- a size that is greatly different from the size of the reference portion of the person may be designated, or random may be designated.
- the person's state may be tentatively determined again.
- the data processing apparatus 1 determines the state of a person to be recognized as a person (hereinafter sometimes referred to as a foreground person), and determines the state of a background person. Do each.
- a multi-person state control instruction and individual person state control instruction for determining the foreground person state and a multi-person state control instruction and individual person state control instruction for determining the background person state are separately determined by the operator. It may be done.
- the crowd state control instruction storage means 23 stores a plurality of person state control instructions for determining the foreground person state and a plurality of person state control instructions for determining the background person state.
- the person state control instruction storage unit 24 stores an individual person state control instruction for determining the foreground person state and an individual person state control instruction for determining the background person state. Further, the multiple person state control instruction and the individual person state control instruction may not be divided into those for determining the foreground person state and those for determining the background person state.
- the person image storage means 25 includes a plurality of person images (persons) to which person state information such as person direction, person shooting angle, person lighting, person posture, clothing, body shape, hairstyle and the like is added for each person image. Image group). That is, the data processing apparatus 1 can read a person image that matches the determined state from the person image storage unit 25.
- the person area image storage means 26 stores a person area image group corresponding to the person image group stored in the person image storage means 25.
- the person area image is an image showing a person area in the person image stored in the person image storage means 25.
- FIG. 4 is a diagram illustrating an example of a person image stored in the person image storage unit 25 and a person area image corresponding to the person image.
- FIG. 4 illustrates four examples of person images and person area images.
- the person area image for example, the area of the person shown in the person image is represented by a single color (white in the example shown in FIG. 4), and the area other than the person is represented by another single color (black in the example shown in FIG. 4). It may be an image.
- the person area image is not limited to such an example.
- the person area image may be an image that can specify a person area in the person image.
- the person area image is used for cutting out only a person from the corresponding person image (in other words, cutting out only the person area).
- the data processing apparatus 1 includes a background extraction unit 11, a person state determination unit 15, a crowd state image synthesis unit 14, and a control unit 16.
- the background extraction unit 11 selects a background image from the background image group stored in the background image storage unit 21. Further, the background extraction unit 11 calculates the aspect ratio of the crowd patch size stored in the learning local image information storage unit 22.
- the background extracting means 11 temporarily extracts a background of an appropriate position and an appropriate size from the selected background image so as to satisfy the aspect ratio. Further, the background extraction unit 11 enlarges or reduces the temporarily extracted background so as to match the crowd patch size stored in the learning local image information storage unit 22. In this manner, normalization may be described as enlarging or reducing the area extracted from the image so as to match the crowd patch size.
- the background extraction unit 11 When the background extraction unit 11 temporarily extracts an appropriate position and an appropriately sized background, an area having a random size may be extracted at a random position so as to satisfy the aspect ratio. Also, on the assumption that the size of the reference part of the person at each position in the image is known, the background extraction means 11 is used for learning in accordance with the size of the reference part that is known at each position in the image. You may obtain
- the person state determination unit 15 temporarily determines the person state based on the plural person state control instruction stored in the crowd state control instruction storage unit 23 and the individual person state control instruction stored in the person state control instruction storage unit 24. While determining, the final human state is determined based on the condition regarding the size of the reference part of the person with respect to the crowd patch size and the appearance of the reference part.
- an instruction such as “random” may be included in those instructions, so that an appropriate person state can be obtained. There may be no. In that case, determination of the state of the person who satisfies the plural person state control instruction and the individual person state control instruction is performed again. When an appropriate person state is obtained, the person state is finally determined. In this way, the determination of the person's state may be redone, so the expression “temporary determination” may be used.
- the person state determination unit 15 determines the foreground person state and the background person state. At this time, when the person state determination means 15 determines whether or not the provisionally determined foreground person state is appropriate, the size of the reference portion that is the same as the size of the reference portion of the person with respect to the crowd patch size is obtained. Whether or not the reference part appears is determined. In addition, when the person state determination unit 15 determines whether or not the temporarily determined background person state is appropriate, whether or not a reference part size significantly different from the person reference part size with respect to the crowd patch size is obtained. Alternatively, the determination is made based on the appearance of the reference part.
- the person state determination unit 15 includes a background person state determination unit 12 and a foreground person state determination unit 13.
- the background person state determination means 12 is arranged according to the plural person state control instructions stored in the crowd state control instruction storage means 23 and the individual person state control instructions stored in the person state control instruction storage means 24.
- the person who corresponds to the background by specifying the direction of the person, the number of people, the shooting angle of the person, the lighting of the person, the posture of the person, the clothes of the person, the figure of the person, the hair of the person, the size of the person when combining with the crowd patch, etc. Is temporarily determined.
- the background person state determination means 12 determines whether or not the temporarily determined person state satisfies the background person state condition. If the background person state condition does not satisfy the background person state condition, Execute tentative decision again. If the temporarily determined person's state satisfies the condition, the background person state determining means 12 finally determines the temporarily determined person's state as the state of the person corresponding to the background.
- the condition of the person state of the background is, for example, that the person's reference part is in an arrangement state that does not fit in the crowd patch, or the reference stored in the learning local image information storage unit 22 That is, the size of the reference part of the person at the time of composition corresponds to either being extremely large or extremely small relative to the size of the part.
- the state of the person corresponding to the background is finally determined based on the size of the reference part of the person with respect to the crowd patch size and the appearance of the reference part.
- the conditions listed here are merely examples, and other conditions may be used as the conditions of the background person state.
- the phrase “a person's reference part fits in the crowd patch” means a state in which an area of a predetermined ratio or more in the area representing the person's reference part appears in the crowd patch.
- the phrase “a person's reference part does not fit in the crowd patch” refers to a state in which an area less than a predetermined ratio of the area representing the person's reference part appears in the crowd patch.
- the predetermined ratio is predetermined as 80%. In this case, for example, if 85% of the region representing the reference portion is in the crowd patch, it can be said that the reference portion of the person is included in the crowd patch.
- the first threshold value representing a size larger than the size of the reference part stored in the learning local image information storage unit 22 and the reference stored in the learning local image information storage unit 22 are used.
- a second threshold value representing a size smaller than the size of the part is determined in advance.
- the size of the reference part of the person at the time of composition is the same as the size of the reference part of the person at the time of composition with respect to the size of the reference part stored in the local image information storage means 22 for learning. It means that it is not less than the second threshold value and not more than the first threshold value.
- the size of the reference part of the person at the time of composition is extremely large compared to the size of the reference part stored in the local image information storage unit 22 for learning. It means that it is larger than the first threshold value.
- the size of the reference part of the person at the time of composition is extremely small compared to the size of the reference part stored in the local image information storage unit 22 for learning. It means less than the second threshold.
- FIG. 5 is a schematic diagram showing an example in which the condition of the background person state is satisfied.
- the local size information for learning is stored in such a way that the vertical size of the human reference region (head in this example) is 1 / ⁇ times the vertical size h pixels of the crowd patch (ie, h / ⁇ pixels). Assume that it is stored in the means 22.
- the person states illustrated in FIGS. 5A and 5B are in an arrangement state in which the reference portion of the person does not appear in the crowd patch, and therefore the background person state condition is satisfied.
- the human state illustrated in FIG. 5C satisfies the condition of the background human state because the size of the reference portion is extremely small with respect to the determined size of the reference portion.
- the human state illustrated in FIG. 5D satisfies the condition of the background human state because the size of the reference part is extremely larger than the size of the determined reference part.
- the foreground person state determining means 13 is arranged according to the plural person state control instructions stored in the crowd state control instruction storage means 23 and the individual person state control instructions stored in the person state control instruction storage means 24.
- the person who corresponds to the foreground by specifying the direction of the person, the number of people, the shooting angle of the person, the lighting of the person, the posture of the person, the clothes of the person, the figure of the person, the hair style of the person, the person size when combining with the crowd patch, etc. Is temporarily determined.
- the foreground person state determination means 13 determines whether or not the temporarily determined person state satisfies the foreground person state condition, and if the foreground person state condition does not satisfy the foreground person state condition, Execute tentative decision again. If the temporarily determined person's state satisfies the condition, the foreground person state determining means 13 finally determines the temporarily determined person's state as the state of the person corresponding to the foreground.
- the condition of the foreground person state is, for example, an arrangement state in which the reference part of the person is accommodated in the crowd patch, and the size of the reference part stored in the learning local image information storage unit 22 On the other hand, the size of the reference part of the person at the time of composition is about the same.
- the state of the person corresponding to the foreground is finally determined based on the size of the reference part of the person with respect to the crowd patch size and the appearance of the reference part.
- the conditions listed here are examples, and other conditions may be used as the condition of the foreground person state.
- FIG. 6 is a schematic diagram showing an example in which the condition of the foreground person state is satisfied.
- the vertical size of the human reference region head in this example
- the vertical size of the human reference region is 1 / ⁇ times the vertical size h pixel of the crowd patch (that is, h / ⁇ pixel). It is assumed that it is stored in the learning local image information storage means 22.
- the reference part of the person is stored in the crowd patch, and the size of the reference part is stored in the learning local image information storage unit 22. It is about the same size as the part. Accordingly, any of the person states shown in FIGS. 6A to 6D satisfies the condition of the foreground person state.
- the multiple person state control instruction and the individual person state control instruction for determining the foreground person state, and the multiple person state control instruction and the individual person state control instruction for determining the background person state. May be determined separately by the operator.
- the background person state determination means 12 may tentatively determine the person state in accordance with the multiple person state control instruction and the individual person state control instruction for determining the background person state.
- the foreground person state determination means 13 may tentatively determine the person state in accordance with the multiple person state control instruction and the individual person state control instruction for determining the foreground person state.
- the number of foreground persons is determined. It is possible to change the number of people in the background.
- the crowd state image synthesizing unit 14 is the person state finally determined by the background person state determining unit 12 (the direction of the person, the number of persons, the shooting angle of the person, the lighting of the person, the posture of the person, the clothes of the person, the figure of the person) A person image satisfying the person's hairstyle, etc.) is read from the person image storage means 25, and a person area image corresponding to the person image is read from the person area image storage means 26. Then, using the person area image, the crowd state image synthesizing unit 14 cuts out an image of only the person portion from the person image (in other words, cuts out only the person area).
- the crowd state image synthesizing unit 14 reads a person image satisfying the person state finally determined by the foreground person state determining unit 13 from the person image storage unit 25, and further acquires a person area image corresponding to the person image. , Read from the person area image storage means 26. Then, the crowd state image synthesizing unit 14 uses the person region image to cut out an image of only the person portion from the person image.
- the crowd state image synthesizing unit 14 synthesizes the image of only the person portion cut out as described above with the background image. At this time, with respect to the image of only the person portion cut out based on the person state finally determined by the background person state determining unit 12, the crowd state image synthesizing unit 14 determines that “the character of the person determined by the background person state determining unit 12”. It is combined with the background image in accordance with “placement” and “person size when combining with the crowd patch”. For the image of only the person portion cut out based on the person state finally determined by the foreground person state determination unit 13, the crowd state image composition unit 14 determines the “personal arrangement” determined by the foreground person state determination unit 13. ”And“ the person size when compositing to the crowd patch ”are combined with the background image.
- this background image is an image after normalization by the background extraction means 11. The result of this synthesis is a crowd patch.
- the crowd state image combining unit 14 When the image of only the person portion is combined with the background image, the crowd state image combining unit 14 combines the images in order from the image of the person corresponding to the disposition position farther from the camera. For example, when the upper part of the image is farther from the camera, the crowd state image synthesizing unit 14 superimposes and superimposes the person images on the upper part of the screen in order. In addition, when camera calibration information is given, the crowd state image synthesis unit 14 superimposes and synthesizes the person images in order from the camera in consideration of the three-dimensional position of the person images.
- the crowd state image synthesizing unit 14 uses the person area image to cut out an image of only the person part from the person image, and synthesizes the image of only the person part with the background image.
- the crowd state image composition means 14 divides the person image read from the person image storage means 25 into a person area and other areas based on the person area image corresponding to the person image.
- the area and other areas may be weighted, and the person image may be blended according to the weight to be combined.
- the weight of the person area is set larger than the weights of the other areas. Further, the weight may be changed within the region.
- the data processing apparatus 1 may be configured to include person image generation means (not shown) that generates a person image that matches a specified person state by CG or the like.
- a person image generation unit (not shown) generates a person image that matches the person state determined by the background person state determination unit 12 or the person state determined by the foreground person state determination unit 13, and the crowd state image synthesis is performed.
- the means 14 may generate a crowd patch by combining the person images.
- the crowd state image composition unit 14 reads the teacher label from the crowd state control instruction storage unit 23 and the person state control instruction storage unit 24 when the crowd patch is generated. That is, the crowd state image synthesizing unit 14 reads the contents of the multiple person state control instruction of the item corresponding to the designation of the teacher label from the crowd state control instruction storage unit 23, and the item of the item corresponding to the designation of the teacher label. The contents of the individual person state control instruction are read from the person state control instruction storage unit 24. Then, the crowd state image composition means 14 outputs a set of crowd patches and teacher labels. The crowd patch and the teacher label are used as teacher data for machine learning for recognizing the crowd state in the image.
- the control unit 16 repeats a series of processes by the background extraction unit 11, the person state determination unit 15 (specifically, the background person state determination unit 12 and the foreground person state determination unit 13) and the crowd state image composition unit 14. .
- the data processing apparatus 1 outputs a large number of sets of crowd patches and teacher labels.
- the data processing apparatus 1 sets the plural person state control instruction, the individual person state control instruction, and whether or not the teacher label is specified, thereby resetting the data processing apparatus 1.
- a large number of sets of crowd patches and teacher labels according to the setting are output. Therefore, the operator can obtain a large amount of desired teacher data.
- FIG. 7 is a block diagram showing a configuration example of the crowd state recognition device of the present invention.
- the crowd state recognition device 30 of the present invention recognizes the crowd state in a given image.
- the crowd state recognition device 30 includes an image acquisition device 3, a data processing device 4 that operates under program control, and a storage device 5 that stores information.
- the image acquisition device 3 is a camera that acquires an image that is a recognition target of the crowd state.
- the storage device 5 includes search window storage means 51 and crowd state recognition dictionary storage means 52.
- the search window storage means 51 stores a rectangular area group indicating a recognition target location of the crowd state on the image. This rectangular area may be referred to as a search window.
- the rectangular region group includes a camera parameter indicating the position, posture, focal length, and lens distortion of the image acquisition device 3 and the size of the reference portion corresponding to the crowd patch size (the reference portion stored in the local image information storage unit for learning 22). May be set by determining the size of the crowd patch according to the position on the image. For example, from the camera parameters as described above, the size of the reference portion of the person shown in the image can be derived.
- the size of the crowd patch is enlarged or reduced by an enlargement ratio or a reduction ratio when the size of the reference part of the person stored in the local image information storage unit 22 for learning is enlarged or reduced.
- the size of the rectangular area may be set.
- a rectangular area group may be set so as to cover the positions on the image.
- the rectangular area group is not limited to these methods and may be set freely. Further, the rectangular area group may be set in an overlapping manner.
- the crowd state recognition dictionary storage means 52 stores a dictionary of discriminators learned by teacher data (a large number of sets of crowd patches and teacher labels) generated by the teacher data generation device 10 shown in FIG.
- the discriminator is an algorithm for recognizing the crowd state
- the dictionary of the discriminator is a dictionary used when performing the crowd state recognition process according to the algorithm.
- the classifier dictionary stored in the crowd state recognition dictionary storage unit 52 is obtained, for example, by performing machine learning using a large number of sets of crowd patches and teacher labels generated by the teacher data generation device 10. This machine learning may be known machine learning.
- the data processing device 4 includes crowd state recognition means 41.
- the crowd state recognition unit 41 extracts a local region image corresponding to the rectangular region group stored in the search window storage unit 51 from the image acquired by the image acquisition device 3, and the extracted local region image is converted into a crowd patch size. Normalize to match.
- the crowd state recognizing means 41 uses the classifier dictionary stored in the crowd state recognition dictionary storage means 52 according to the recognition algorithm (that is, the classifier) for the crowd state, and the crowd in the normalized local region image. Recognize (determine) the state.
- the crowd state recognition means 41 recognizes the crowd state in the local region image. Therefore, the crowd state recognition device 30 can recognize various crowd states.
- FIG. 8 is a schematic diagram showing an example of recognizing the degree of congestion (number of people) as the crowd state in the image.
- the operator of the teacher data generation apparatus 10 mainly obtains a large number of crowd patches and teacher labels by controlling the “number of people” in stages (see the upper part of FIG. 8).
- the classifier dictionary obtained by machine learning from the teacher data is stored in the crowd state recognition dictionary storage unit 52.
- a rectangular area from which a local area image is extracted is indicated by a broken line.
- the recognition result of the crowd state regarding the local area image extracted in accordance with the rectangular area is represented in correspondence with the area indicated by the broken line. This also applies to FIGS.
- the actual rectangular area is basically set comprehensively over the entire screen, but here, only a few rectangular areas are shown as an example in order to show the recognition result in an easy-to-understand manner.
- the crowd state recognition means 41 can recognize the number of people (congestion degree) in various regions within the image 61 as shown in FIG.
- FIG. 9 is a schematic diagram showing an example of recognizing the direction of the crowd as the crowd state in the image.
- the operator of the teacher data generation apparatus 10 mainly obtains a large number of crowd patches and teacher labels by controlling the “person direction” (see the upper part of FIG. 9).
- the classifier dictionary obtained by machine learning from the teacher data is stored in the crowd state recognition dictionary storage unit 52.
- the crowd state recognition means 41 can recognize the direction of the crowd in various regions within the image 62 as shown in FIG.
- FIG. 10 is a schematic diagram illustrating an example of recognizing a non-abnormal crowd (a crowd that is not extremely crowded) or an abnormal crowd (a crowd that is extremely crowded) as a crowd state in an image.
- an operator of the teacher data generation apparatus 10 mainly obtains a large number of crowd patches and teacher labels by controlling the “number of people”.
- a large amount of teacher data is obtained by dividing into two classes, when the number of persons is less than n and when the number of persons is n or more (see the upper part of FIG. 10).
- the classifier dictionary obtained by machine learning from the teacher data is stored in the crowd state recognition dictionary storage unit 52.
- the crowd state recognition means 41 can recognize whether the crowd state in various regions in the image 63 is a non-abnormal crowd or an abnormal crowd, as shown in FIG.
- FIG. 11 is a schematic diagram illustrating an example of recognizing whether the crowd state in the image is in a disordered state (a state where the direction of the person is not unified) or an ordered state (a state where the direction of the person is unified).
- the operator of the teacher data generation apparatus 10 obtains a large amount of teacher data by dividing into two classes, when the “person direction” is unified and when it is not unified (see the upper part of FIG. 11).
- the classifier dictionary obtained by machine learning from the teacher data is stored in the crowd state recognition dictionary storage unit 52.
- the crowd state recognition means 41 can recognize whether the crowd state in various regions in the image 64 is a disordered state or an ordered state, as shown in FIG.
- the crowd state recognition means 41 may have a discrete state in which the crowd is scattered, It is possible to recognize various states, such as a gathering state that gathers in one place, an avoidance state that avoids the crowd, a hangout state that indicates a special crowd of people, and a matrix state.
- FIG. 12 is a flowchart showing an example of processing progress of the teacher data generation apparatus 10.
- the background extraction unit 11 selects a background image from the background image group stored in the background image storage unit 21, and extracts an image to be used as the background of the crowd patch (step S1).
- FIG. 13 is a flowchart showing an example of processing progress of step S1.
- the background extraction unit 11 first selects one background image from the background image group stored in the background image storage unit 21 (step S101). This selection method is not particularly limited. For example, the background extraction unit 11 may arbitrarily select one background image from the background image group.
- the background extraction unit 11 calculates the aspect ratio of the crowd patch size stored in the learning local image information storage unit 22, and selects an appropriate position from the selected background image so as to satisfy the aspect ratio.
- a background having an appropriate size is temporarily extracted (step S102).
- the background extracting unit 11 obtains an image as a background of the crowd patch by enlarging or reducing the temporarily extracted background image so as to match the crowd patch size (in other words, normalizing) (step S103). ). Step S1 is complete
- the background person state determination means 12 determines the state of the person corresponding to the background (step S2).
- FIG. 14 is a flowchart showing an example of processing progress of step S2.
- the background person state determination means 12 is arranged according to the plural person state control instructions stored in the crowd state control instruction storage means 23 and the individual person state control instructions stored in the person state control instruction storage means 24.
- the person who corresponds to the background by specifying the direction of the person, the number of people, the shooting angle of the person, the lighting of the person, the posture of the person, the clothes of the person, the figure of the person, the hair of the person, the size of the person when combining with the crowd patch, etc. Is temporarily determined (step S201).
- the background person state determination means 12 determines whether or not the state of the person provisionally determined in step S201 satisfies the condition of the background person state (step S202). Since this condition has already been described, the description is omitted here.
- step S201 Since the plural person state control instruction and the individual person state control instruction may include an instruction such as “random”, the state provisionally determined in step S201 may not satisfy the background person state condition. In such a case (No in step S202), the background person state determination unit 12 repeats the processing from step S201.
- step S201 If the state provisionally determined in step S201 satisfies the condition of the background person state (Yes in step S202), the background person state determination unit 12 determines the state of the person temporarily determined in the latest step S201. Is determined as the state of the person corresponding to the background (step S203). Step S2 is complete
- the foreground person state determination means 13 determines the state of the person corresponding to the foreground (step S3).
- FIG. 15 is a flowchart showing an example of processing progress of step S3.
- the foreground person state determining means 13 is arranged according to the plural person state control instructions stored in the crowd state control instruction storage means 23 and the individual person state control instructions stored in the person state control instruction storage means 24.
- the person who corresponds to the foreground by specifying the direction of the person, the number of people, the shooting angle of the person, the lighting of the person, the posture of the person, the clothes of the person, the figure of the person, the hair style of the person, the person size when combining with the crowd patch, etc. Is temporarily determined (step S301).
- the foreground person state determination means 13 determines whether or not the state of the person provisionally determined in step S301 satisfies the foreground person state condition (step S302). Since this condition has already been described, the description is omitted here.
- the state temporarily determined in step S301 may not satisfy the condition of the foreground person state. In such a case (No in step S302), the foreground person state determination means 13 repeats the processing from step S301.
- step S301 If the state provisionally determined in step S301 satisfies the foreground person state condition (Yes in step S302), the foreground person state determination unit 13 determines the person state temporarily determined in the latest step S301. Is determined as the state of the person corresponding to the foreground (step S303). Step S3 is complete
- the crowd state image synthesizing unit 14 After step S3, the crowd state image synthesizing unit 14 generates a crowd patch based on the state of the person determined in steps S2 and S3, reads the teacher label corresponding to the crowd patch, the crowd patch and the teacher label. Are output (step S4).
- FIG. 16 is a flowchart showing an example of processing progress of step S4.
- the crowd state image synthesizing unit 14 determines the person state (person direction, number of persons, person photographing angle, person lighting, person posture, person clothes, person body shape, person hairstyle, etc. determined in steps S2 and S3. ) Is selected from the group of person images in the person image storage means 25 and read (step S401).
- the crowd state image composition means 14 reads each person area image corresponding to each person image selected in step S401 from the person area image storage means 26. For each person image, the crowd state image composition unit 14 cuts out an image of only the person portion using a person area image corresponding to the person image (step S402).
- the crowd state image synthesizing unit 14 adjusts the “personal arrangement” and “person size when synthesizing to the crowd patch” determined in steps S2 and S3 for each image of only the person portion generated in step S402.
- the arrangement state is determined (step S403).
- the crowd state image synthesizing unit 14 generates a crowd patch by synthesizing each image of only the person portion with the background image obtained in step S1 in accordance with the arrangement state (step S404).
- the crowd state image composition means 14 acquires a teacher label corresponding to the crowd patch (step S405). That is, the crowd state image synthesizing unit 14 reads the contents of the multiple person state control instruction of the item corresponding to the designation of the teacher label from the crowd state control instruction storage unit 23, and the item of the item corresponding to the designation of the teacher label. The contents of the individual person state control instruction are read from the person state control instruction storage unit 24. These read contents correspond to the teacher label.
- the crowd state image synthesizing unit 14 outputs a set of the crowd patch generated in step S404 and the teacher label acquired in step S405 (step S406). Step S4 is complete
- step S5 the control means 16 determines whether or not the number of repetitions of the processes of steps S1 to S4 has reached a predetermined number (step S5).
- the control unit 16 performs background extraction unit 11, person state determination unit 15 (specifically, background person state determination unit). 12 and the foreground person state determining means 13) and the crowd state image synthesizing means 14 are caused to execute the processes of steps S1 to S4 again.
- step S5 When the number of repetitions of the processes in steps S1 to S4 reaches a predetermined number (Yes in step S5), the process is terminated.
- ⁇ By performing the processing of steps S1 to S4 once, one set of crowd patch and teacher label is obtained. Therefore, a large amount of teacher data can be obtained by the data processing apparatus 1 repeating the processes of steps S1 to S4 a predetermined number of times. For example, if the predetermined number of times is set to 100,000, 100,000 pairs of crowd patches and teacher labels that match the multiple person state control instruction and the individual person state control instruction can be obtained.
- steps S1, S2, and S3 may be interchanged.
- FIG. 17 is a flowchart illustrating an example of processing progress of the crowd state recognition device 30.
- the image acquisition device 3 acquires an image that is a recognition target of the crowd state, and inputs the image to the crowd state recognition means 41 (step S21).
- the crowd state recognition unit 41 determines whether or not all the rectangular area groups stored in the search window storage unit 51 have been selected (step S22).
- the crowd state recognizing unit 41 selects an unselected rectangular area from the rectangular area group. Is selected (step S23).
- the crowd state recognition means 41 extracts a local area image corresponding to the selected rectangular area from the image input in step S21 (step S24). Then, the crowd state recognizing means 41 normalizes the local region image so as to match the crowd patch size (step S25).
- the crowd state recognizing means 41 recognizes the crowd state in the normalized local region image using the classifier dictionary stored in the crowd state recognition dictionary storage means 52 (step S26).
- the crowd state recognizing means 41 repeats the processing after step S22.
- the crowd state recognizing unit 41 determines that all the rectangular region groups have been selected (Yes in step S22)
- the crowd state recognition unit 41 ends the process.
- a plurality of person state control instructions (state instructions regarding a plurality of persons such as “person arrangement”, “person direction”, “number of persons”, etc.) determined by the operator, and individual Person state control instructions (“person's shooting angle”, “lighting to person”, “person's posture”, “person's clothing”, “person's body shape”, “person's hairstyle”, “when combining with a crowd patch”
- person state determination means 15 determines the states of the persons making up the crowd.
- the crowd state image synthesizing unit 14 generates a crowd patch by synthesizing the image of the person in the determined state, and reads a teacher label corresponding to the crowd patch. Then, the process of determining the state of the person, generating the crowd patch, and identifying the teacher label is repeated a predetermined number of times, so various teacher data (a combination of the crowd patch and the teacher label) of the crowd state intended by the operator can be varied. A large amount can be automatically generated.
- the classifier dictionary can be machine-learned from the teacher data.
- the crowd state recognition device 30 can easily recognize a complex crowd state in a still image by using the dictionary.
- the crowd state recognition means 41 of the crowd state recognition device 30 uses the dictionary learned based on the crowd patch representing the crowd and the teacher label corresponding to the crowd patch. Recognize the state. Therefore, the crowd state recognizing means 41 recognizes the crowd state not in the head of a person or in a single unit such as a person but in a large unit such as a crowd that is a set of persons in which a reference part is reflected. Therefore, it is also possible to recognize the crowd state in a small-sized area where head recognition and individual person recognition cannot be performed.
- the crowd state recognition means 41 recognizes the crowd state using the above dictionary (discriminator dictionary). Accordingly, the recognition accuracy of the crowd state does not depend on the frame rate. Therefore, the crowd state recognition apparatus of the present invention can recognize the crowd state in the image satisfactorily without depending on the frame rate. For example, the crowd state recognition device 30 of the present invention can recognize the crowd state in a still image even if it is a single still image.
- the teacher data generation apparatus 10 of the above embodiment also determines a person state related to “placement of persons” such as the overlapping state of persons by a plurality of person state control instructions, and generates a crowd patch representing such a person state. Generate. If machine learning is performed using such a crowd patch, a state including occlusion between persons is also learned. Therefore, by using the dictionary obtained as a result of the learning, the crowd state recognition device 30 can be used even if there is an overlap (occlusion) between persons that is difficult to recognize by head recognition or person recognition. The state can be recognized well.
- the teacher data generation device 10 includes information for instructing a person state across multiple persons (multi-person state control instruction), information for instructing a person state of each person (individual person state control instruction), and The state of the person is determined according to the above, and a crowd patch in which the person in the state is copied is generated, and a teacher label corresponding to the crowd patch is specified. Therefore, the operator can easily obtain teacher data for recognizing crowd states having different properties by determining a multiple person state control instruction or an individual person state control instruction. Then, the machine state recognition device 30 for recognizing crowd states having different properties can be easily made by machine learning of the teacher data.
- the camera parameters are used to limit the environment.
- a multiple person state control instruction and an individual person state control instruction can be determined.
- the teacher data generation apparatus 10 determines a person state based on such a multiple person state control instruction or an individual person state control instruction and generates teacher data, a dictionary of discriminators suitable for an environment for photographing a crowd is learned. be able to.
- the crowd state recognition device 30 can realize the recognition of a complex crowd state in a still image or the like with high accuracy.
- the state of the person related to a plurality of persons for each local region on the image And the person state of each person can be controlled.
- a large number of crowd patches intended by the operator and teacher labels corresponding to the crowd patches can be automatically generated by synthesizing the person images based on the controlled person state.
- Based on the crowd patch and the teacher label it is possible to learn a dictionary of classifiers for each local area on the image. By using a dictionary of multiple classifiers for each area on the image, the recognition accuracy of a complex crowd state Can be raised.
- FIG. 18 is a block diagram showing an example of a specific configuration of the teacher data generation apparatus of the present invention.
- the same elements as those shown in FIG. 1 are denoted by the same reference numerals as those in FIG.
- background image storage means 21 learning local image information storage means 22
- crowd state control instruction storage means 23 person state control instruction storage means 24, person image storage means 25,
- a storage device 2 including a person area image storage means 26 is connected to the computer 100.
- a computer-readable storage medium 102 that stores the teacher data generation program 101 is also connected to the computer 100.
- the computer-readable storage medium 102 is realized by, for example, a magnetic disk or a semiconductor memory.
- the computer 100 reads the teacher data generation program 101 from the computer-readable storage medium 102, for example, at startup. Then, in accordance with the teacher data generation program 101, the computer 100 performs background extraction means 11, person state determination means 15 (more specifically, background person state determination means 12 and foreground person state in the data processing apparatus 1 shown in FIG. It operates as a determination means 13), a crowd state image composition means 14 and a control means 16.
- FIG. 19 is a block diagram showing an example of a specific configuration of the crowd state recognition device of the present invention.
- the same elements as those shown in FIG. 7 are denoted by the same reference numerals as those in FIG. 7, and detailed description thereof is omitted.
- a storage device 5 including a search window storage unit 51 and a crowd state recognition dictionary storage unit 52 is connected to a computer 150.
- a computer readable storage medium 104 that stores the crowd state recognition program 103 is also connected to the computer 150.
- the computer-readable storage medium 104 is realized by, for example, a magnetic disk or a semiconductor memory.
- the computer 150 reads the crowd state recognition program 103 from the computer-readable storage medium 104 at the time of startup, for example.
- the computer 150 operates as crowd state recognition means 41 in the data processing device 4 shown in FIG. 7 according to the crowd state recognition program 103.
- the crowd state recognition dictionary storage unit 52 stores a dictionary obtained by learning using the teacher data generated in the teacher data generation device 10 (see FIG. 1).
- the case has been described as an example.
- a plurality of sets of crowd patches obtained by synthesizing images of persons matching the state of a person controlled to a desired state and teacher labels for the crowd patches are used.
- the case where the dictionary obtained by performing machine learning is stored in the crowd state recognition dictionary storage means 52 is shown.
- the crowd state recognition dictionary storage means 52 may store a dictionary obtained by machine learning using data other than the teacher data generated by the teacher data generation device 10 as teacher data. Even for teacher data other than the teacher data generated by the teacher data generation device 10, a person whose reference part is represented with a size similar to the size of the person's reference part determined with respect to the size of the crowd patch. A plurality of sets of crowd patches to be included and teacher labels for the crowd patches may be prepared and used as teacher data. That is, a classifier dictionary obtained as a result of machine learning using a plurality of sets of such crowd patches and teacher labels may be stored in the crowd state recognition dictionary storage unit 52. Even in such a case, there is an effect that the crowd state in the image can be well recognized without depending on the frame rate.
- FIG. 20 is a block diagram showing the main part of the teacher data generation apparatus of the present invention.
- the teacher data generation apparatus of the present invention includes a background extraction unit 71, a person state determination unit 72, and a crowd state image composition unit 73.
- the background extraction unit 71 selects a background image from a plurality of background images prepared in advance, extracts an area in the background image, and sets an image corresponding to the extracted area to a predetermined size. Enlarge or reduce the image.
- the person state determination unit 72 (for example, the person state determination unit 15) includes a plurality of person state control instructions which are instruction information of person states related to a plurality of persons, and an instruction of the states of individual persons among the plurality of persons.
- the crowd person state is determined according to the individual person state control instruction which is information.
- the crowd state image composition unit 73 is a crowd state image (image obtained by combining the image of the person corresponding to the person state determined by the person state determination unit 72 with the image of the predetermined size obtained by the background extraction unit 71 ( For example, a crowd patch) is generated, a teacher label for the crowd state image is specified, and a set of the crowd state image and the teacher label is output.
- the background extraction unit 71, the person state determination unit 72, and the crowd state image composition unit 73 repeat operations in order. However, the operations of the background extraction unit 71, the person state determination unit 72, and the crowd state image composition unit 73 may not be in order. For example, the background extraction unit 71 and the person state determination unit 72 may operate in parallel.
- FIG. 21 is a block diagram showing the main part of the crowd state recognition device of the present invention.
- the crowd state recognition device of the present invention includes a rectangular area group storage unit 81, a crowd state recognition dictionary storage unit 82, and a crowd state recognition unit 83.
- the rectangular area group storage unit 81 (for example, the search window storage unit 51) stores a rectangular area group indicating a recognition target location of the crowd state on the image.
- the crowd state recognition dictionary storage unit 82 (for example, the crowd state recognition dictionary storage unit 52) is an image of a predetermined size representing the crowd state, and has the same size as the reference portion of the person defined for the predetermined size. It is obtained by performing machine learning using a plurality of sets of a crowd state image (for example, a crowd patch) that is an image including a person whose reference part is represented with a certain size and a teacher label for the crowd state image. Store the identified classifier dictionary.
- a crowd state image for example, a crowd patch
- the crowd state recognizing unit 83 (for example, the crowd state recognizing unit 41) extracts each region indicated by the rectangular region group stored in the rectangular region group storage unit 81 from the given image, and extracts based on the dictionary. Recognize the state of the crowd in the image.
- the crowd state in the image can be recognized well without depending on the frame rate.
- a background extraction unit that selects a background image from a plurality of background images prepared in advance, extracts an area in the background image, and expands or reduces an image corresponding to the extracted area to an image of a predetermined size;
- a person state determining means for determining;
- a crowd state image which is an image obtained by combining an image of a person corresponding to the person state determined by the person state determination unit with an image of a predetermined size obtained by the background extraction unit, and a teacher label for the crowd state image
- a crowd state image composition means for outputting a set of crowd state images and teacher labels.
- the person state determination means In accordance with the multiple person state control instruction and the individual person state control instruction, the person state of the crowd is provisionally determined, and the provisionally determined person state is the size of the reference part of the person determined for a predetermined size and the appearance of the reference part When the condition regarding the direction is satisfied, the temporarily determined person state is determined as the crowd person state, and when the temporarily determined person state does not satisfy the above condition, the crowd person state is temporarily determined again.
- the teacher data generation device according to appendix 1.
- (Appendix 3) A plurality of person state control instructions determined for each item, and a crowd state control instruction storage means for storing presence / absence of designation of a teacher label specified for each item;
- a personal state control instruction storage means for storing individual person state control instructions determined for each item, and storing presence / absence of designation of a teacher label determined for each item,
- the person state determining means determines the person state of the crowd according to the plural person state control instructions stored in the crowd state control instruction storage means and the individual person state control instructions stored in the person state control instruction storage means,
- the crowd state image composition means A plurality of person state control instructions for an item determined to have a teacher label specified are read from the crowd state control instruction storage means, and an individual person state control instruction for an item determined to have a teacher label specified are read from the person state control instruction storage means.
- the teacher data generation device according to attachment 1 or 2, wherein the teacher label is specified by reading.
- the crowd state control instruction storage means For at least one item, memorize the teacher label designation as valid,
- the crowd state image composition means The teacher data generation device according to appendix 3, wherein a plurality of person state control instructions for an item determined to have a teacher label specified are read from a crowd state control instruction storage unit.
- the crowd state control instruction storage means stores a plurality of person state control instructions and the presence / absence of designation of a teacher label for each item relating to person arrangement, person direction, and number of persons, and a plurality of person state control instructions corresponding to each item.
- the person state control instruction storage means is individually provided for each item relating to the shooting angle of the person, the lighting of the person, the posture of the person, the clothes of the person, the figure of the person, the hair of the person, and the person size when combining with the crowd state image.
- the personal state control instruction and the presence / absence of designation of the teacher label are stored, and the individual person state control instruction corresponding to each item is stored in any one of the first aspect, the second aspect, and the third aspect.
- the person state determining means determines the person state of the crowd according to the plural person state control instruction stored in the crowd state control instruction storage means and the individual person state control instruction stored in the person state control instruction storage means. 4.
- the teacher data generation device according to 4.
- the crowd state image composition means determines the person's state, the number of persons, the person's shooting angle, the lighting of the person, the person's posture, the person's clothes, the person's body shape, and the person's hairstyle An image is selected from a group of human images prepared in advance, and an image of only the human part is generated by cutting out the human region from the selected human image, and the human image determined as the human state is generated. According to the arrangement and the person size when combining with the crowd state image, only the person portion is combined with the image of the predetermined size obtained by the background extraction unit.
- the teacher data generation device according to claim 1.
- Appendix 7 The teacher data generation device according to appendix 6, wherein the crowd state image synthesis unit synthesizes an image of a predetermined size obtained by the background extraction unit in order from an image of only a person portion corresponding to an arrangement position farther from the camera.
- the person state determination means In accordance with the multi-person state control instruction and the individual person state control instruction, the crowd state image is temporarily determined as the background of the crowd, and the provisionally determined person state is determined for a predetermined size that is the size of the crowd state image.
- the temporarily determined person state is determined as the person state of the crowd as the background, and the temporarily determined person state is
- a background person state determination unit that repeats tentatively determining the person state of the crowd as a background again
- the provisionally determined person state is determined as the person state of the crowd as the foreground, and the provisionally determined person state is The teacher according to any one of appendix 1 to appendix 7, further comprising foreground person state determining means for repeatedly determining the person state of the crowd as the foreground again when the second condition is not satisfied Data generator.
- the first condition is that the person's reference part does not fit in the crowd state image, or the size of the reference part is extremely larger than the size of the person's reference part determined for a predetermined size. Or being in an extremely small state,
- the second condition is that the reference part of the person is included in the crowd state image, and the size of the reference part is approximately the same as the size of the reference part of the person determined for a predetermined size.
- Rectangular area group storage means for storing a rectangular area group indicating a recognition target location of the crowd state on the image;
- a crowd state image that is an image of a predetermined size that represents a crowd state and that includes a person whose reference portion is represented in a size similar to the size of the reference portion of the person defined for the predetermined size
- a crowd state recognition dictionary storage means for storing a dictionary of classifiers obtained by performing machine learning using a plurality of sets of teacher labels for the crowd state image,
- a crowd state recognizing unit for extracting each region indicated by the rectangular region group stored in the rectangular region group storage unit from the given image, and recognizing the crowd state in the extracted image based on the dictionary;
- a crowd state recognition device characterized by comprising:
- the crowd state recognition dictionary storage means sets a plurality of sets of a crowd state image obtained by synthesizing a person image that matches a state of a person controlled to a desired state, and a teacher label for the crowd state image. Using the machine learning to memorize the classifier dictionary, The crowd state recognition device according to claim 10, wherein the crowd state recognition means recognizes the state of the crowd in the image based on the dictionary.
- the rectangular area group storage means determines the size based on the camera parameters indicating the position, posture, focal length, and lens distortion of the image acquisition device that acquires the image, and the size of the reference portion of the person determined for the predetermined size. Stores a set of rectangular areas, The crowd state recognition device according to claim 10 or 11, wherein the crowd state recognition unit extracts regions indicated by the rectangular region group from a given image.
- the crowd state recognition dictionary storage means is obtained by changing the number of persons represented in the crowd state image and performing machine learning using a plurality of pairs of crowd state images and teacher labels prepared for each number of persons.
- the dictionary of discriminators The crowd state recognition device according to any one of supplementary note 10 to supplementary note 12, wherein the crowd state recognition means recognizes the number of crowds in the image based on the dictionary.
- the crowd state recognition dictionary storage means changes the direction of the person represented in the crowd state image, and performs machine learning using a pair of the crowd state image and the teacher label prepared for each person direction. Store the resulting classifier dictionary, The crowd state recognition device according to any one of supplementary note 10 to supplementary note 13, wherein the crowd state recognition unit recognizes the direction of the crowd in the image based on the dictionary.
- the crowd state recognition dictionary storage means is obtained by performing machine learning using a set of crowd state images and teacher labels prepared for a crowd that is not extremely crowded and a crowd that is extremely crowded.
- the dictionary of discriminators The crowd state recognition device according to any one of supplementary note 10 to supplementary note 14, wherein the crowd state recognition means recognizes whether or not the crowd shown in the image is extremely crowded based on the dictionary.
- the crowd state recognition dictionary storage means uses machine learning using a set of a crowd state image and a teacher label each of which is prepared for a crowd in which the direction of the person is unified and a crowd in which the direction of the person is not unified.
- the dictionary of classifiers obtained by The crowd state recognizing device recognizes whether or not the direction of the person is unified in the crowd shown in the image based on the dictionary, The crowd state recognition device according to any one of appendix 10 to appendix 15 .
- a person state determination step to be determined;
- a crowd state image which is an image obtained by synthesizing an image of a person corresponding to the person state determined in the person state determination step with an image of a predetermined size obtained in the background extraction step, and a teacher label for the crowd state image
- a crowd state image synthesizing step for outputting a set of crowd state images and teacher labels.
- the person state of the crowd is provisionally determined, and the provisionally determined person state is the size of the reference part of the person determined for a predetermined size and the appearance of the reference part
- the temporarily determined person state is determined as the crowd person state, and when the temporarily determined person state does not satisfy the above condition, the crowd person state is temporarily determined again.
- the crowd state control instruction storage means stores a plurality of person state control instructions determined for each item, stores the presence / absence of designation of a teacher label determined for each item
- the person state control instruction storage means stores the individual person state control instruction determined for each item, stores the presence / absence of designation of a teacher label determined for each item
- In the person state determination step determine the person state of the crowd according to the multiple person state control instruction stored in the crowd state control instruction storage means and the individual person state control instruction stored in the person state control instruction storage means
- the crowd state image composition step A plurality of person state control instructions for an item determined to have a teacher label specified are read from the crowd state control instruction storage means, and an individual person state control instruction for an item determined to have a teacher label specified are read from the person state control instruction storage means.
- the teacher data generation method according to appendix 17 or appendix 18, wherein the teacher label is specified by reading.
- the crowd state control instruction storage means stores the designation of the teacher label as valid for at least one item, In the crowd state image composition step, The teacher data generation method according to appendix 19, wherein a plural person state control instruction of an item determined to have a teacher label specified is read from the crowd state control instruction storage unit.
- the crowd state control instruction storage means stores the plural person state control instruction and the presence / absence of designation of the teacher label for each item relating to the person arrangement, the person direction, and the number of persons, and the plural person state control instruction corresponding to each item , A first mode for instructing a specific state, a second mode for instructing that an arbitrary state may be set, and a third mode for instructing that a state may be set within a set rule range
- the personal state control instruction and the presence / absence of designation of the teacher label are stored, and the individual person state control instruction corresponding to each item is stored in any one of the first aspect, the second aspect, and the third aspect.
- the person state of the crowd is determined according to the multiple person state control
- a person who matches the person's direction, number of persons, person's shooting angle, person's lighting, person's posture, person's clothes, person's body shape, and person's hairstyle determined as the person's state in the crowd state image composition step An image is selected from a group of human images prepared in advance, and an image of only the human part is generated by cutting out the human region from the selected human image, and the human image determined as the human state is generated. According to the arrangement and the person size when combining with the crowd state image, the image of only the person portion is combined with the image of the predetermined size obtained by the background extraction unit.
- the teacher data generation method according to the above.
- the person state determination step In accordance with the multi-person state control instruction and the individual person state control instruction, the crowd state image is temporarily determined as the background of the crowd, and the provisionally determined person state is determined for a predetermined size that is the size of the crowd state image.
- the temporarily determined person state is determined as the person state of the crowd as the background, and the temporarily determined person state is
- a background person state determination step that repeats tentatively determining the person state of the crowd as a background again;
- the person state of the crowd that is the foreground in the crowd state image is provisionally determined, and the provisionally determined person state is determined for a predetermined size that is the size of the crowd state image.
- the provisionally determined person state is determined as the person state of the crowd as the foreground, and the provisionally determined person state is 24.
- the first condition is that the person's reference part does not fit in the crowd state image, or the size of the reference part is extremely larger than the size of the person's reference part determined for a predetermined size. Or being in an extremely small state,
- the second condition is that the reference part of the person is included in the crowd state image, and the size of the reference part is approximately the same as the size of the reference part of the person determined for the predetermined size.
- a rectangular area group storage means stores a rectangular area group indicating a recognition target location of the crowd state on the image
- a person whose crowd state recognition dictionary storage means is an image of a predetermined size representing the crowd state, and in which the reference part is represented with a size similar to the size of the reference part of the person defined for the predetermined size
- Storing a dictionary of classifiers obtained by performing machine learning using a plurality of sets of a crowd state image that is an image including a teacher label for the crowd state image A crowd state recognition step of extracting each of the regions indicated by the rectangular region group stored in the rectangular region group storage means from the given image, and recognizing the crowd state reflected in the extracted image based on the dictionary
- a crowd state recognition method comprising:
- the crowd state recognition dictionary storage means sets a plurality of pairs of a crowd state image obtained by synthesizing a person image that matches the state of the person controlled to a desired state and a teacher label for the crowd state image. Using the machine learning to memorize the classifier dictionary, 27.
- Appendix 28 Based on the camera parameters indicating the position, posture, focal length, and lens distortion of the image acquisition device that acquires the image, and the size of the reference region of the person determined for the predetermined size Stores a set of rectangular areas, 28.
- the crowd state recognition dictionary storage means is obtained by performing machine learning using a set of crowd state images and teacher labels prepared for each number of people by changing the number of persons represented in the crowd state image.
- the dictionary of discriminators The crowd state recognition method according to any one of appendix 26 to appendix 28, wherein in the crowd state recognition step, the number of crowds in the image is recognized based on the dictionary.
- the crowd state recognition dictionary storage means changes the direction of the person represented in the crowd state image, and performs machine learning using a plurality of pairs of crowd state images and teacher labels prepared for each person direction. Store the resulting classifier dictionary, The crowd state recognition method according to any one of appendix 26 to appendix 29, wherein in the crowd state recognition step, the direction of the crowd shown in the image is recognized based on the dictionary.
- the crowd state recognition dictionary storage means is obtained by performing machine learning using a set of crowd state images and teacher labels prepared for a crowd that is not extremely crowded and a crowd that is extremely crowded.
- the dictionary of discriminators The crowd state recognition method according to any one of supplementary note 26 to supplementary note 30, wherein, in the crowd state recognition step, it is recognized based on the dictionary whether the crowd shown in the image is extremely crowded.
- Machine state recognition dictionary storage means using a set of crowd state images and teacher labels prepared for each of a crowd in which the direction of the person is unified and a crowd in which the direction of the person is not unified.
- the dictionary of classifiers obtained by The crowd state recognition method according to any one of appendix 26 to appendix 31, wherein in the crowd state recognition step, based on the dictionary, it is recognized whether or not the direction of the person is unified in the crowd shown in the image .
- a background extraction process for selecting a background image from a plurality of background images prepared in advance, extracting an area in the background image, and enlarging or reducing an image corresponding to the extracted area to an image of a predetermined size;
- the person state determination process to be determined, and
- a crowd state image which is an image obtained by combining an image of a person corresponding to the person state determined in the person state determination process with an image of a predetermined size obtained in the background extraction process, and a teacher label for the crowd state image
- a teacher data generation program for executing a crowd state image synthesis process for specifying a crowd state image and a set of teacher labels.
- the person state of the crowd is provisionally determined, and the provisionally determined person state is the size of the reference part of the person determined for a predetermined size and the appearance of the reference part
- the temporarily determined person state is determined as the crowd person state, and when the temporarily determined person state does not satisfy the above condition, the crowd person state is temporarily determined again.
- the teacher data generation program according to attachment 33 which is repeated.
- Appendix 35 A plurality of person state control instructions determined for each item, and a crowd state control instruction storage means for storing the presence / absence of designation of a teacher label specified for each item, and individual person state control determined for each item
- a computer comprising a person state control instruction storage means for storing instructions and storing the presence / absence of designation of a teacher label determined for each item
- the person state of the crowd is determined according to the multiple person state control instruction stored in the crowd state control instruction storage unit and the individual person state control instruction stored in the person state control instruction storage unit
- the crowd state image composition process A plurality of person state control instructions for an item determined to have a teacher label specified are read from the crowd state control instruction storage means, and an individual person state control instruction for an item determined to have a teacher label specified are read from the person state control instruction storage means.
- the teacher data generation program according to appendix 33 or appendix 34, wherein the teacher label is specified by reading.
- At least one item is provided with a computer having a crowd state control instruction storage means for storing the designation of the teacher label as being, In the crowd state image composition process, 36.
- the teacher data generation program according to appendix 35 wherein a plurality of person state control instructions for an item determined to have a teacher label specified are read from the crowd state control instruction storage means.
- the first mode to be stored, the second mode to instruct that it can be set to any state, and the third mode to instruct that the state may be set within the range of the set rule are stored in any mode
- the individual person state control instruction and the presence / absence of designation of the teacher label are stored, and the individual person state control instruction corresponding to each item is stored in any one of the first aspect, the second aspect, and the third aspect.
- a computer with a subject-state control instruction storing means for ⁇ In the person state determination processing, the person state of the crowd is determined according to the multiple person state control instruction stored in the crowd state control instruction storage means and the individual person state control instruction stored in the person state control instruction storage means.
- a person who matches the person's direction, number of persons, person's shooting angle, person's lighting, person's posture, person's clothing, person's body shape, and person's hairstyle, determined as the person's state in the crowd state image composition process By selecting an image from a group of human images prepared in advance and cutting out the person's area from the selected human image, an image of only the human part is generated, and the person's state determined as the human state is determined. In accordance with the arrangement and the person size when combining with the crowd state image, only the person portion is combined with an image of a predetermined size obtained by the background extraction process.
- Appendix 39 On the computer, 39.
- the teacher data generation program according to appendix 38 wherein in the crowd state image synthesis process, an image of a predetermined size obtained by the background extraction process is synthesized in order from an image of only a person portion corresponding to an arrangement position farther from the camera.
- the crowd state image is temporarily determined as the background of the crowd, and the provisionally determined person state is determined for a predetermined size that is the size of the crowd state image.
- the temporarily determined person state is determined as the person state of the crowd as the background, and the temporarily determined person state is If the first condition is not satisfied, a background person state determination process that repeats tentatively determining the person state of the crowd as a background again, and According to the multi-person state control instruction and the individual person state control instruction, the person state of the crowd that is the foreground in the crowd state image is provisionally determined, and the provisionally determined person state is determined for a predetermined size that is the size of the crowd state image.
- the provisionally determined person state is determined as the person state of the crowd as the foreground, and the provisionally determined person state is The teacher according to any one of supplementary note 33 to supplementary note 39, which executes a foreground person state determination process that repeats tentatively determining a person state of a crowd as a foreground again when the second condition is not satisfied Data generation program.
- the first condition is that the person's reference part does not fit in the crowd state image, or the size of the reference part is extremely larger than the size of the person's reference part determined for a predetermined size. Or being in an extremely small state,
- the second condition is that the reference part of the person is included in the crowd state image, and the size of the reference part is approximately the same as the size of the reference part of the person determined for the predetermined size.
- a rectangular area group storage means for storing a rectangular area group indicating a recognition target location of the crowd state on the image, and a predetermined size image representing the crowd state, the reference portion of the person defined for the predetermined size Obtained by performing machine learning using a plurality of sets of a crowd state image, which is an image including a person whose reference part is represented with a size similar to the size of the above, and a teacher label for the crowd state image
- a computer having crowd status recognition dictionary storage means for storing a dictionary of discriminators,
- a crowd state recognition process is performed for extracting regions indicated by the rectangular region group stored in the rectangular region group storage unit from the given image, and recognizing the crowd state in the extracted image based on the dictionary.
- a crowd status recognition program for execution.
- Appendix 43 By performing machine learning using a plurality of sets of a crowd state image obtained by synthesizing a person image that matches the state of a person controlled to a desired state and a teacher label for the crowd state image
- a computer provided with a crowd state recognition dictionary storage means for storing a dictionary of the obtained classifier, 43.
- the crowd state recognition program according to appendix 42 wherein in the crowd state recognition process, the state of the crowd shown in the image is recognized based on the dictionary.
- Appendix 44 A rectangular region group whose size is determined based on the camera parameters indicating the position, posture, focal length, and lens distortion of the image acquisition device that acquires the image, and the size of the reference portion of the person determined for the predetermined size In a computer having a rectangular area group storage means for storing 45.
- Appendix 45 The number of persons represented in the crowd state image is changed, and a classifier dictionary obtained by performing machine learning using a set of crowd state images and teacher labels prepared for each number of persons is stored.
- Appendix 46 A classifier dictionary obtained by performing machine learning using a set of crowd status images and teacher labels prepared for each person direction by changing the direction of the person represented in the crowd status image.
- a computer having a crowd state recognition dictionary storage means for storing, 46.
- the crowd state recognition program according to any one of appendix 42 to appendix 45, wherein the crowd state recognition process recognizes the direction of the crowd in the image based on the dictionary.
- Appendix 47 Stores a classifier dictionary obtained by performing machine learning using a set of crowd status images and teacher labels prepared for each of a crowd that is not extremely crowded and a crowd that is extremely crowded A computer equipped with a crowd state recognition dictionary storage means, 47.
- the crowd state recognition program according to any one of appendix 42 to appendix 46, wherein in the crowd state recognition process, it is recognized based on the dictionary whether the crowd shown in the image is extremely crowded.
- Appendix 48 Discrimination obtained by performing machine learning using a set of crowd status images and teacher labels prepared for each of a crowd in which the direction of the person is unified and a crowd in which the direction of the person is not unified
- a computer having a crowd state recognition dictionary storage means for storing a dictionary of vessels The crowd state recognition program according to any one of appendix 42 to appendix 47, wherein in the crowd state recognition process, based on the dictionary, it is recognized whether or not the direction of the person is unified within the crowd shown in the image. .
- the present invention can be suitably applied to a teacher data generation apparatus that generates teacher data when learning a classifier dictionary used for recognition of a crowd state.
- the present invention is preferably applied to a crowd state recognition device that recognizes a crowd state in an image.
- a crowd state recognition device that recognizes a crowd state in an image.
- it can be suitably used for recognition of a crowd state in a still image or an image with a low frame rate.
- the present invention can be suitably used when the frame rate is not stable and the crowd state recognition process using the time information cannot be performed.
- it can be suitably used for processing for recognizing a complicated crowd state including overlapping of persons from still images.
- the present invention can also be used for suspicious person recognition, suspicious object leaving recognition, gate co-recognition, abnormal state recognition, abnormal behavior recognition, etc. in the field of surveillance that recognizes the crowd state from images obtained by cameras. It is.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
Description
予め用意された複数の背景画像から背景画像を選択し、前記背景画像中の領域を抽出し、抽出した領域に該当する画像を、所定サイズの画像に拡大または縮小する背景抽出手段と、
複数の人物に関係する人物の状態の指示情報である複数人物状態制御指示と、前記複数の人物中の個別の人物の状態の指示情報である個別人物状態制御指示とに従って、群衆の人物状態を決定する人物状態決定手段と、
背景抽出手段によって得られた所定サイズの画像に対して、人物状態決定手段が決定した人物状態に該当する人物の画像を合成した画像である群衆状態画像を生成し、当該群衆状態画像に対する教師ラベルを特定し、群衆状態画像および教師ラベルの組を出力する群衆状態画像合成手段とを備えることを特徴とする教師データ生成装置。
人物状態決定手段は、
複数人物状態制御指示と個別人物状態制御指示とに従って、群衆の人物状態を仮決定し、仮決定した人物状態が、所定サイズに対して定められた人物の基準部位のサイズおよび前記基準部位の表れ方に関する条件を満たしている場合に、仮決定した人物状態を群衆の人物状態として決定し、仮決定した人物状態が前記条件を満たしていない場合に、再度群衆の人物状態を仮決定することを繰り返す
付記1に記載の教師データ生成装置。
項目毎に定められた複数人物状態制御指示を記憶するとともに、前記項目毎に定められた教師ラベルの指定の有無を記憶する群衆状態制御指示記憶手段と、
項目毎に定められた個別人物状態制御指示を記憶するとともに、前記項目毎に定められた教師ラベルの指定の有無を記憶する人物状態制御指示記憶手段とを備え、
人物状態決定手段は、群衆状態制御指示記憶手段に記憶された複数人物状態制御指示および人物状態制御指示記憶手段に記憶された個別人物状態制御指示に従って、群衆の人物状態を決定し、
群衆状態画像合成手段は、
教師ラベルの指定ありと定められた項目の複数人物状態制御指示を群衆状態制御指示記憶手段から読み込み、教師ラベルの指定ありと定められた項目の個別人物状態制御指示を人物状態制御指示記憶手段から読み込むことによって、教師ラベルを特定する
付記1または付記2に記載の教師データ生成装置。
群衆状態制御指示記憶手段は、
少なくとも1つの項目について、教師ラベルの指定をありとして記憶し、
群衆状態画像合成手段は、
教師ラベルの指定ありと定められた項目の複数人物状態制御指示を群衆状態制御指示記憶手段から読み込む
付記3に記載の教師データ生成装置。
群衆状態制御指示記憶手段は、人物の配置、人物の方向、および人数に関する項目毎に、複数人物状態制御指示および教師ラベルの指定の有無を記憶するとともに、各項目に対応する複数人物状態制御指示を、具体的な状態を指示する第1の態様、任意の状態に定めてよい旨を指示する第2の態様、定められたルールの範囲内で状態を定めてよいことを指示する第3の態様のいずれかの態様で記憶し、
人物状態制御指示記憶手段は、人物の撮影角度、人物への照明、人物の姿勢、人物の服装、人物の体型、人物の髪型、群衆状態画像に合成するときの人物サイズに関する項目毎に、個別人物状態制御指示および教師ラベルの指定の有無を記憶するとともに、各項目に対応する個別人物状態制御指示を、前記第1の態様、前記第2の態様、前記第3の態様のいずれかの態様で記憶し、
人物状態決定手段は、群衆状態制御指示記憶手段に記憶された複数人物状態制御指示および人物状態制御指示記憶手段に記憶された個別人物状態制御指示に従って、群衆の人物状態を決定する
付記3または付記4に記載の教師データ生成装置。
群衆状態画像合成手段は、人物の状態として決定された、人物の方向、人数、人物の撮影角度、人物への照明、人物の姿勢、人物の服装、人物の体型、人物の髪型に合致する人物画像を、予め用意された人物画像群の中から選択し、選択した人物画像の中から、人物の領域を切り出すことによって、人物部分のみの画像を生成し、人物の状態として決定された人物の配置、および群衆状態画像に合成するときの人物サイズに合わせて、前記人物部分のみを画像を、背景抽出手段によって得られた所定サイズの画像に対して合成する
付記1から付記5のうちのいずれかに記載の教師データ生成装置。
群衆状態画像合成手段は、カメラからより遠い配置位置に該当する人物部分のみの画像から順に、背景抽出手段によって得られた所定サイズの画像に対して合成する
付記6に記載の教師データ生成装置。
人物状態決定手段は、
複数人物状態制御指示と個別人物状態制御指示とに従って、群衆状態画像で背景となる群衆の人物状態を仮決定し、仮決定した人物状態が、群衆状態画像のサイズである所定サイズに対して定められた人物の基準部位のサイズおよび前記基準部位の表れ方に関する第1の条件を満たしている場合に、仮決定した人物状態を背景となる群衆の人物状態として決定し、仮決定した人物状態が前記第1の条件を満たしていない場合に、再度、背景となる群衆の人物状態を仮決定することを繰り返す背景人物状態決定手段と、
複数人物状態制御指示と個別人物状態制御指示とに従って、群衆状態画像で前景となる群衆の人物状態を仮決定し、仮決定した人物状態が、群衆状態画像のサイズである所定サイズに対して定められた人物の基準部位のサイズおよび前記基準部位の表れ方に関する第2の条件を満たしている場合に、仮決定した人物状態を前景となる群衆の人物状態として決定し、仮決定した人物状態が前記第2の条件を満たしていない場合に、再度、前景となる群衆の人物状態を仮決定することを繰り返す前景人物状態決定手段とを含む
付記1から付記7のうちのいずれかに記載の教師データ生成装置。
第1の条件は、人物の基準部位が群衆状態画像に納まらない状態であること、あるいは、所定サイズに対して定められた人物の基準部位のサイズに対して、基準部位のサイズが極端に大きいか、または、極端に小さい状態であることのいずれかに該当することであり、
第2の条件は、人物の基準部位が群衆状態画像に納まり、かつ、前記基準部位のサイズが、所定サイズに対して定められた人物の基準部位のサイズと同程度であることである
付記8に記載の教師データ生成装置。
画像上の群衆状態の認識対象箇所を示す矩形領域群を記憶する矩形領域群記憶手段と、
群衆状態を表した所定サイズの画像であって、前記所定サイズに対して定められた人物の基準部位のサイズと同程度のサイズで基準部位が表されている人物を含む画像である群衆状態画像と、当該群衆状態画像に対する教師ラベルとの組を複数組用いて機械学習を行うことによって得られた識別器の辞書を記憶する群衆状態認識辞書記憶手段と、
与えられた画像から、矩形領域群記憶手段に記憶される矩形領域群が示す領域をそれぞれ抽出し、前記辞書に基づいて、抽出した画像に写っている群衆の状態を認識する群衆状態認識手段とを備える
ことを特徴とする群衆状態認識装置。
群衆状態認識辞書記憶手段は、所望の状態に制御された人物の状態に合致する人物の画像を合成することによって得られた群衆状態画像と、当該群衆状態画像に対する教師ラベルとの組を複数組用いて機械学習を行うことによって得られた識別器の辞書を記憶し、
群衆状態認識手段は、前記辞書に基づいて、画像に写っている群衆の状態を認識する
付記10に記載の群衆状態認識装置。
矩形領域群記憶手段は、画像を取得する画像取得装置の位置、姿勢、焦点距離、レンズ歪みを示すカメラパラメータと、所定サイズに対して定められた人物の基準部位のサイズとに基づいて、サイズが定められた矩形領域群を記憶し、
群衆状態認識手段は、与えられた画像から、前記矩形領域群が示す領域をそれぞれ抽出する
付記10または付記11に記載の群衆状態認識装置。
群衆状態認識辞書記憶手段は、群衆状態画像に表される人物の人数を変えて、人数毎に複数組用意された群衆状態画像と教師ラベルとの組を用いて機械学習を行うことによって得られた識別器の辞書を記憶し、
群衆状態認識手段は、前記辞書に基づいて、画像に写っている群衆の人数を認識する
付記10から付記12のうちのいずれかに記載の群衆状態認識装置。
群衆状態認識辞書記憶手段は、群衆状態画像に表される人物の方向を変えて、人物の方向毎に複数組用意された群衆状態画像と教師ラベルとの組を用いて機械学習を行うことによって得られた識別器の辞書を記憶し、
群衆状態認識手段は、前記辞書に基づいて、画像に写っている群衆の方向を認識する
付記10から付記13のうちのいずれかに記載の群衆状態認識装置。
群衆状態認識辞書記憶手段は、極端に混雑していない群衆と、極端に混雑した群衆とについてそれぞれ複数組用意された群衆状態画像と教師ラベルとの組を用いて機械学習を行うことによって得られた識別器の辞書を記憶し、
群衆状態認識手段は、前記辞書に基づいて、画像に写っている群衆が極端に混雑しているか否かを認識する
付記10から付記14のうちのいずれかに記載の群衆状態認識装置。
群衆状態認識辞書記憶手段は、人物の方向が統一されている群衆と、人物の方向が統一されていない群衆とについてそれぞれ複数組用意された群衆状態画像と教師ラベルとの組を用いて機械学習を行うことによって得られた識別器の辞書を記憶し、
群衆状態認識手段は、前記辞書に基づいて、画像に写っている群衆内で人物の方向が統一されているか否かを認識する
付記10から付記15のうちのいずれかに記載の群衆状態認識装置。
予め用意された複数の背景画像から背景画像を選択し、前記背景画像中の領域を抽出し、抽出した領域に該当する画像を、所定サイズの画像に拡大または縮小する背景抽出ステップと、
複数の人物に関係する人物の状態の指示情報である複数人物状態制御指示と、前記複数の人物中の個別の人物の状態の指示情報である個別人物状態制御指示とに従って、群衆の人物状態を決定する人物状態決定ステップと、
背景抽出ステップで得られた所定サイズの画像に対して、人物状態決定ステップで決定した人物状態に該当する人物の画像を合成した画像である群衆状態画像を生成し、当該群衆状態画像に対する教師ラベルを特定し、群衆状態画像および教師ラベルの組を出力する群衆状態画像合成ステップとを含むことを特徴とする教師データ生成方法。
人物状態決定ステップで、
複数人物状態制御指示と個別人物状態制御指示とに従って、群衆の人物状態を仮決定し、仮決定した人物状態が、所定サイズに対して定められた人物の基準部位のサイズおよび前記基準部位の表れ方に関する条件を満たしている場合に、仮決定した人物状態を群衆の人物状態として決定し、仮決定した人物状態が前記条件を満たしていない場合に、再度群衆の人物状態を仮決定することを繰り返す
付記17に記載の教師データ生成方法。
群衆状態制御指示記憶手段が、項目毎に定められた複数人物状態制御指示を記憶するとともに、前記項目毎に定められた教師ラベルの指定の有無を記憶し、
人物状態制御指示記憶手段が、項目毎に定められた個別人物状態制御指示を記憶するとともに、前記項目毎に定められた教師ラベルの指定の有無を記憶し、
人物状態決定ステップで、群衆状態制御指示記憶手段に記憶された複数人物状態制御指示および人物状態制御指示記憶手段に記憶された個別人物状態制御指示に従って、群衆の人物状態を決定し、
群衆状態画像合成ステップで、
教師ラベルの指定ありと定められた項目の複数人物状態制御指示を群衆状態制御指示記憶手段から読み込み、教師ラベルの指定ありと定められた項目の個別人物状態制御指示を人物状態制御指示記憶手段から読み込むことによって、教師ラベルを特定する
付記17または付記18に記載の教師データ生成方法。
群衆状態制御指示記憶手段が、少なくとも1つの項目について、教師ラベルの指定をありとして記憶し、
群衆状態画像合成ステップで、
教師ラベルの指定ありと定められた項目の複数人物状態制御指示を群衆状態制御指示記憶手段から読み込む
付記19に記載の教師データ生成方法。
群衆状態制御指示記憶手段が、人物の配置、人物の方向、および人数に関する項目毎に、複数人物状態制御指示および教師ラベルの指定の有無を記憶するとともに、各項目に対応する複数人物状態制御指示を、具体的な状態を指示する第1の態様、任意の状態に定めてよい旨を指示する第2の態様、定められたルールの範囲内で状態を定めてよいことを指示する第3の態様のいずれかの態様で記憶し、
人物状態制御指示記憶手段が、人物の撮影角度、人物への照明、人物の姿勢、人物の服装、人物の体型、人物の髪型、群衆状態画像に合成するときの人物サイズに関する項目毎に、個別人物状態制御指示および教師ラベルの指定の有無を記憶するとともに、各項目に対応する個別人物状態制御指示を、前記第1の態様、前記第2の態様、前記第3の態様のいずれかの態様で記憶し、
人物状態決定ステップで、群衆状態制御指示記憶手段に記憶された複数人物状態制御指示および人物状態制御指示記憶手段に記憶された個別人物状態制御指示に従って、群衆の人物状態を決定する
付記19または付記20に記載の教師データ生成方法。
群衆状態画像合成ステップで、人物の状態として決定された、人物の方向、人数、人物の撮影角度、人物への照明、人物の姿勢、人物の服装、人物の体型、人物の髪型に合致する人物画像を、予め用意された人物画像群の中から選択し、選択した人物画像の中から、人物の領域を切り出すことによって、人物部分のみの画像を生成し、人物の状態として決定された人物の配置、および群衆状態画像に合成するときの人物サイズに合わせて、前記人物部分のみを画像を、背景抽出手段によって得られた所定サイズの画像に対して合成する
付記17から付記21のうちのいずれかに記載の教師データ生成方法。
群衆状態画像合成ステップで、カメラからより遠い配置位置に該当する人物部分のみの画像から順に、背景抽出手段によって得られた所定サイズの画像に対して合成する
付記22に記載の教師データ生成方法。
人物状態決定ステップは、
複数人物状態制御指示と個別人物状態制御指示とに従って、群衆状態画像で背景となる群衆の人物状態を仮決定し、仮決定した人物状態が、群衆状態画像のサイズである所定サイズに対して定められた人物の基準部位のサイズおよび前記基準部位の表れ方に関する第1の条件を満たしている場合に、仮決定した人物状態を背景となる群衆の人物状態として決定し、仮決定した人物状態が前記第1の条件を満たしていない場合に、再度、背景となる群衆の人物状態を仮決定することを繰り返す背景人物状態決定ステップと、
複数人物状態制御指示と個別人物状態制御指示とに従って、群衆状態画像で前景となる群衆の人物状態を仮決定し、仮決定した人物状態が、群衆状態画像のサイズである所定サイズに対して定められた人物の基準部位のサイズおよび前記基準部位の表れ方に関する第2の条件を満たしている場合に、仮決定した人物状態を前景となる群衆の人物状態として決定し、仮決定した人物状態が前記第2の条件を満たしていない場合に、再度、前景となる群衆の人物状態を仮決定することを繰り返す前景人物状態決定ステップとを含む
付記17から付記23のうちのいずれかに記載の教師データ生成方法。
第1の条件は、人物の基準部位が群衆状態画像に納まらない状態であること、あるいは、所定サイズに対して定められた人物の基準部位のサイズに対して、基準部位のサイズが極端に大きいか、または、極端に小さい状態であることのいずれかに該当することであり、
第2の条件は、人物の基準部位が群衆状態画像に納まり、かつ、前記基準部位のサイズが、所定サイズに対して定められた人物の基準部位のサイズと同程度であることである
付記24に記載の教師データ生成方法。
矩形領域群記憶手段が、画像上の群衆状態の認識対象箇所を示す矩形領域群を記憶し、
群衆状態認識辞書記憶手段が、群衆状態を表した所定サイズの画像であって、前記所定サイズに対して定められた人物の基準部位のサイズと同程度のサイズで基準部位が表されている人物を含む画像である群衆状態画像と、当該群衆状態画像に対する教師ラベルとの組を複数組用いて機械学習を行うことによって得られた識別器の辞書を記憶し、
与えられた画像から、矩形領域群記憶手段に記憶される矩形領域群が示す領域をそれぞれ抽出し、前記辞書に基づいて、抽出した画像に写っている群衆の状態を認識する群衆状態認識ステップを含む
ことを特徴とする群衆状態認識方法。
群衆状態認識辞書記憶手段が、所望の状態に制御された人物の状態に合致する人物の画像を合成することによって得られた群衆状態画像と、当該群衆状態画像に対する教師ラベルとの組を複数組用いて機械学習を行うことによって得られた識別器の辞書を記憶し、
群衆状態認識ステップで、前記辞書に基づいて、画像に写っている群衆の状態を認識する
付記26に記載の群衆状態認識方法。
矩形領域群記憶手段が、画像を取得する画像取得装置の位置、姿勢、焦点距離、レンズ歪みを示すカメラパラメータと、所定サイズに対して定められた人物の基準部位のサイズとに基づいて、サイズが定められた矩形領域群を記憶し、
群衆状態認識ステップで、与えられた画像から、前記矩形領域群が示す領域をそれぞれ抽出する
付記26または付記27に記載の群衆状態認識方法。
群衆状態認識辞書記憶手段が、群衆状態画像に表される人物の人数を変えて、人数毎に複数組用意された群衆状態画像と教師ラベルとの組を用いて機械学習を行うことによって得られた識別器の辞書を記憶し、
群衆状態認識ステップで、前記辞書に基づいて、画像に写っている群衆の人数を認識する
付記26から付記28のうちのいずれかに記載の群衆状態認識方法。
群衆状態認識辞書記憶手段が、群衆状態画像に表される人物の方向を変えて、人物の方向毎に複数組用意された群衆状態画像と教師ラベルとの組を用いて機械学習を行うことによって得られた識別器の辞書を記憶し、
群衆状態認識ステップで、前記辞書に基づいて、画像に写っている群衆の方向を認識する
付記26から付記29のうちのいずれかに記載の群衆状態認識方法。
群衆状態認識辞書記憶手段が、極端に混雑していない群衆と、極端に混雑した群衆とについてそれぞれ複数組用意された群衆状態画像と教師ラベルとの組を用いて機械学習を行うことによって得られた識別器の辞書を記憶し、
群衆状態認識ステップで、前記辞書に基づいて、画像に写っている群衆が極端に混雑しているか否かを認識する
付記26から付記30のうちのいずれかに記載の群衆状態認識方法。
群衆状態認識辞書記憶手段が、人物の方向が統一されている群衆と、人物の方向が統一されていない群衆とについてそれぞれ複数組用意された群衆状態画像と教師ラベルとの組を用いて機械学習を行うことによって得られた識別器の辞書を記憶し、
群衆状態認識ステップで、前記辞書に基づいて、画像に写っている群衆内で人物の方向が統一されているか否かを認識する
付記26から付記31のうちのいずれかに記載の群衆状態認識方法。
コンピュータに、
予め用意された複数の背景画像から背景画像を選択し、前記背景画像中の領域を抽出し、抽出した領域に該当する画像を、所定サイズの画像に拡大または縮小する背景抽出処理、
複数の人物に関係する人物の状態の指示情報である複数人物状態制御指示と、前記複数の人物中の個別の人物の状態の指示情報である個別人物状態制御指示とに従って、群衆の人物状態を決定する人物状態決定処理、および、
背景抽出処理で得られた所定サイズの画像に対して、人物状態決定処理で決定した人物状態に該当する人物の画像を合成した画像である群衆状態画像を生成し、当該群衆状態画像に対する教師ラベルを特定し、群衆状態画像および教師ラベルの組を出力する群衆状態画像合成処理を実行させる
ための教師データ生成プログラム。
コンピュータに、
人物状態決定処理で、
複数人物状態制御指示と個別人物状態制御指示とに従って、群衆の人物状態を仮決定させ、仮決定した人物状態が、所定サイズに対して定められた人物の基準部位のサイズおよび前記基準部位の表れ方に関する条件を満たしている場合に、仮決定した人物状態を群衆の人物状態として決定させ、仮決定した人物状態が前記条件を満たしていない場合に、再度群衆の人物状態を仮決定することを繰り返させる
付記33に記載の教師データ生成プログラム。
項目毎に定められた複数人物状態制御指示を記憶するとともに、前記項目毎に定められた教師ラベルの指定の有無を記憶する群衆状態制御指示記憶手段と、項目毎に定められた個別人物状態制御指示を記憶するとともに、前記項目毎に定められた教師ラベルの指定の有無を記憶する人物状態制御指示記憶手段とを備えたコンピュータに、
人物状態決定処理で、群衆状態制御指示記憶手段に記憶された複数人物状態制御指示および人物状態制御指示記憶手段に記憶された個別人物状態制御指示に従って、群衆の人物状態を決定させ、
群衆状態画像合成処理で、
教師ラベルの指定ありと定められた項目の複数人物状態制御指示を群衆状態制御指示記憶手段から読み込み、教師ラベルの指定ありと定められた項目の個別人物状態制御指示を人物状態制御指示記憶手段から読み込むことによって、教師ラベルを特定させる
付記33または付記34に記載の教師データ生成プログラム。
少なくとも1つの項目について、教師ラベルの指定をありとして記憶する群衆状態制御指示記憶手段を備えたコンピュータに、
群衆状態画像合成処理で、
教師ラベルの指定ありと定められた項目の複数人物状態制御指示を群衆状態制御指示記憶手段から読み込ませる
付記35に記載の教師データ生成プログラム。
人物の配置、人物の方向、および人数に関する項目毎に、複数人物状態制御指示および教師ラベルの指定の有無を記憶するとともに、各項目に対応する複数人物状態制御指示を、具体的な状態を指示する第1の態様、任意の状態に定めてよい旨を指示する第2の態様、定められたルールの範囲内で状態を定めてよいことを指示する第3の態様のいずれかの態様で記憶する群衆状態制御指示記憶手段と、人物の撮影角度、人物への照明、人物の姿勢、人物の服装、人物の体型、人物の髪型、群衆状態画像に合成するときの人物サイズに関する項目毎に、個別人物状態制御指示および教師ラベルの指定の有無を記憶するとともに、各項目に対応する個別人物状態制御指示を、前記第1の態様、前記第2の態様、前記第3の態様のいずれかの態様で記憶する人物状態制御指示記憶手段とを備えたコンピュータに、
人物状態決定処理で、群衆状態制御指示記憶手段に記憶された複数人物状態制御指示および人物状態制御指示記憶手段に記憶された個別人物状態制御指示に従って、群衆の人物状態を決定させる
付記35または付記36に記載の教師データ生成プログラム。
コンピュータに、
群衆状態画像合成処理で、人物の状態として決定された、人物の方向、人数、人物の撮影角度、人物への照明、人物の姿勢、人物の服装、人物の体型、人物の髪型に合致する人物画像を、予め用意された人物画像群の中から選択させ、選択した人物画像の中から、人物の領域を切り出すことによって、人物部分のみの画像を生成させ、人物の状態として決定された人物の配置、および群衆状態画像に合成するときの人物サイズに合わせて、前記人物部分のみを画像を、背景抽出処理によって得られた所定サイズの画像に対して合成させる
付記33から付記37のうちのいずれかに記載の教師データ生成プログラム。
コンピュータに、
群衆状態画像合成処理で、カメラからより遠い配置位置に該当する人物部分のみの画像から順に、背景抽出処理で得られた所定サイズの画像に対して合成させる
付記38に記載の教師データ生成プログラム。
コンピュータに、
人物状態決定処理で、
複数人物状態制御指示と個別人物状態制御指示とに従って、群衆状態画像で背景となる群衆の人物状態を仮決定し、仮決定した人物状態が、群衆状態画像のサイズである所定サイズに対して定められた人物の基準部位のサイズおよび前記基準部位の表れ方に関する第1の条件を満たしている場合に、仮決定した人物状態を背景となる群衆の人物状態として決定し、仮決定した人物状態が前記第1の条件を満たしていない場合に、再度、背景となる群衆の人物状態を仮決定することを繰り返す背景人物状態決定処理、および、
複数人物状態制御指示と個別人物状態制御指示とに従って、群衆状態画像で前景となる群衆の人物状態を仮決定し、仮決定した人物状態が、群衆状態画像のサイズである所定サイズに対して定められた人物の基準部位のサイズおよび前記基準部位の表れ方に関する第2の条件を満たしている場合に、仮決定した人物状態を前景となる群衆の人物状態として決定し、仮決定した人物状態が前記第2の条件を満たしていない場合に、再度、前景となる群衆の人物状態を仮決定することを繰り返す前景人物状態決定処理
を実行させる付記33から付記39のうちのいずれかに記載の教師データ生成プログラム。
第1の条件は、人物の基準部位が群衆状態画像に納まらない状態であること、あるいは、所定サイズに対して定められた人物の基準部位のサイズに対して、基準部位のサイズが極端に大きいか、または、極端に小さい状態であることのいずれかに該当することであり、
第2の条件は、人物の基準部位が群衆状態画像に納まり、かつ、前記基準部位のサイズが、所定サイズに対して定められた人物の基準部位のサイズと同程度であることである
付記40に記載の教師データ生成プログラム。
画像上の群衆状態の認識対象箇所を示す矩形領域群を記憶する矩形領域群記憶手段と、群衆状態を表した所定サイズの画像であって、前記所定サイズに対して定められた人物の基準部位のサイズと同程度のサイズで基準部位が表されている人物を含む画像である群衆状態画像と、当該群衆状態画像に対する教師ラベルとの組を複数組用いて機械学習を行うことによって得られた識別器の辞書を記憶する群衆状態認識辞書記憶手段とを備えたコンピュータに、
与えられた画像から、矩形領域群記憶手段に記憶される矩形領域群が示す領域をそれぞれ抽出し、前記辞書に基づいて、抽出した画像に写っている群衆の状態を認識する群衆状態認識処理を実行させる
ための群衆状態認識プログラム。
所望の状態に制御された人物の状態に合致する人物の画像を合成することによって得られた群衆状態画像と、当該群衆状態画像に対する教師ラベルとの組を複数組用いて機械学習を行うことによって得られた識別器の辞書を記憶する群衆状態認識辞書記憶手段を備えたコンピュータに、
群衆状態認識処理で、前記辞書に基づいて、画像に写っている群衆の状態を認識させる
付記42に記載の群衆状態認識プログラム。
画像を取得する画像取得装置の位置、姿勢、焦点距離、レンズ歪みを示すカメラパラメータと、所定サイズに対して定められた人物の基準部位のサイズとに基づいて、サイズが定められた矩形領域群を記憶する矩形領域群記憶手段を備えたコンピュータに、
群衆状態認識処理で、与えられた画像から、前記矩形領域群が示す領域をそれぞれ抽出させる
付記42または付記43に記載の群衆状態認識プログラム。
群衆状態画像に表される人物の人数を変えて、人数毎に複数組用意された群衆状態画像と教師ラベルとの組を用いて機械学習を行うことによって得られた識別器の辞書を記憶する群衆状態認識辞書記憶手段を備えたコンピュータに、
群衆状態認識処理で、前記辞書に基づいて、画像に写っている群衆の人数を認識させる
付記42から付記44のうちのいずれかに記載の群衆状態認識プログラム。
群衆状態画像に表される人物の方向を変えて、人物の方向毎に複数組用意された群衆状態画像と教師ラベルとの組を用いて機械学習を行うことによって得られた識別器の辞書を記憶する群衆状態認識辞書記憶手段を備えたコンピュータに、
群衆状態認識処理で、前記辞書に基づいて、画像に写っている群衆の方向を認識させる
付記42から付記45のうちのいずれかに記載の群衆状態認識プログラム。
極端に混雑していない群衆と、極端に混雑した群衆とについてそれぞれ複数組用意された群衆状態画像と教師ラベルとの組を用いて機械学習を行うことによって得られた識別器の辞書を記憶する群衆状態認識辞書記憶手段を備えたコンピュータに、
群衆状態認識処理で、前記辞書に基づいて、画像に写っている群衆が極端に混雑しているか否かを認識させる
付記42から付記46のうちのいずれかに記載の群衆状態認識プログラム。
人物の方向が統一されている群衆と、人物の方向が統一されていない群衆とについてそれぞれ複数組用意された群衆状態画像と教師ラベルとの組を用いて機械学習を行うことによって得られた識別器の辞書を記憶する群衆状態認識辞書記憶手段を備えたコンピュータに、
群衆状態認識処理で、前記辞書に基づいて、画像に写っている群衆内で人物の方向が統一されているか否かを認識させる
付記42から付記47のうちのいずれかに記載の群衆状態認識プログラム。
12 背景人物状態決定手段
13 前景人物状態決定手段
14 群衆状態画像合成手段
15 人物状態決定手段
16 制御手段
21 背景画像記憶手段
22 学習用局所画像情報記憶手段
23 群衆状態制御指示記憶手段
24 人物状態制御指示記憶手段
25 人物画像記憶手段
26 人物領域画像記憶手段
41 群衆状態認識手段
51 探索窓記憶手段
52 群衆状態認識辞書記憶手段
Claims (20)
- 予め用意された複数の背景画像から背景画像を選択し、前記背景画像中の領域を抽出し、抽出した領域に該当する画像を、所定サイズの画像に拡大または縮小する背景抽出手段と、
複数の人物に関係する人物の状態の指示情報である複数人物状態制御指示と、前記複数の人物中の個別の人物の状態の指示情報である個別人物状態制御指示とに従って、群衆の人物状態を決定する人物状態決定手段と、
背景抽出手段によって得られた所定サイズの画像に対して、人物状態決定手段が決定した人物状態に該当する人物の画像を合成した画像である群衆状態画像を生成し、当該群衆状態画像に対する教師ラベルを特定し、群衆状態画像および教師ラベルの組を出力する群衆状態画像合成手段とを備える
ことを特徴とする教師データ生成装置。 - 人物状態決定手段は、
複数人物状態制御指示と個別人物状態制御指示とに従って、群衆の人物状態を仮決定し、仮決定した人物状態が、所定サイズに対して定められた人物の基準部位のサイズおよび前記基準部位の表れ方に関する条件を満たしている場合に、仮決定した人物状態を群衆の人物状態として決定し、仮決定した人物状態が前記条件を満たしていない場合に、再度群衆の人物状態を仮決定することを繰り返す
請求項1に記載の教師データ生成装置。 - 項目毎に定められた複数人物状態制御指示を記憶するとともに、前記項目毎に定められた教師ラベルの指定の有無を記憶する群衆状態制御指示記憶手段と、
項目毎に定められた個別人物状態制御指示を記憶するとともに、前記項目毎に定められた教師ラベルの指定の有無を記憶する人物状態制御指示記憶手段とを備え、
人物状態決定手段は、群衆状態制御指示記憶手段に記憶された複数人物状態制御指示および人物状態制御指示記憶手段に記憶された個別人物状態制御指示に従って、群衆の人物状態を決定し、
群衆状態画像合成手段は、
教師ラベルの指定ありと定められた項目の複数人物状態制御指示を群衆状態制御指示記憶手段から読み込み、教師ラベルの指定ありと定められた項目の個別人物状態制御指示を人物状態制御指示記憶手段から読み込むことによって、教師ラベルを特定する
請求項1または請求項2に記載の教師データ生成装置。 - 群衆状態制御指示記憶手段は、
少なくとも1つの項目について、教師ラベルの指定をありとして記憶し、
群衆状態画像合成手段は、
教師ラベルの指定ありと定められた項目の複数人物状態制御指示を群衆状態制御指示記憶手段から読み込む
請求項3に記載の教師データ生成装置。 - 群衆状態制御指示記憶手段は、人物の配置、人物の方向、および人数に関する項目毎に、複数人物状態制御指示および教師ラベルの指定の有無を記憶するとともに、各項目に対応する複数人物状態制御指示を、具体的な状態を指示する第1の態様、任意の状態に定めてよい旨を指示する第2の態様、定められたルールの範囲内で状態を定めてよいことを指示する第3の態様のいずれかの態様で記憶し、
人物状態制御指示記憶手段は、人物の撮影角度、人物への照明、人物の姿勢、人物の服装、人物の体型、人物の髪型、群衆状態画像に合成するときの人物サイズに関する項目毎に、個別人物状態制御指示および教師ラベルの指定の有無を記憶するとともに、各項目に対応する個別人物状態制御指示を、前記第1の態様、前記第2の態様、前記第3の態様のいずれかの態様で記憶し、
人物状態決定手段は、群衆状態制御指示記憶手段に記憶された複数人物状態制御指示および人物状態制御指示記憶手段に記憶された個別人物状態制御指示に従って、群衆の人物状態を決定する
請求項3または請求項4に記載の教師データ生成装置。 - 群衆状態画像合成手段は、人物の状態として決定された、人物の方向、人数、人物の撮影角度、人物への照明、人物の姿勢、人物の服装、人物の体型、人物の髪型に合致する人物画像を、予め用意された人物画像群の中から選択し、選択した人物画像の中から、人物の領域を切り出すことによって、人物部分のみの画像を生成し、人物の状態として決定された人物の配置、および群衆状態画像に合成するときの人物サイズに合わせて、前記人物部分のみを画像を、背景抽出手段によって得られた所定サイズの画像に対して合成する
請求項1から請求項5のうちのいずれか1項に記載の教師データ生成装置。 - 群衆状態画像合成手段は、カメラからより遠い配置位置に該当する人物部分のみの画像から順に、背景抽出手段によって得られた所定サイズの画像に対して合成する
請求項6に記載の教師データ生成装置。 - 人物状態決定手段は、
複数人物状態制御指示と個別人物状態制御指示とに従って、群衆状態画像で背景となる群衆の人物状態を仮決定し、仮決定した人物状態が、群衆状態画像のサイズである所定サイズに対して定められた人物の基準部位のサイズおよび前記基準部位の表れ方に関する第1の条件を満たしている場合に、仮決定した人物状態を背景となる群衆の人物状態として決定し、仮決定した人物状態が前記第1の条件を満たしていない場合に、再度、背景となる群衆の人物状態を仮決定することを繰り返す背景人物状態決定手段と、
複数人物状態制御指示と個別人物状態制御指示とに従って、群衆状態画像で前景となる群衆の人物状態を仮決定し、仮決定した人物状態が、群衆状態画像のサイズである所定サイズに対して定められた人物の基準部位のサイズおよび前記基準部位の表れ方に関する第2の条件を満たしている場合に、仮決定した人物状態を前景となる群衆の人物状態として決定し、仮決定した人物状態が前記第2の条件を満たしていない場合に、再度、前景となる群衆の人物状態を仮決定することを繰り返す前景人物状態決定手段とを含む
請求項1から請求項7のうちのいずれか1項に記載の教師データ生成装置。 - 第1の条件は、人物の基準部位が群衆状態画像に納まらない状態であること、あるいは、所定サイズに対して定められた人物の基準部位のサイズに対して、基準部位のサイズが極端に大きいか、または、極端に小さい状態であることのいずれかに該当することであり、
第2の条件は、人物の基準部位が群衆状態画像に納まり、かつ、前記基準部位のサイズが、所定サイズに対して定められた人物の基準部位のサイズと同程度であることである
請求項8に記載の教師データ生成装置。 - 画像上の群衆状態の認識対象箇所を示す矩形領域群を記憶する矩形領域群記憶手段と、
群衆状態を表した所定サイズの画像であって、前記所定サイズに対して定められた人物の基準部位のサイズと同程度のサイズで基準部位が表されている人物を含む画像である群衆状態画像と、当該群衆状態画像に対する教師ラベルとの組を複数組用いて機械学習を行うことによって得られた識別器の辞書を記憶する群衆状態認識辞書記憶手段と、
与えられた画像から、矩形領域群記憶手段に記憶される矩形領域群が示す領域をそれぞれ抽出し、前記辞書に基づいて、抽出した画像に写っている群衆の状態を認識する群衆状態認識手段とを備える
ことを特徴とする群衆状態認識装置。 - 群衆状態認識辞書記憶手段は、所望の状態に制御された人物の状態に合致する人物の画像を合成することによって得られた群衆状態画像と、当該群衆状態画像に対する教師ラベルとの組を複数組用いて機械学習を行うことによって得られた識別器の辞書を記憶し、
群衆状態認識手段は、前記辞書に基づいて、画像に写っている群衆の状態を認識する
請求項10に記載の群衆状態認識装置。 - 矩形領域群記憶手段は、画像を取得する画像取得装置の位置、姿勢、焦点距離、レンズ歪みを示すカメラパラメータと、所定サイズに対して定められた人物の基準部位のサイズとに基づいて、サイズが定められた矩形領域群を記憶し、
群衆状態認識手段は、与えられた画像から、前記矩形領域群が示す領域をそれぞれ抽出する
請求項10または請求項11に記載の群衆状態認識装置。 - 群衆状態認識辞書記憶手段は、群衆状態画像に表される人物の人数を変えて、人数毎に複数組用意された群衆状態画像と教師ラベルとの組を用いて機械学習を行うことによって得られた識別器の辞書を記憶し、
群衆状態認識手段は、前記辞書に基づいて、画像に写っている群衆の人数を認識する
請求項10から請求項12のうちのいずれか1項に記載の群衆状態認識装置。 - 群衆状態認識辞書記憶手段は、群衆状態画像に表される人物の方向を変えて、人物の方向毎に複数組用意された群衆状態画像と教師ラベルとの組を用いて機械学習を行うことによって得られた識別器の辞書を記憶し、
群衆状態認識手段は、前記辞書に基づいて、画像に写っている群衆の方向を認識する
請求項10から請求項13のうちのいずれか1項に記載の群衆状態認識装置。 - 群衆状態認識辞書記憶手段は、極端に混雑していない群衆と、極端に混雑した群衆とについてそれぞれ複数組用意された群衆状態画像と教師ラベルとの組を用いて機械学習を行うことによって得られた識別器の辞書を記憶し、
群衆状態認識手段は、前記辞書に基づいて、画像に写っている群衆が極端に混雑しているか否かを認識する
請求項10から請求項14のうちのいずれか1項に記載の群衆状態認識装置。 - 群衆状態認識辞書記憶手段は、人物の方向が統一されている群衆と、人物の方向が統一されていない群衆とについてそれぞれ複数組用意された群衆状態画像と教師ラベルとの組を用いて機械学習を行うことによって得られた識別器の辞書を記憶し、
群衆状態認識手段は、前記辞書に基づいて、画像に写っている群衆内で人物の方向が統一されているか否かを認識する
請求項10から請求項15のうちのいずれか1項に記載の群衆状態認識装置。 - 予め用意された複数の背景画像から背景画像を選択し、前記背景画像中の領域を抽出し、抽出した領域に該当する画像を、所定サイズの画像に拡大または縮小する背景抽出ステップと、
複数の人物に関係する人物の状態の指示情報である複数人物状態制御指示と、前記複数の人物中の個別の人物の状態の指示情報である個別人物状態制御指示とに従って、群衆の人物状態を決定する人物状態決定ステップと、
背景抽出ステップで得られた所定サイズの画像に対して、人物状態決定ステップで決定した人物状態に該当する人物の画像を合成した画像である群衆状態画像を生成し、当該群衆状態画像に対する教師ラベルを特定し、群衆状態画像および教師ラベルの組を出力する群衆状態画像合成ステップとを含む
ことを特徴とする教師データ生成方法。 - 矩形領域群記憶手段が、画像上の群衆状態の認識対象箇所を示す矩形領域群を記憶し、
群衆状態認識辞書記憶手段が、群衆状態を表した所定サイズの画像であって、前記所定サイズに対して定められた人物の基準部位のサイズと同程度のサイズで基準部位が表されている人物を含む画像である群衆状態画像と、当該群衆状態画像に対する教師ラベルとの組を複数組用いて機械学習を行うことによって得られた識別器の辞書を記憶し、
与えられた画像から、矩形領域群記憶手段に記憶される矩形領域群が示す領域をそれぞれ抽出し、前記辞書に基づいて、抽出した画像に写っている群衆の状態を認識する群衆状態認識ステップを含む
ことを特徴とする群衆状態認識方法。 - コンピュータに、
予め用意された複数の背景画像から背景画像を選択し、前記背景画像中の領域を抽出し、抽出した領域に該当する画像を、所定サイズの画像に拡大または縮小する背景抽出処理、
複数の人物に関係する人物の状態の指示情報である複数人物状態制御指示と、前記複数の人物中の個別の人物の状態の指示情報である個別人物状態制御指示とに従って、群衆の人物状態を決定する人物状態決定処理、および、
背景抽出処理で得られた所定サイズの画像に対して、人物状態決定処理で決定した人物状態に該当する人物の画像を合成した画像である群衆状態画像を生成し、当該群衆状態画像に対する教師ラベルを特定し、群衆状態画像および教師ラベルの組を出力する群衆状態画像合成処理を実行させる
ための教師データ生成プログラム。 - 画像上の群衆状態の認識対象箇所を示す矩形領域群を記憶する矩形領域群記憶手段と、群衆状態を表した所定サイズの画像であって、前記所定サイズに対して定められた人物の基準部位のサイズと同程度のサイズで基準部位が表されている人物を含む画像である群衆状態画像と、当該群衆状態画像に対する教師ラベルとの組を複数組用いて機械学習を行うことによって得られた識別器の辞書を記憶する群衆状態認識辞書記憶手段とを備えたコンピュータに、
与えられた画像から、矩形領域群記憶手段に記憶される矩形領域群が示す領域をそれぞれ抽出し、前記辞書に基づいて、抽出した画像に写っている群衆の状態を認識する群衆状態認識処理を実行させる
ための群衆状態認識プログラム。
Priority Applications (14)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810432697.8A CN108647631B (zh) | 2013-06-28 | 2014-05-21 | 人群状态识别设备、方法和计算机可读记录介质 |
US14/899,340 US9875431B2 (en) | 2013-06-28 | 2014-05-21 | Training data generating device, method, and program, and crowd state recognition device, method, and program |
EP17199965.9A EP3312770B1 (en) | 2013-06-28 | 2014-05-21 | Crowd state recognition device, method, and program |
CN201480036661.8A CN105593901B (zh) | 2013-06-28 | 2014-05-21 | 训练数据生成设备、方法和程序以及人群状态识别设备、方法和程序 |
JP2015523834A JP6008045B2 (ja) | 2013-06-28 | 2014-05-21 | 教師データ生成装置、方法、プログラム、および群衆状態認識装置、方法、プログラム |
EP14817648.0A EP3016069A4 (en) | 2013-06-28 | 2014-05-21 | Teaching data generating device, method, and program, and crowd state recognition device, method, and program |
HK16108787.2A HK1220795A1 (zh) | 2013-06-28 | 2016-07-21 | 訓練數據生成設備、方法和程序以及人群狀態識別設備、方法和程序 |
US15/667,937 US10223620B2 (en) | 2013-06-28 | 2017-08-03 | Training data generating device, method, and program, and crowd state recognition device, method, and program |
US16/209,290 US10515294B2 (en) | 2013-06-28 | 2018-12-04 | Training data generating device, method, and program, and crowd state recognition device, method, and program |
US16/209,277 US10776674B2 (en) | 2013-06-28 | 2018-12-04 | Training data generating device, method, and program, and crowd state recognition device, method, and program |
US16/689,406 US11132587B2 (en) | 2013-06-28 | 2019-11-20 | Training data generating device, method, and program, and crowd state recognition device, method, and program |
US17/379,392 US11836586B2 (en) | 2013-06-28 | 2021-07-19 | Training data generating device, method, and program, and crowd state recognition device, method, and program |
US18/218,916 US20230351259A1 (en) | 2013-06-28 | 2023-07-06 | Training data generating device, method, and program, and crowd state recognition device, method, and program |
US18/218,903 US20230351258A1 (en) | 2013-06-28 | 2023-07-06 | Training data generating device, method, and program, and crowd state recognition device, method, and program |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2013135915 | 2013-06-28 | ||
JP2013-135915 | 2013-06-28 |
Related Child Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/889,340 A-371-Of-International US9584051B2 (en) | 2013-05-13 | 2014-05-12 | Method for estimating the angular position of the rotor of a polyphase rotary electrical machine, and application to the control of a polyphase inverter for such a machine |
US14/899,340 A-371-Of-International US9875431B2 (en) | 2013-06-28 | 2014-05-21 | Training data generating device, method, and program, and crowd state recognition device, method, and program |
US15/667,937 Division US10223620B2 (en) | 2013-06-28 | 2017-08-03 | Training data generating device, method, and program, and crowd state recognition device, method, and program |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2014207991A1 true WO2014207991A1 (ja) | 2014-12-31 |
Family
ID=52141367
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2014/002670 WO2014207991A1 (ja) | 2013-06-28 | 2014-05-21 | 教師データ生成装置、方法、プログラム、および群衆状態認識装置、方法、プログラム |
Country Status (6)
Country | Link |
---|---|
US (8) | US9875431B2 (ja) |
EP (2) | EP3016069A4 (ja) |
JP (1) | JP6008045B2 (ja) |
CN (2) | CN108647631B (ja) |
HK (3) | HK1220795A1 (ja) |
WO (1) | WO2014207991A1 (ja) |
Cited By (25)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106033548A (zh) * | 2015-03-13 | 2016-10-19 | 中国科学院西安光学精密机械研究所 | 基于改进字典学习的拥挤人群异常检测方法 |
WO2017154655A1 (ja) * | 2016-03-07 | 2017-09-14 | 日本電気株式会社 | 群衆種類識別システム、群衆種類識別方法および群衆種類識別プログラムを記憶する記憶媒体 |
WO2017163955A1 (ja) | 2016-03-23 | 2017-09-28 | 日本電気株式会社 | 監視システム、画像処理装置、画像処理方法およびプログラム記録媒体 |
JPWO2016114134A1 (ja) * | 2015-01-14 | 2017-10-26 | 日本電気株式会社 | 移動状況推定装置、移動状況推定方法およびプログラム |
WO2018025831A1 (ja) * | 2016-08-04 | 2018-02-08 | 日本電気株式会社 | 人流推定装置、表示制御装置、人流推定方法および記録媒体 |
JP2018060360A (ja) * | 2016-10-05 | 2018-04-12 | 日本電信電話株式会社 | 人数推定方法、人数推定装置、人数推定プログラム、及び記録媒体 |
JP2018148367A (ja) * | 2017-03-03 | 2018-09-20 | キヤノン株式会社 | 画像処理装置、画像処理システム、画像処理方法、及びプログラム |
WO2018216648A1 (ja) * | 2017-05-22 | 2018-11-29 | 日本電気株式会社 | 群衆状態認識装置、学習方法および学習プログラム |
WO2019235192A1 (ja) * | 2018-06-06 | 2019-12-12 | 日本電信電話株式会社 | 領域抽出モデル学習装置、領域抽出モデル学習方法、プログラム |
JP2020013467A (ja) * | 2018-07-20 | 2020-01-23 | 株式会社壽屋 | 分類器生成方法、機械学習用画像及び学習データ生成方法 |
WO2020105146A1 (ja) * | 2018-11-21 | 2020-05-28 | 日本電気株式会社 | 情報処理装置、制御方法、及びプログラム |
JP2020080003A (ja) * | 2018-11-12 | 2020-05-28 | 株式会社イシダ | 教師データ生成方法、教師データ生成プログラム、教師データ生成装置、及び商品識別装置 |
KR20200061478A (ko) * | 2018-11-23 | 2020-06-03 | 연세대학교 산학협력단 | 실시간 객체 탐지 시스템 학습을 위한 합성 데이터 생성 장치 및 방법 |
US10699422B2 (en) | 2016-03-18 | 2020-06-30 | Nec Corporation | Information processing apparatus, control method, and program |
CN111742342A (zh) * | 2018-03-12 | 2020-10-02 | 日立产业控制解决方案有限公司 | 图像生成方法、图像生成装置以及图像生成系统 |
US10810442B2 (en) | 2016-09-13 | 2020-10-20 | Nec Corporation | People flow estimation device, people flow estimation method, and recording medium |
US10846838B2 (en) | 2016-11-25 | 2020-11-24 | Nec Corporation | Image generation device, image generation method, and storage medium storing program |
JP2021106044A (ja) * | 2016-03-09 | 2021-07-26 | 日本電気株式会社 | 画像処理装置、画像処理方法およびプログラム |
KR20210092842A (ko) * | 2019-01-22 | 2021-07-26 | 미쓰비시덴키 가부시키가이샤 | 정보 처리 장치, 프로그램을 기록한 기록 매체 및 정보 처리 방법 |
JP2021196755A (ja) * | 2020-06-11 | 2021-12-27 | 日本電信電話株式会社 | 画像処理装置、画像処理方法及び画像処理プログラム |
WO2022080407A1 (ja) * | 2020-10-14 | 2022-04-21 | 国立研究開発法人海洋研究開発機構 | 魚数算出方法、魚数算出プログラム、及び、魚数算出装置 |
US11386562B2 (en) | 2018-12-28 | 2022-07-12 | Cyberlink Corp. | Systems and methods for foreground and background processing of content in a live video |
JP2022140776A (ja) * | 2020-09-28 | 2022-09-27 | ソフトバンク株式会社 | 情報処理方法、プログラムおよび情報処理装置 |
JP2023501690A (ja) * | 2019-11-20 | 2023-01-18 | オムロン株式会社 | 群衆のダイナミックを予測する方法及びシステム |
CN115880648A (zh) * | 2023-03-08 | 2023-03-31 | 城云科技(中国)有限公司 | 无人机角度下的人群聚集识别方法、系统及其应用 |
Families Citing this family (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108647631B (zh) * | 2013-06-28 | 2023-04-07 | 日本电气株式会社 | 人群状态识别设备、方法和计算机可读记录介质 |
AU2015203771B2 (en) * | 2014-07-08 | 2020-11-05 | Iomniscient Pty Ltd | A method and apparatus for surveillance |
JP2017097510A (ja) * | 2015-11-20 | 2017-06-01 | ソニー株式会社 | 画像処理装置と画像処理方法およびプログラム |
US10292585B1 (en) | 2016-12-23 | 2019-05-21 | X Development Llc | Mental state measurement using sensors attached to non-wearable objects |
EP3566175A4 (en) * | 2017-01-06 | 2020-07-29 | Sportlogiq Inc. | SYSTEMS AND METHODS FOR UNDERSTANDING BEHAVIOR FROM TRAJECTORIES |
EP3566177A1 (en) * | 2017-02-03 | 2019-11-13 | Siemens Aktiengesellschaft | A method and apparatus for detecting objects of interest in images |
CN110291539A (zh) * | 2017-02-24 | 2019-09-27 | 欧姆龙株式会社 | 用于生成学习数据的处理方法、系统、程序和存储介质、以及生成学习数据的方法和系统 |
FR3074942B1 (fr) * | 2017-12-08 | 2021-04-02 | Valeo Comfort & Driving Assistance | Procede de determination de l'etat d'un individu a partir du traitement d'images de sa tete et dispositif associe |
US11019283B2 (en) * | 2018-01-18 | 2021-05-25 | GumGum, Inc. | Augmenting detected regions in image or video data |
SG10201802673VA (en) * | 2018-03-29 | 2019-10-30 | Nec Asia Pacific Pte Ltd | Method and system for integration and automatic switching of crowd estimation techniques |
JP6908183B2 (ja) | 2018-04-12 | 2021-07-21 | 日本電気株式会社 | 学習用画像生成装置、学習用画像生成方法及びプログラム |
CN109284681B (zh) * | 2018-08-20 | 2020-11-27 | 北京市商汤科技开发有限公司 | 位姿检测方法及装置、电子设备和存储介质 |
US11107239B2 (en) | 2018-08-20 | 2021-08-31 | Beijing Sensetime Technology Development Co., Ltd. | Pose detection method and device, electronic device and storage medium |
US11157931B2 (en) * | 2018-08-21 | 2021-10-26 | International Business Machines Corporation | Predicting the crowdedness of a location |
TWI686748B (zh) * | 2018-12-07 | 2020-03-01 | 國立交通大學 | 人流分析系統及人流分析方法 |
CN109583509B (zh) * | 2018-12-12 | 2020-11-03 | 南京旷云科技有限公司 | 数据生成方法、装置及电子设备 |
JP7075056B2 (ja) | 2018-12-27 | 2022-05-25 | オムロン株式会社 | 画像判定装置、画像判定方法及び画像判定プログラム |
CN116916080A (zh) * | 2019-05-17 | 2023-10-20 | 上海哔哩哔哩科技有限公司 | 视频数据处理方法、装置、计算机设备及可读存储介质 |
US11048948B2 (en) * | 2019-06-10 | 2021-06-29 | City University Of Hong Kong | System and method for counting objects |
KR102359289B1 (ko) * | 2019-12-30 | 2022-02-08 | 한국과학기술연구원 | 학습된 기계학습 모델의 성능을 개선하기 위한 가상 학습데이터 생성 방법 및 이를 수행하는 장치 |
US11410443B2 (en) | 2020-03-13 | 2022-08-09 | Nec Corporation | Labelling training method and system for implementing the same |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2007249588A (ja) * | 2006-03-15 | 2007-09-27 | Omron Corp | 顔画像登録装置、顔画像登録方法、顔画像登録プログラム、および記録媒体 |
JP2007329762A (ja) * | 2006-06-08 | 2007-12-20 | Fujitsu Ten Ltd | 物体候補領域検出装置、物体候補領域検出方法、歩行者認識装置、および車両制御装置 |
JP2008140107A (ja) * | 2006-12-01 | 2008-06-19 | Seiko Epson Corp | 画像処理装置、画像処理方法、制御プログラム及び記録媒体 |
JP2010198566A (ja) | 2009-02-27 | 2010-09-09 | Nec Corp | 人数計測装置、方法及びプログラム |
JP2011076316A (ja) * | 2009-09-30 | 2011-04-14 | Fujifilm Corp | 群衆監視装置および方法ならびにプログラム |
JP2011100175A (ja) | 2009-11-04 | 2011-05-19 | Nippon Hoso Kyokai <Nhk> | 人物行動判定装置及びそのプログラム |
JP2012022370A (ja) | 2010-07-12 | 2012-02-02 | Hitachi Kokusai Electric Inc | 監視システムおよび監視方法 |
Family Cites Families (33)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7121946B2 (en) * | 1998-08-10 | 2006-10-17 | Cybernet Systems Corporation | Real-time head tracking system for computer games and other applications |
US7139409B2 (en) * | 2000-09-06 | 2006-11-21 | Siemens Corporate Research, Inc. | Real-time crowd density estimation from video |
JP3981391B2 (ja) | 2003-10-21 | 2007-09-26 | 松下電器産業株式会社 | 監視装置 |
US20080166020A1 (en) * | 2005-01-28 | 2008-07-10 | Akio Kosaka | Particle-Group Movement Analysis System, Particle-Group Movement Analysis Method and Program |
JP4640155B2 (ja) * | 2005-12-15 | 2011-03-02 | ソニー株式会社 | 画像処理装置および方法、並びにプログラム |
US8195598B2 (en) * | 2007-11-16 | 2012-06-05 | Agilence, Inc. | Method of and system for hierarchical human/crowd behavior detection |
JP5176572B2 (ja) * | 2008-02-05 | 2013-04-03 | ソニー株式会社 | 画像処理装置および方法、並びにプログラム |
EP2093698A1 (en) * | 2008-02-19 | 2009-08-26 | British Telecommunications Public Limited Company | Crowd congestion analysis |
EP2618290A3 (en) * | 2008-04-02 | 2014-08-06 | Google, Inc. | Method and apparatus to incorporate automatic face recognition in digital image collections |
GB2472932B (en) * | 2008-06-13 | 2012-10-03 | Lockheed Corp | Method and system for crowd segmentation |
CN101777114B (zh) * | 2009-01-08 | 2013-04-24 | 北京中星微电子有限公司 | 视频监控智能分析系统和方法及头肩检测跟踪系统和方法 |
JP5361524B2 (ja) * | 2009-05-11 | 2013-12-04 | キヤノン株式会社 | パターン認識システム及びパターン認識方法 |
JP2011248548A (ja) * | 2010-05-25 | 2011-12-08 | Fujitsu Ltd | コンテンツ決定プログラムおよびコンテンツ決定装置 |
CN101980245B (zh) * | 2010-10-11 | 2013-07-17 | 北京航空航天大学 | 一种基于自适应模板匹配的客流统计方法 |
CN101959060B (zh) * | 2010-10-26 | 2012-06-06 | 镇江科大船苑计算机网络工程有限公司 | 视频客流监控系统及方法 |
CN102063613B (zh) * | 2010-12-28 | 2012-12-05 | 北京智安邦科技有限公司 | 基于头部识别的人群计数方法及装置 |
US9117147B2 (en) * | 2011-04-29 | 2015-08-25 | Siemens Aktiengesellschaft | Marginal space learning for multi-person tracking over mega pixel imagery |
US9208386B1 (en) * | 2012-01-09 | 2015-12-08 | The United States Of America As Represented By The Secretary Of The Navy | Crowd state characterization system and method |
CN102663369B (zh) * | 2012-04-20 | 2013-11-20 | 西安电子科技大学 | 基于surf高效匹配核的人体运动跟踪方法 |
CN102722699A (zh) * | 2012-05-22 | 2012-10-10 | 湖南大学 | 基于多尺度韦伯局部特征和核组稀疏表示的人脸识别方法 |
CN108647631B (zh) * | 2013-06-28 | 2023-04-07 | 日本电气株式会社 | 人群状态识别设备、方法和计算机可读记录介质 |
WO2015106808A1 (en) * | 2014-01-15 | 2015-07-23 | Nec Europe Ltd. | Method and system for crowd detection in an area |
JP2015204561A (ja) * | 2014-04-15 | 2015-11-16 | 株式会社デンソー | 情報提示システム、及び、提示装置 |
US10846536B2 (en) * | 2014-06-27 | 2020-11-24 | Nec Corporation | Abnormality detection device and abnormality detection method |
JP6708122B2 (ja) * | 2014-06-30 | 2020-06-10 | 日本電気株式会社 | 誘導処理装置及び誘導方法 |
JP6597611B2 (ja) * | 2014-07-25 | 2019-10-30 | 日本電気株式会社 | 画像処理装置、監視システム、画像処理方法、及びプログラム |
CN105654021B (zh) * | 2014-11-12 | 2019-02-01 | 株式会社理光 | 检测人群对目标位置关注度的方法及设备 |
JP6356266B2 (ja) * | 2014-12-24 | 2018-07-11 | 株式会社日立国際電気 | 群集監視システム |
US10122483B2 (en) * | 2015-02-04 | 2018-11-06 | Ebay Inc. | Crowd sentiment detection and analysis |
US20170017846A1 (en) * | 2015-07-15 | 2017-01-19 | Umm Al-Qura University | Crowd and traffic monitoring apparatus and method |
US10497132B2 (en) * | 2015-07-17 | 2019-12-03 | Nec Corporation | Irradiation system, irradiation method, and program storage medium |
JP2019114821A (ja) * | 2016-03-23 | 2019-07-11 | 日本電気株式会社 | 監視システム、装置、方法およびプログラム |
WO2019226898A1 (en) * | 2018-05-23 | 2019-11-28 | Trustees Of Boston University | Fusion-based occupancy sensing for building systems |
-
2014
- 2014-05-21 CN CN201810432697.8A patent/CN108647631B/zh active Active
- 2014-05-21 JP JP2015523834A patent/JP6008045B2/ja active Active
- 2014-05-21 US US14/899,340 patent/US9875431B2/en active Active
- 2014-05-21 EP EP14817648.0A patent/EP3016069A4/en active Pending
- 2014-05-21 EP EP17199965.9A patent/EP3312770B1/en active Active
- 2014-05-21 CN CN201480036661.8A patent/CN105593901B/zh active Active
- 2014-05-21 WO PCT/JP2014/002670 patent/WO2014207991A1/ja active Application Filing
-
2016
- 2016-07-21 HK HK16108787.2A patent/HK1220795A1/zh unknown
- 2016-07-21 HK HK18116543.8A patent/HK1257339A1/zh unknown
-
2017
- 2017-08-03 US US15/667,937 patent/US10223620B2/en active Active
-
2018
- 2018-06-13 HK HK18107654.2A patent/HK1250542A1/zh unknown
- 2018-12-04 US US16/209,277 patent/US10776674B2/en active Active
- 2018-12-04 US US16/209,290 patent/US10515294B2/en active Active
-
2019
- 2019-11-20 US US16/689,406 patent/US11132587B2/en active Active
-
2021
- 2021-07-19 US US17/379,392 patent/US11836586B2/en active Active
-
2023
- 2023-07-06 US US18/218,903 patent/US20230351258A1/en active Pending
- 2023-07-06 US US18/218,916 patent/US20230351259A1/en active Pending
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2007249588A (ja) * | 2006-03-15 | 2007-09-27 | Omron Corp | 顔画像登録装置、顔画像登録方法、顔画像登録プログラム、および記録媒体 |
JP2007329762A (ja) * | 2006-06-08 | 2007-12-20 | Fujitsu Ten Ltd | 物体候補領域検出装置、物体候補領域検出方法、歩行者認識装置、および車両制御装置 |
JP2008140107A (ja) * | 2006-12-01 | 2008-06-19 | Seiko Epson Corp | 画像処理装置、画像処理方法、制御プログラム及び記録媒体 |
JP2010198566A (ja) | 2009-02-27 | 2010-09-09 | Nec Corp | 人数計測装置、方法及びプログラム |
JP2011076316A (ja) * | 2009-09-30 | 2011-04-14 | Fujifilm Corp | 群衆監視装置および方法ならびにプログラム |
JP2011100175A (ja) | 2009-11-04 | 2011-05-19 | Nippon Hoso Kyokai <Nhk> | 人物行動判定装置及びそのプログラム |
JP2012022370A (ja) | 2010-07-12 | 2012-02-02 | Hitachi Kokusai Electric Inc | 監視システムおよび監視方法 |
Cited By (65)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2021036437A (ja) * | 2015-01-14 | 2021-03-04 | 日本電気株式会社 | 移動状況推定装置、移動状況推定方法およびプログラム記録媒体 |
JPWO2016114134A1 (ja) * | 2015-01-14 | 2017-10-26 | 日本電気株式会社 | 移動状況推定装置、移動状況推定方法およびプログラム |
JP7163945B2 (ja) | 2015-01-14 | 2022-11-01 | 日本電気株式会社 | 移動状況推定装置、移動状況推定方法およびプログラム記録媒体 |
CN106033548B (zh) * | 2015-03-13 | 2021-04-20 | 中国科学院西安光学精密机械研究所 | 基于改进字典学习的拥挤人群异常检测方法 |
CN106033548A (zh) * | 2015-03-13 | 2016-10-19 | 中国科学院西安光学精密机械研究所 | 基于改进字典学习的拥挤人群异常检测方法 |
US10789484B2 (en) | 2016-03-07 | 2020-09-29 | Nec Corporation | Crowd type classification system, crowd type classification method and storage medium for storing crowd type classification program |
US11727578B2 (en) | 2016-03-07 | 2023-08-15 | Nec Corporation | Crowd type classification system, crowd type classification method and storage medium for storing crowd type classification program |
US10885350B2 (en) | 2016-03-07 | 2021-01-05 | Nec Corporation | Crowd type classification system, crowd type classification method and storage medium for storing crowd type classification program |
JPWO2017154655A1 (ja) * | 2016-03-07 | 2019-01-10 | 日本電気株式会社 | 群衆種類識別システム、群衆種類識別方法および群衆種類識別プログラム |
US10891493B2 (en) | 2016-03-07 | 2021-01-12 | Nec Corporation | Crowd type classification system, crowd type classification method and storage medium for storing crowd type classification program |
WO2017154655A1 (ja) * | 2016-03-07 | 2017-09-14 | 日本電気株式会社 | 群衆種類識別システム、群衆種類識別方法および群衆種類識別プログラムを記憶する記憶媒体 |
JP7180710B2 (ja) | 2016-03-09 | 2022-11-30 | 日本電気株式会社 | 画像処理装置、画像処理方法およびプログラム |
JP2021106044A (ja) * | 2016-03-09 | 2021-07-26 | 日本電気株式会社 | 画像処理装置、画像処理方法およびプログラム |
US11205275B2 (en) | 2016-03-18 | 2021-12-21 | Nec Corporation | Information processing apparatus, control method, and program |
US11361452B2 (en) | 2016-03-18 | 2022-06-14 | Nec Corporation | Information processing apparatus, control method, and program |
US11823398B2 (en) | 2016-03-18 | 2023-11-21 | Nec Corporation | Information processing apparatus, control method, and program |
US10699422B2 (en) | 2016-03-18 | 2020-06-30 | Nec Corporation | Information processing apparatus, control method, and program |
US11158068B2 (en) | 2016-03-18 | 2021-10-26 | Nec Corporation | Information processing apparatus, control method, and program |
US11030464B2 (en) | 2016-03-23 | 2021-06-08 | Nec Corporation | Privacy processing based on person region depth |
EP3435667A4 (en) * | 2016-03-23 | 2019-10-30 | Nec Corporation | MONITORING SYSTEM, IMAGE PROCESSING DEVICE, PICTURE PROCESSING METHOD AND PROGRAMMING MEDIUM |
CN108781277A (zh) * | 2016-03-23 | 2018-11-09 | 日本电气株式会社 | 监视系统,图像处理设备,图像处理方法和程序记录介质 |
WO2017163955A1 (ja) | 2016-03-23 | 2017-09-28 | 日本電気株式会社 | 監視システム、画像処理装置、画像処理方法およびプログラム記録媒体 |
JPWO2018025831A1 (ja) * | 2016-08-04 | 2019-06-13 | 日本電気株式会社 | 人流推定装置、人流推定方法およびプログラム |
US11106920B2 (en) | 2016-08-04 | 2021-08-31 | Nec Corporation | People flow estimation device, display control device, people flow estimation method, and recording medium |
WO2018025831A1 (ja) * | 2016-08-04 | 2018-02-08 | 日本電気株式会社 | 人流推定装置、表示制御装置、人流推定方法および記録媒体 |
US10936882B2 (en) | 2016-08-04 | 2021-03-02 | Nec Corporation | People flow estimation device, display control device, people flow estimation method, and recording medium |
US11074461B2 (en) | 2016-08-04 | 2021-07-27 | Nec Corporation | People flow estimation device, display control device, people flow estimation method, and recording medium |
US10970559B2 (en) | 2016-09-13 | 2021-04-06 | Nec Corporation | People flow estimation device, people flow estimation method, and recording medium |
US10810442B2 (en) | 2016-09-13 | 2020-10-20 | Nec Corporation | People flow estimation device, people flow estimation method, and recording medium |
US10970558B2 (en) | 2016-09-13 | 2021-04-06 | Nec Corporation | People flow estimation device, people flow estimation method, and recording medium |
JP2018060360A (ja) * | 2016-10-05 | 2018-04-12 | 日本電信電話株式会社 | 人数推定方法、人数推定装置、人数推定プログラム、及び記録媒体 |
US11989859B2 (en) | 2016-11-25 | 2024-05-21 | Nec Corporation | Image generation device, image generation method, and storage medium storing program |
US10878549B2 (en) | 2016-11-25 | 2020-12-29 | Nec Corporation | Image generation device, image generation method, and storage medium storing program |
US10846838B2 (en) | 2016-11-25 | 2020-11-24 | Nec Corporation | Image generation device, image generation method, and storage medium storing program |
US11620739B2 (en) | 2016-11-25 | 2023-04-04 | Nec Corporation | Image generation device, image generation method, and storage medium storing program |
JP2018148367A (ja) * | 2017-03-03 | 2018-09-20 | キヤノン株式会社 | 画像処理装置、画像処理システム、画像処理方法、及びプログラム |
WO2018216648A1 (ja) * | 2017-05-22 | 2018-11-29 | 日本電気株式会社 | 群衆状態認識装置、学習方法および学習プログラム |
US11157749B2 (en) | 2017-05-22 | 2021-10-26 | Nec Corporation | Crowd state recognition device, learning method, and learning program |
JPWO2018216648A1 (ja) * | 2017-05-22 | 2020-03-19 | 日本電気株式会社 | 群衆状態認識装置、学習方法および学習プログラム |
CN111742342A (zh) * | 2018-03-12 | 2020-10-02 | 日立产业控制解决方案有限公司 | 图像生成方法、图像生成装置以及图像生成系统 |
WO2019235192A1 (ja) * | 2018-06-06 | 2019-12-12 | 日本電信電話株式会社 | 領域抽出モデル学習装置、領域抽出モデル学習方法、プログラム |
JP2019212106A (ja) * | 2018-06-06 | 2019-12-12 | 日本電信電話株式会社 | 領域抽出モデル学習装置、領域抽出モデル学習方法、プログラム |
JP2020013467A (ja) * | 2018-07-20 | 2020-01-23 | 株式会社壽屋 | 分類器生成方法、機械学習用画像及び学習データ生成方法 |
JP2020080003A (ja) * | 2018-11-12 | 2020-05-28 | 株式会社イシダ | 教師データ生成方法、教師データ生成プログラム、教師データ生成装置、及び商品識別装置 |
JP7300699B2 (ja) | 2018-11-12 | 2023-06-30 | 株式会社イシダ | 教師データ生成方法、教師データ生成プログラム、教師データ生成装置、及び商品識別装置 |
JP7211428B2 (ja) | 2018-11-21 | 2023-01-24 | 日本電気株式会社 | 情報処理装置、制御方法、及びプログラム |
WO2020105146A1 (ja) * | 2018-11-21 | 2020-05-28 | 日本電気株式会社 | 情報処理装置、制御方法、及びプログラム |
JPWO2020105146A1 (ja) * | 2018-11-21 | 2021-10-07 | 日本電気株式会社 | 情報処理装置、制御方法、及びプログラム |
KR20200061478A (ko) * | 2018-11-23 | 2020-06-03 | 연세대학교 산학협력단 | 실시간 객체 탐지 시스템 학습을 위한 합성 데이터 생성 장치 및 방법 |
KR102163573B1 (ko) * | 2018-11-23 | 2020-10-12 | 연세대학교 산학협력단 | 실시간 객체 탐지 시스템 학습을 위한 합성 데이터 생성 장치 및 방법 |
US11386562B2 (en) | 2018-12-28 | 2022-07-12 | Cyberlink Corp. | Systems and methods for foreground and background processing of content in a live video |
KR20210092842A (ko) * | 2019-01-22 | 2021-07-26 | 미쓰비시덴키 가부시키가이샤 | 정보 처리 장치, 프로그램을 기록한 기록 매체 및 정보 처리 방법 |
US11967081B2 (en) | 2019-01-22 | 2024-04-23 | Mitsubishi Electric Corporation | Information processing apparatus, non-transitory computer-readable storage medium, and information processing method |
KR102355578B1 (ko) | 2019-01-22 | 2022-01-25 | 미쓰비시덴키 가부시키가이샤 | 정보 처리 장치, 프로그램을 기록한 기록 매체 및 정보 처리 방법 |
JP2023501690A (ja) * | 2019-11-20 | 2023-01-18 | オムロン株式会社 | 群衆のダイナミックを予測する方法及びシステム |
JP7276607B2 (ja) | 2019-11-20 | 2023-05-18 | オムロン株式会社 | 群衆のダイナミックを予測する方法及びシステム |
JP2021196755A (ja) * | 2020-06-11 | 2021-12-27 | 日本電信電話株式会社 | 画像処理装置、画像処理方法及び画像処理プログラム |
JP7481171B2 (ja) | 2020-06-11 | 2024-05-10 | 日本電信電話株式会社 | 画像処理装置、画像処理方法及び画像処理プログラム |
JP7265672B2 (ja) | 2020-09-28 | 2023-04-26 | ソフトバンク株式会社 | 情報処理方法、プログラムおよび情報処理装置 |
JP2022140776A (ja) * | 2020-09-28 | 2022-09-27 | ソフトバンク株式会社 | 情報処理方法、プログラムおよび情報処理装置 |
JP7287734B2 (ja) | 2020-10-14 | 2023-06-06 | 国立研究開発法人海洋研究開発機構 | 魚数算出方法、魚数算出プログラム、及び、魚数算出装置 |
WO2022080407A1 (ja) * | 2020-10-14 | 2022-04-21 | 国立研究開発法人海洋研究開発機構 | 魚数算出方法、魚数算出プログラム、及び、魚数算出装置 |
JPWO2022080407A1 (ja) * | 2020-10-14 | 2022-04-21 | ||
CN115880648B (zh) * | 2023-03-08 | 2023-05-12 | 城云科技(中国)有限公司 | 无人机角度下的人群聚集识别方法、系统及其应用 |
CN115880648A (zh) * | 2023-03-08 | 2023-03-31 | 城云科技(中国)有限公司 | 无人机角度下的人群聚集识别方法、系统及其应用 |
Also Published As
Publication number | Publication date |
---|---|
EP3016069A4 (en) | 2017-06-14 |
US20210350191A1 (en) | 2021-11-11 |
US9875431B2 (en) | 2018-01-23 |
JP6008045B2 (ja) | 2016-10-19 |
US10776674B2 (en) | 2020-09-15 |
US20190102660A1 (en) | 2019-04-04 |
US20230351259A1 (en) | 2023-11-02 |
JPWO2014207991A1 (ja) | 2017-02-23 |
EP3312770B1 (en) | 2023-05-10 |
US11132587B2 (en) | 2021-09-28 |
HK1250542A1 (zh) | 2018-12-21 |
CN105593901B (zh) | 2020-06-12 |
US20190102661A1 (en) | 2019-04-04 |
US20170330061A1 (en) | 2017-11-16 |
HK1257339A1 (zh) | 2019-10-18 |
US11836586B2 (en) | 2023-12-05 |
US20200090013A1 (en) | 2020-03-19 |
CN108647631B (zh) | 2023-04-07 |
EP3312770A1 (en) | 2018-04-25 |
HK1220795A1 (zh) | 2017-05-12 |
US20160132755A1 (en) | 2016-05-12 |
CN108647631A (zh) | 2018-10-12 |
US10223620B2 (en) | 2019-03-05 |
CN105593901A (zh) | 2016-05-18 |
US10515294B2 (en) | 2019-12-24 |
US20230351258A1 (en) | 2023-11-02 |
EP3016069A1 (en) | 2016-05-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6008045B2 (ja) | 教師データ生成装置、方法、プログラム、および群衆状態認識装置、方法、プログラム | |
US11017588B2 (en) | Image processing apparatus that generates a virtual view image from multiple images captured from different directions and method controlling the same | |
JP4799105B2 (ja) | 情報処理装置及びその制御方法、コンピュータプログラム、記憶媒体 | |
JP4799104B2 (ja) | 情報処理装置及びその制御方法、コンピュータプログラム、記憶媒体 | |
CN105184249A (zh) | 用于人脸图像处理的方法和装置 | |
US20170116705A1 (en) | Method for automatic facial impression transformation, recording medium and device for performing the method | |
KR102280201B1 (ko) | 머신 러닝을 이용하여 은닉 이미지를 추론하는 방법 및 장치 | |
CN105096354A (zh) | 一种图像处理的方法和装置 | |
KR20220063256A (ko) | 캐빈 내부 환경의 조절 방법 및 장치 | |
KR20110014450A (ko) | 얼굴 인식률 개선 장치 및 방법 | |
CN109964229A (zh) | 用于人脸对准的形状预测 | |
KR20180074124A (ko) | 얼굴 인식을 통해 전자 장치를 제어하는 방법 및 이를 수행하는 전자 장치 | |
JP2011232845A (ja) | 特徴点抽出装置および方法 | |
JP7385416B2 (ja) | 画像処理装置、画像処理システム、画像処理方法及び画像処理プログラム | |
JP6820489B2 (ja) | 画像処理装置、および、画像処理プログラム | |
CN109492755B (zh) | 图像处理方法、图像处理装置和计算机可读存储介质 | |
JP2011198006A (ja) | オブジェクト検出装置、オブジェクト検出方法、およびオブジェクト検出プログラム | |
WO2023189601A1 (ja) | 情報処理装置、記録媒体及び情報処理方法 | |
JP2022066413A (ja) | 画像判別装置,画像判別方法ならびに画像判別装置のプログラムおよびそのプログラムを格納した記録媒体 | |
JP5962491B2 (ja) | 画角調整装置、方法、およびプログラム | |
JP2017049655A (ja) | 画像表示装置及び方法、プログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14817648 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2015523834 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2014817648 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 14899340 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |