CN102446210B - Image processing apparatus and image processing method - Google Patents

Image processing apparatus and image processing method Download PDF

Info

Publication number
CN102446210B
CN102446210B CN201110259511.1A CN201110259511A CN102446210B CN 102446210 B CN102446210 B CN 102446210B CN 201110259511 A CN201110259511 A CN 201110259511A CN 102446210 B CN102446210 B CN 102446210B
Authority
CN
China
Prior art keywords
user
display
information
display information
weight
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201110259511.1A
Other languages
Chinese (zh)
Other versions
CN102446210A (en
Inventor
鹤见辰吾
望月俊助
前川宗周
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Corp
Original Assignee
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from JP2010198986A external-priority patent/JP2012058838A/en
Application filed by Sony Corp filed Critical Sony Corp
Publication of CN102446210A publication Critical patent/CN102446210A/en
Application granted granted Critical
Publication of CN102446210B publication Critical patent/CN102446210B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

Provide image processing apparatus and image processing method.This image processing apparatus includes: recognition unit, for identifying the multiple users in input picture；Information acquisition unit, for obtaining the display information being associated with each user；Weight determining unit, for determining the weight of each user；Output image generation unit, for determined by weight based on each user with the display position of the display information that each user is associated and over an input image determined by show that in position, this display information overlapping produces output image, the display information being wherein associated with the user with the biggest weight displacement away from default display position is the least, wherein the display position of the first display information of first user is determined so that repulsion and gravitation are in poised state, repulsion depends on the distance between the first display information and the display information of another user, gravitation depends on weight and the displacement away from default display position of the first display information of first user.

Description

Image processing apparatus and image processing method
Technical field
It relates to image processing apparatus, program and image processing method.
Background technology
In recent years, the technology being referred to as augmented reality (AR) has caused concern, and this technology exists By overlay information on the image that capture realistic space obtains and present the image as result to user. The information using AR technology overlapping on image is polytype.Such as, JP 2010-158056A Disclose a kind of hyperlinked information is attached to present in input picture, movement in realistic space On object and present the technology of the image as result.
Summary of the invention
But, when input picture comprises multiple object that should be attached information, information to be shown can Can mix, and therefore possible loss presents to the intelligibility exporting image of user.Such as, Between the user via the image of augmented reality in communication, if not showing with making any distinction between and playing an active part in Communication user-dependent information and with another the user-dependent information being positioned in peripheral region, then Information owing to mixing may interfere with smooth communication, it is thus possible to is not easy to know which user Have sent the situation of which information.
In view of above-mentioned, it is desirable to provide at image processing apparatus, program and the image of a kind of new improvement Reason method, it can be to be easier to understand in the case of information is mingled in the image of augmented reality Mode presents information.
According to embodiment of the disclosure, it is provided that a kind of image processing apparatus, comprising: recognition unit, It is arranged to identify multiple users present in the input picture captured by imaging device；Acquisition of information Unit, is arranged to obtain the display to be shown being associated with each user of recognition unit identification Information；Weight determining unit, is arranged to determine the weight of each user of recognition unit identification； And output image generation unit, it is configured to each use determined based on weight determining unit The weight at family determines the display position over an input image of the display information being associated with each user Show overlapping described display information in position determined by, produce output image, wherein said defeated Go out image generation unit and determine the display position of described display information so that with the use with the biggest weight The display information that family the is associated displacement away from default display position is the least, and wherein said output image produces Unit determines the display position of the first display information of first user so that repulsion and gravitation are in balance State, described repulsion depends on the first display information of first user in the picture and the aobvious of another user Show the distance between information, and described gravitation depends on weight and the first display letter of first user The breath displacement away from default display position.
Weight determining unit can be according to each user time span present in the described input picture Determine the described weight of each user.
Information acquisition unit can also obtain the attribute information of each user of recognition unit identification；And Weight determining unit can use described attribute information that information acquisition unit obtains to determine each use The described weight at family.
Weight determining unit may determine that the described weight of each user so that belongs to particular sex or spy The weight of the user of dating group goes above the weight of other users.
The described display position of the described display information determined when described weight based on each user is full Foot display in terms of restriction time, output image generation unit can be by described display information placement described In display position.
Output image generation unit can be based on relative with each user distance away from described imaging device The parameter answered is to before determining which the display information in overlapped display information to be superimposed over.
When in described output image, the first display information is overlapped with the second display information, if The user being associated with the first display information distance away from described imaging device is shorter than and the second display letter The user of the manner of breathing association distance away from described imaging device, then output image generation unit can be by described First display information is placed in before described second display information.
Recognition unit can also identify the facial zone of each user present in the described input picture Size；And output image generation unit can use each user described of recognition unit identification The described size of facial zone is as described parameter.
Output image generation unit can measure each user of recognition unit identification in the input image The time span existed or the translational speed of each user；And output image generation unit can basis The described time span measured for the described user being associated with described display information or described movement Speed is arranged on described input picture the transparency of the described display information of overlap.
Output image generation unit can will exist the user's of long period on described input picture The described transparency of described display information is set to relatively low level.
Output image generation unit can be by the institute of the described display information of user relatively low for translational speed State transparency and be set to relatively low level.
Output image generation unit can determine according to each user distance away from described imaging device The display size of the described display information being associated with each user.
Another embodiment according to the disclosure, it is provided that a kind of program, this program makes to control at image The computer of reason device has following functions: recognition unit, is arranged to identify and is caught by imaging device Multiple users present in the input picture obtained；Information acquisition unit, is arranged to obtain and identify The display information to be shown that each user of unit identification is associated；Weight determining unit, is configured For determining the weight of each user of recognition unit identification；And output image generation unit, joined Put the described weight for each user by determining based on weight determining unit to determine and each use The display position of the described display information that family is associated and on described input picture determined by aobvious Show overlapping described display information in position, produce output image.
Output image generation unit may determine that the described display position of described display information so that with tool There is the displacement away from default display position of display information that the user of greater weight is associated less.
Output image generation unit may determine that the display position of the first display information of first user, makes Obtaining repulsion and gravitation is in poised state, described repulsion depends on first user described in described image Described first display information and the display information of another user between distance, and described gravitation takes Certainly in weight and the described first display information displacement away from default display position of described first user.
Weight determining unit can be according to each user time span present in the described input picture Determine the described weight of each user.
Information acquisition unit can also obtain the attribute information of each user of recognition unit identification；And Weight determining unit can use described attribute information that information acquisition unit obtains to determine each use The described weight at family.
Another embodiment according to the disclosure, it is provided that a kind of information processing method, comprising: identify Multiple users present in the input picture captured by imaging device；Obtain the user's phase with each identification The display information to be shown of association；Determine the weight of the user of each identification；Based on described user Determined by weight determine the display position of the described display information being associated with each user；And By on described input picture determined by show that in position, overlapping described display information produces Output image, wherein, the display position of described display information is determined so that and has the biggest weight The displacement away from default display position of the display information that is associated of user the least, wherein, first user First display information display position be determined so that repulsion and gravitation are in poised state, described in scold Power depends between the first display information and the display information of another user of first user in the picture Distance, and described gravitation depends on that the weight of first user and the first display information are away from default aobvious Show the displacement of position.
As it has been described above, in the case of information is mingled in the image of augmented reality, real according to the present invention Execute the image processing apparatus of example, program and image processing method to allow to present in a more easily understood manner Information.
Accompanying drawing explanation
Fig. 1 is the schematic diagram of the general introduction illustrating image processing system；
Fig. 2 is the explanation of the example illustrating the output image utilizing the image processing system of Fig. 1 to show Figure；
Fig. 3 is the block diagram of the exemplary configuration illustrating the image processing apparatus according to first embodiment；
Fig. 4 is the explanatory diagram of the example illustrating the user interface for registering new user；
Fig. 5 is the explanatory diagram of the example arrangement illustrating display object data；
Fig. 6 A is the explanatory diagram of the first example of the shape illustrating display object；
Fig. 6 B is the explanatory diagram of the second example of the shape illustrating display object；
Fig. 6 C is the explanatory diagram of the 3rd example of the shape illustrating display object；
Fig. 6 D is the explanatory diagram of the 4th example of the shape illustrating display object；
Fig. 7 is the explanatory diagram of the display position illustrating the display object according to first embodiment；
Fig. 8 A is the explanatory diagram of the example illustrating that transparency (transparency) arranges process；
Fig. 8 B is the explanatory diagram of another example illustrating that transparency arranges process；
Fig. 9 A is the first explanatory diagram of the example illustrating that layer arranges process；
Fig. 9 B is the second explanatory diagram of the example illustrating that layer arranges process；
Figure 10 is the explanatory diagram of the example illustrating the output image according to first embodiment；
Figure 11 is the flow chart of the exemplary flow illustrating the image procossing according to first embodiment；
Figure 12 is the block diagram of the exemplary configuration illustrating the image processing apparatus according to the second embodiment；
Figure 13 is the explanatory diagram of the example illustrating that weight determines process；
Figure 14 A is the first explanatory diagram of the first example illustrating that display position determines process；
Figure 14 B is the second explanatory diagram of the first example illustrating that display position determines process；
Figure 14 C is the 3rd explanatory diagram of the first example illustrating that display position determines process；
Figure 15 is the explanatory diagram of the second example illustrating that display position determines process；
Figure 16 is the explanatory diagram of the example illustrating the output image according to the second embodiment；
Figure 17 is the flow chart of the exemplary flow illustrating the image procossing according to the second embodiment；
Figure 18 is to illustrate the first exemplary flow that the display position according to the second embodiment determines process Flow chart；
Figure 19 is to illustrate the second exemplary flow that the display position according to the second embodiment determines process Flow chart.
Detailed description of the invention
Hereinafter, preferred embodiment of the present disclosure will be described in detail by referring to the drawings.Note, in this explanation In book and accompanying drawing, carry out labelling by identical reference and there is the structure of function and the structure being substantially the same Become key element, and eliminate the repeat specification to these elements.
Image processing apparatus according to disclosure embodiment may include that recognition unit, is arranged to Identify multiple users present in the input picture captured by imaging device；Information acquisition unit, is joined Put the display information to be shown being associated for acquisition with each user of described recognition unit identification； Weight determining unit, is arranged to determine the weight of each user of described recognition unit identification；With And output image generation unit, it is configured in the following manner to produce output image: based on institute State the described weight of each user that weight determining unit determines to determine and be associated with each user The display position of described display information, and on described input picture determined by show in position weight Folded described display information.
Image processing method according to disclosure embodiment may include that identification is captured by imaging device Multiple users present in input picture；Obtain to be shown the showing being associated with the user of each identification Show information；Determine the weight of the user of each identification；Come really based on weight determined by each user The display position of the fixed described display information being associated with described user；And by described input figure On as determined by show that in position, overlapping described display information is to produce output image.
Image processing apparatus according to disclosure embodiment may include that recognition unit, is arranged to Identify multiple users present in the input picture captured by imaging device；Information acquisition unit, is joined Put the display information to be shown being associated for acquisition with each user of described recognition unit identification； And output image generation unit, it is configured to over an input image in default display position Overlapping described display information produces output image.
Image processing method according to disclosure embodiment may include that identification is captured by imaging device Multiple users present in input picture；Obtain to be shown the showing being associated with the user of each identification Show information；And by described input picture in default display position overlapping described display information Produce output image.
To describe " detailed description of the invention " according to order below
1. system survey
2. the description of first embodiment
The exemplary configuration of 2-1. image processing apparatus
The attribute of 2-2. display object
The example of 2-3. output image
2-4. handling process
The conclusion of 2-5. first embodiment
3. the description of the second embodiment
The exemplary configuration of 3-1. image processing apparatus
The example of 3-2. output image
3-3. handling process
The conclusion of 3-4. the second embodiment
＜ 1. system survey ＞
First, the general introduction of image processing system according to an embodiment of the invention will be described with reference to Fig. 1. Fig. 1 is the schematic diagram of the general introduction of the image processing system 1 illustrating an embodiment according to the disclosure. With reference to Fig. 1, image processing system 1 includes image processing apparatus 100, data base 102, imaging dress Put 104 and display device 106.
Image processing apparatus 100 is connected to data base 102, imaging device 104 and display device 106. Image processing apparatus 100 can be the logical of the most such as PC (personal computer) or work station etc With computer or special-purpose computer for a specific purpose.Discussed in more detail below, image processing apparatus 100 obtain the image that captured by imaging device 104 as input picture, and to display device 106 Export processed output image.
Data base 102 is the dress processing information to be used for storing image processing apparatus 100 execution Put.Data base 102 is not limited to the example of Fig. 1, and can build in image processing apparatus 100. In data base 102, the example of the information of storage includes: uniquely identify each of image processing system 1 The identifier (hereinafter referred to as ID) of user, for identify each user characteristic quantity information, The attribute information of each user and view data.In response to when image processing apparatus 100 execution processes Request, can from data base 102 in output database 102 storage information.Alternatively, Image processing apparatus 100 can download the information being stored in data base 102 termly.
Imaging device 104 is the device that capture user may reside in the image of real space therein. Imaging device 104 is disposed in the upside of screen 107 so that imaging device 104 is facing to screen 107 The space in front.Imaging device 104 captures the image of the real space in screen 107 front, and with Seasonal effect in time series mode exports image series (that is, video) to image processing apparatus 100.
Display device 106 is to show the dress of the series output image produced by image processing apparatus 100 Put.In the example of fig. 1, display device 106 is projector.Display device 106 will be at image The output image projection of reason device 100 input is on screen 107.In this case, display device 106 is rear projector for projecting.Noting, display device 106 is not limited to the example of Fig. 1, and permissible It is PDP (plasma display), LCD (liquid crystal display) etc..
Screen 107 is the display screen of display device 106.In image processing system 1, display dress The display screen putting 106 is arranged such that this display screen is facing to wherein there is the true empty of user Between.In the example of fig. 1, before three users Ua, Ub and Uc are positioned at screen 107.
The user of image processing system 1 can use termination to hand over image processing apparatus 1 Mutually.In the example of fig. 1, user Ua keeps termination 105.Termination 105 can be Such as PC, smart phone or PDA (personal digital assistant).Termination 105 is according to such as example As WLAN (LAN),OrAny channel radio of (bluetooth) etc. Letter agreement communicates with image processing apparatus 100.User Ua such as can use termination 105 Input text or voice or the information of registration user.
Fig. 2 is to illustrate the output figure of display in the image processing system 1 being exemplarily illustrated in FIG The explanatory diagram of the example of picture.With reference to Fig. 2, show output image Im01 as example.In output Image Im01 exists three users Ua, Ub and Uc.Display object 12a, 12b and 12c divide Do not overlap on the region around three users Ua, Ub and Uc.Each display to as if be used for showing Show the object of the information being associated with corresponding user.In this manual, by image processing apparatus The information being associated with user of 100 displays should be referred to as " display information ".In the figure 2 example, The face of each corresponding user included as display information in display object 12a, 12b and 12c Image, the pet name and attribute information (such as, hobby).Additionally, display object 13a overlaps user On region near Ua.Display object 13a comprises being inputted by user Ua as display information Message.Will be described in as follows by image processing apparatus 100 overlapping this display object on image.
This image processing system 1 may be used for intensification and is in the most such as party, meeting room or exhibition etc. The place that exchanges each other of many people in user between exchange.Alternatively, image processing system In 1 business scenario that may be used for such as video conference etc..In this case, in each local layout Imaging device and display device so that the video in given local capture can exist with display information Another local display.
Here, when the input picture in image processing system 1 exists multiple user, it may occur however that The situation of multiple display object should be shown in the output image.In this case, it is used for watching output figure The intelligibility of the information of the user of picture depend on each display object where be disposed in and How to be arranged and different.If information indigestion, then may interrupt smooth communication. Therefore, description is used for presenting information in a more easily understood manner to support to smooth by below portion Two embodiments of communication.
The description ＞ of ＜ 2. first embodiment
Exemplary configuration ＞ of ＜ 2-1. image processing apparatus
Fig. 3 is the exemplary configuration illustrating image processing apparatus 100 according to a first embodiment of the present invention Block diagram.With reference to Fig. 3, image processing apparatus 100 includes that image acquisition unit 110, voice obtain Unit 120, recognition unit 130, information acquisition unit 150 and output image generation unit 170. Additionally, recognition unit 130 includes that image identification unit 134, voice recognition unit 138 and individual know Other unit 142.
(image acquisition unit)
Image acquisition unit 110 obtains the serial input picture captured by imaging device 104.Then, Image acquisition unit 110 produces single to image identification unit 134 and the output image of recognition unit 130 Input picture acquired in unit 170 output.
(voice acquiring unit)
Voice acquiring unit 120 obtains the voice as input voice sent by user.Can pass through The voice signal such as receiving termination 105 transmission from user's holding performs to utilize voice to obtain Take the unit 120 capture to voice.Alternatively, can around screen 107 cloth microphone. In the case of the latter, voice acquiring unit 120 obtains input voice via the mike arranged. Then, voice acquiring unit 120 exports acquired to the voice recognition unit 138 of recognition unit 130 Input voice.
(image identification unit)
Image identification unit 134 is known to the input picture application inputted from image acquisition unit 110 Face recognition method (for example, with reference to JP 2008-131405A), and detect present in input picture The facial zone of user.Additionally, for the facial zone of detection, image identification unit 134 calculates to be used Characteristic quantity (referred to below as image feature amount) in individuality identification.Then, image identification unit 134 The image feature amount calculated is exported to individual's recognition unit 142.
Then, the user corresponding to each facial zone has been identified (when individual's recognition unit 142 Identify the ID corresponding to each facial zone) time, image identification unit 134 by user with Identified ID is associated, and represents each face to output image generation unit 170 output The information of the positions and dimensions in region.
Additionally, image feature amount based on each user, image identification unit 134 can identify as follows Attribute: the facial expression of such as user (such as, smiling), (whether user for the talk situation of user In speech) or the sex of user or age group etc..In the case, image identification unit 134 is to defeated Go out image generation unit 170 output and represent identified facial expression, talk situation, sex, age The information of group etc..
Additionally, image identification unit 134 can also detect the hand district of user present in input picture Territory, and the mobile route of position based on the hand region detected identifies the posture of user.At this In the case of, image identification unit 134 represents identified appearance to output image generation unit 170 output The information of the type of gesture.
(voice recognition unit)
Voice recognition unit 138 is to known to the input voice application inputted from voice recognition unit 120 Audio recognition method, and voice user sent is extracted as text data (hereinafter referred to as " voice number According to ").Then, the speech data extracted is associated by voice recognition unit 138 with ID, And it is output to information acquisition unit 150.
When obtaining input voice via termination 105, voice recognition unit 138 can be based on work The user of correspondence is identified for the device identification of termination 105 of transmission sources, account identification etc..With Time, when obtaining input voice via the speech input device being arranged in around screen 107, such as, By in contradistinction to checking from input with the phonetic feature amount of user that pre-registers in data base 102 The phonetic feature amount that voice extracts, voice recognition unit 138 can identify unique user.Additionally, example As, voice recognition unit 138 can estimate to input the direction of the speech source of voice, and based on being estimated The direction of the speech source of meter identifies unique user.
(individual's recognition unit)
Individual's recognition unit 142 identifies present in the input picture captured by imaging device 104 one Or each user in multiple user.More specifically, such as, individual's recognition unit 142 and data base The image feature amount of the face of the known users pre-registered in 102 in contradistinction to checks to be known from image The image feature amount of other unit 134 input (for inspection method, see for example JP 2009- 53916A).Then, individual will be identified by image identification unit 134 each of recognition unit 142 Region, portion is associated with the identified ID as the user checking result.Alternatively, example As, individual's recognition unit 142 can be with the voice of known users that pre-registers in data base 102 Phonetic feature amount in contradistinction to checks the phonetic feature amount from voice recognition unit 138 input.
(information acquisition unit)
What information acquisition unit 150 acquisition was associated with each user identified by recognition unit 130 wants The display information of display.In this embodiment, the display information to be shown being associated with each user The example attribute information that can include user and the input information that inputted by user.
Information acquisition unit 150 obtains such as from the attribute information of user of data base 102.By believing Breath acquiring unit 150 is in advance data base 102 by user from the attribute information that data base 102 obtains Middle registration.In data base 102, the attribute information of registration can be any information, such as user The pet name, age, sex, hobby or group/department, or user is for the answer of particular problem.Replace For property ground, such as, information acquisition unit 150 can obtain as attribute information by image recognition list The sex of each user of unit 134 identification, age group etc..
The input information as display information that information acquisition unit 150 obtains includes such as via literary composition The text of this input equipment input.Such as, user can use termination 105 to input as text Device inputs text, then from termination 105, the text is sent to image as input information Processing means 100.Additionally, the input information as display information that information acquisition unit 150 obtains Including the aforementioned voice data such as identified by voice recognition unit 138.
Additionally, information acquisition unit 150 may search for defeated with the attribute information or user being included in user The oracle of the given information that the keyword in the input information entered matches, then obtains conduct The information (hereinafter referred to as searching for information) that Search Results obtains is as display information.Oracle can To be the most online dictionary services, SNS (social networking service) or knowledge sharing service etc The service relevant to network.
Additionally, information acquisition unit 150 provides such as user's registration and new user-dependent information User interface (UI).Screen 107 is may be displayed on for the UI of user's registration.For with The UI of family registration can be the UI such as using the image as being exemplarily illustrated in Fig. 4.According to figure The example of the UI shown in 4, selects the posture of territory 19a or 19b to answer display by carrying out touching Problem 18 on screen 107, user can register his/her attribute in image processing system 1 Information.Alternatively, can be via the specific screens of image processing apparatus 100 or termination 105 Screen provide for user registration user interface.
(output image generation unit)
Output image generation unit 170 is by the input picture inputted by image acquisition unit 110 The overlapping display information obtained by information acquisition unit 150 is to produce output image.More specifically, it is defeated Go out image generation unit 170 and first determine the display letter obtained by information acquisition unit 150 for display The attribute of the display object of breath.The example of the attribute of display object includes and the display shape of object, face Color, size, display position, data that transparency is relevant with layer.Wherein, layer represents that each display is right Sequence number for the order overlapped for display object of elephant.Such as, when multiple display object phases Time the most overlapping, the display object of lower level is placed in before relatively.Every determine for each display information After the attribute of individual display object, output image generation unit 170 according to determined by attribute produce every The image of individual display object.Next part will be described in further detail utilization output image generation unit 170 Determine the standard of the attribute of each display object.Then, output image generation unit 170 is by defeated Enter on image the image of overlapping produced display object to produce output image, and to display device 106 sequentially export produced output image.
[attribute of 2-2. display object]
(1) example of attribute
Fig. 5 is to illustrate the display number of objects included by exporting the property value that image generation unit 170 determines Explanatory diagram according to the example arrangement of 180.With reference to Fig. 5, display object data 180 have include right As mark 181, ID 182, shape 183, color 184, size 185, display position 186, transparency 187, layer 188 and nine data item of display information 189.
(2) object identity and ID
Object identity 181 is the mark of each display object overlapping in unique mark single image Know symbol.ID 182 is the ID representing the user being associated with display object, wherein Show to as if identify with object identity 181.Such as, from the of display object data 180 One record 190a and second record 190b be appreciated that two display object D01A and D02A with User Ua is associated.Additionally, be appreciated that display object D01B and use from the 3rd record 190c Family Ub is associated, and, it is appreciated that display object D01C and use from the 4th record 190d Family Uc is associated.
(3) shape
Shape 183 represents the shape of display object.In the example of hgure 5, pre-defined by regulation Display type Obj1 of object, Obj2...... in any one identify display object shape.
Fig. 6 A to Fig. 6 D is the explanatory diagram of the example of the type each illustrating display object.With reference to figure 6A, it is shown that display object 12a and 13a illustrated in Fig. 2.In fig. 6, display The type of object 12a is type " Obj1 ", and shows that the type of object 13a is type " Obj2 ". The display object of these types Obj1 and Obj2 is respectively provided with so-called speech balloon (speech balloon) Shape.
It follows that with reference to Fig. 6 B, it is shown that display object 14a.The type of display object 14a is “Obj3”.Display object 14a has the shape hanging sign board piggyback.Such as, Fig. 6 A illustrates Type Obj1 and Fig. 6 B shown in type Obj3 be displayed for the attribute information of user.With Time, such as, type Obj2 shown in Fig. 6 A is displayed for the display information inputted by user.
Additionally, with reference to Fig. 6 C, it is shown that display object 15a.The type of display object 15a is “Obj4”.Such as, type Obj4 can be also used for the display information that display is inputted by user.
Here, the shape of type Obj2 shown in Fig. 6 A is to represent the shape that user talks.Meanwhile, The shape of type Obj4 shown in Fig. 6 C is the shape representing user's idea.When display information be through During the input information inputted by speech input device, output image generation unit 170 can such as will be used The type with the shape representing speech it is set in the display object showing described display information Obj2.Additionally, when the input information that display information is via text input device input, output By being used for, image generation unit 170 such as can show that the display object of described display information is set to There is type Obj4 of the shape representing idea.Alternatively, such as, output image generation unit By the content analyzing display information, 170 can determine that this display information corresponds to thinking of user The information of method also corresponds to the information of the speech of user, and the display that would correspond to user's idea is right As be set to type Obj4 and would correspond to user speech display object be set to type Obj2.
With reference to Fig. 6 D, it is shown that display object 16a.The type of display object 16a is " Obj5 ". Display object 16a also has the shape of speech balloon.But, the tail of the speech balloon of type Obj5 Portion is not directed to user and points up.Such as, output image generation unit 170 can show being used for The display object of information is set to type Obj5, and wherein, this information is by information acquisition unit 150 Obtain from oracle.Include that previous searches is believed from the example of the information of oracle acquisition Breath.
As it has been described above, ought be according to the acquisition approach of display information or the input medium being used for the information that inputs When changing the shape of display object, user is being led to another user using image processing system 1 The type of information can the most intuitively and be exactly understood during letter.Additionally, due to user can selectivity The shape of the object of the information (speech or idea) that ground use inputs for showing user, it is possible to Realize more rich information.
(4) color
Color 184 in Fig. 5 represents that the color of display object (or shows the literary composition of information in display object This color).Such as, output image generation unit 170 is referred to by information acquisition unit 150 The attribute information of each user obtained, and according to the attribute of the instruction sex of user, age group etc. Value changes the color of each display object.
(5) size
Size 185 represents the size of display object.In the example of hgure 5, the size quilt of object is shown It is expressed as the multiplying power (%) of default size.Such as, output image generation unit 170 is according to each use The family distance away from imaging device 104 determines the display information being associated for display with each user The size of display object.In this embodiment, as to measuring each user away from imaging device 104 The replacement of distance, output image generation unit 170 can use the chi of the facial zone of each user Very little as the parameter corresponding with each user distance away from imaging device 104.Such as, facial zone Size can be represented as being identified as belonging to the quantity of the pixel of this facial zone, or can be by It is expressed as the size of the bounding box around facial zone.More specifically, output image generation unit 170 The size of the display object of the display information that the display user bigger with facial zone is associated will be used for It is set to bigger size.Note, the upper limit of the size of display object can be limited in advance.In these feelings Under condition, output image generation unit 170 arranges the size of display object so that near imaging device The size of the display object of 104 users arriving the distance more than or equal to preset distance is less than this upper limit.
(6) display position
The display position of display position 186 instruction display object, say, that represent that display object exists The two-dimensional coordinate of position overlapping in image.In this embodiment, output image generation unit 170 Arrange each display object so that the center (or predetermined turning etc.) of display object is positioned at away from as ginseng The position of the facial zone predetermined offset of the user of examination point.
Fig. 7 is the explanatory diagram of the display position illustrating the display object according to this embodiment.With reference to Fig. 7, Show the position P0 of the center of gravity of the facial zone of user.P0 instruction in position is used for determining display object The reference point of side-play amount of display position.When the number of the display information being associated with given user is When 1, output image generation unit 170 will be used for showing the display of the display object of described display information Position is set to position P1.Alternatively, when the number of the display information being associated with given user During more than 1, output image generation unit 170 shows the aobvious of information by being used for second, third with the 4th Show that position is respectively set to position P2, P3 and P4.Position P0 and each position P1, P2, P3 With the side-play amount between P4 is predefined.In this manual, these display positions should be by It is referred to as " default display position ".Noting, the default display position shown in Fig. 7 is only exemplary.
Such as, when the type being exemplarily illustrated display object in Fig. 6 B is type " Obj3 ", aobvious The default display position showing object is position P5.Meanwhile, such as, when Fig. 6 D is exemplarily illustrated When the type of display object is type " Obj5 ", the default display position of display object can be position P6 and P7.
(7) transparency
Transparency 187 in Fig. 5 represents the transparency of display object.When being provided with for display object During transparency, even if multiple display object is overlapped, it is also possible to allow aobvious below of user's viewing Show object.In this embodiment, output image generation unit 170 is measured and is identified by recognition unit 130 Each user occur that (following, this time should be referred to as " stopping for time span in the input image Time ") or the translational speed of each user.Then, output image generation unit 170 is according to aobvious Show that the measured time of staying or the translational speed of the user that information is associated are provided for showing institute State the transparency of the display object of display information.
Fig. 8 A is the explanation of the example illustrating that the transparency of output image generation unit 170 arranges process Figure.In the example of Fig. 8 A, output image generation unit 170 is according in image during the stop of user Between arrange be associated with user show object transparency.
The horizontal axis representing time axle (time T) of Fig. 8 A, and the longitudinal axis represents the stop indicated by dotted line Time St and transparency level Tr indicated by solid line.In the example of Fig. 8 A, along with in the time T0Have occurred in the user in image and stay in the picture, the time of staying St of user along time Countershaft increases linearly.Meanwhile, at time T0The transparency Tr of display object is 100%.The most just It is to say, when the moment in the picture has just occurred in user, can't see display object.Then, along with stop Time St increases, and the transparency of display object Tr reduces.It is to say, along with user rests on figure In Xiang, the tone of display object gradually deepens.Then, when the transparency Tr of display object is in the time T1When reaching 20%, output image generation unit 170 stops reducing transparency Tr.This makes overlap Display object below can be seen the most to a certain extent.
Here, user stop time span in the picture be considered to represent user level interested or Person user's contribution level to the communication that image processing system 1 provides.Therefore, if the time of staying relatively The transparency of the display object of long user is arranged to reduced levels, then can preferentially make to have Stronger level interested or the user of high contribution level participate in communication.Meanwhile, if the time of staying relatively The transparency of the display object of short user is arranged to higher level, then can avoid with the most accidental The user-dependent unnecessary display information passed by before imaging device 104 is the most relatively Situation about occurring by force.Noting, as the replacement to the time of staying described here, output image produces Raw unit 170 can also use such as from user have started to use image processing system 1 through time Between.
Fig. 8 B is to illustrate another example that the transparency of output image generation unit 170 arranges process Explanatory diagram.In the example of Fig. 8 B, output image generation unit 170 is according to the shifting of user in image Dynamic speed arranges the transparency of the display object being associated with user.
The horizontal axis representing time axle (time T) of Fig. 8 B, and the longitudinal axis represents the movement indicated by dotted line Speed Mv and transparency level Tr indicated by solid line.In the example of Fig. 8 B, at time T0 The translational speed having occurred in the user in image is Mv1.At this moment, the transparency Tr of object is shown It is 100%.Afterwards, the translational speed of user is from time T0To T2Reduce, and the movement of user Speed is at time T2Increase afterwards.Transparency Tr followed by this change of translational speed Mv, from time Between T0To T2After reduction, transparency is at time T2The most again increase.
Here it is possible to think, image processing system 1 communication provided is had the use of stronger interest Family rests on before imaging device 104.Therefore, if the display of the relatively low user of translational speed is right The transparency of elephant is arranged to reduced levels, then can preferentially make this user participate in communication.Additionally, It is user-dependent unnecessary with the most accidentally pass by before imaging device 104 to avoid Display information situation about occurring the most strongly.Note, such as, export image generation unit 170 movements that can calculate user according to the position of centre of gravity of facial zone amount of movement between frames Speed.
Additionally, output image generation unit 170 can also be according to the time of staying of user and translational speed Both arrange the transparency of display object.Such as, even if when the translational speed of user increases, If the time of staying of this user is more than or equal to predetermined threshold, then output image generation unit 170 is permissible The transparency of the display object of this user is maintained low-level.Correspondingly, can avoid when user is temporary Time ground when moving in the picture the display object of this user be difficult to situation about being observed.Note, for Each user, output image generation unit 170 can also measure translational speed less than or equal to predetermined threshold The time span of halted state and translational speed more than the time span of the mobile status of this threshold value, Then the transparency of display object is set according to the ratio of the two time span.
Additionally, output image generation unit 170 can temporarily reduce user aobvious making predetermined gesture Show the transparency of object.Such as, raise one's hand posture, waving posture etc. can be relevant to the reduction of transparency Connection.Correspondingly, can easily make incipient (that is, its time of staying is short) but communication is had The user of stronger interest participates in communication.
(8) layer
Layer 188 in Fig. 5 represent display object for the order of overlapped display object Sequence number.In the example of hgure 5, in display object D01A and D02A, each layer is 1, aobvious The layer showing object D01B is 2, and the layer of display object D01C is 3.Therefore, output image produces Unit 170 carries out overlap to display object so that display object D01A and D02A is positioned at foremost Layer, display object D01B is positioned at next layer, and display object D01C is positioned at rearmost layer.
Output image generation unit 170 is based on corresponding with each user distance away from imaging device 104 Parameter determine the layer of each display object.In this embodiment, with each user away from imaging device The corresponding parameter of the distance of 104 can be the size of aforesaid facial zone.But, this parameter is not It is limited to this, and for example, it is also possible to the distance with distance measuring sensor measured by Shi Yonging or via Mike The volume of the voice of wind input is as parameter.Then, output image generation unit 170 will be used for showing The layer of the display object of the display information that the user shorter with the distance indicated by this parameter is associated sets It is set to lower value (that is, by this display object placement before relatively).
Fig. 9 A and 9B is the explanatory diagram of the example all illustrating that layer arranges process.
It is positioned at before screen 107 with reference to Fig. 9 A, three user Ua to Uc.User Ua, Ub and The Uc distance away from screen 107 is D1, D2 and D3 respectively.Wherein, distance D1 is the shortest, Distance D3 is the longest.In this case, the facial area of appearance user Ua in the input image Territory is maximum.The size of the facial zone of user Ub is second largest, the facial area of user Uc The size in territory is minimum.In the example of Fig. 9 A, the facial zone of user Ua, Ub and Uc Size be 1000 pixels, 800 pixels and 400 pixels respectively.Therefore, output image The layer of the display object being associated with user Ua is set to 1 by generation unit 170, will be with user Ub The layer of the display object being associated is set to 2, is set by the layer of the display object being associated with user Uc It is set to 3.
As it has been described above, when the display object of the user bigger for facial zone overlaps relatively above, Even if easily realizing when showing the display naturally that can also match when object mixes with distance perspective.This Outward, it is also possible to the user that hope participates in communication is directed to the position of closer screen 107.
Additionally, such as, output image generation unit 170 can be temporarily by the layer of following display object Arrange to not considering user's distance away from imaging device 104, wherein, this display object and with Lower user is associated: makes predetermined gesture or has the user of predetermined facial expression or talking User.Here predetermined gesture can be such as raise one's hand posture, posture etc. of waving.Additionally, here Predetermined facial expression can be smile etc..In the example of Fig. 9 B, although user Uc is away from screen 107 Distance the longest, but the layer of the user Uc of posture of waving temporarily is set to 1. This process can provide multiple user wishing to participate in communication to be able to carry out the chance of the extension communicated, and Prevent being located only within the user close to screen 107 and participate in communication strongly.
[example of 2-3. output image]
Figure 10 is to illustrate the output image from image processing apparatus 100 output according to this embodiment The explanatory diagram of example.With reference to Figure 10, it is shown that image Im11 is as example in output.At output image Im11 occurs in that three users Ua, Ub and Uc.Additionally, display object 12a and 13a is overlapping On region around user Ua.Display object 12b overlaps on the region around user Ub. Display object 12c overlaps on the region around user Uc.But, due to user Ua, Ub and Uc is the most close to each other, therefore show object be mingled in by dotted box around region in, and And therefore show that object 13a, 12b and 12c are overlapped.
Right lower quadrant at Figure 10 illustrates parts of images Im12, and it is aforementioned point in output image Im11 The enlarged drawing of the parts of images in wire frame.In parts of images Im12, display object 13a overlaps Foremost.Additionally, display object 12b overlaps after display object 13a, and show object 12c overlaps after display object 12b.That is, it will be appreciated that with play an active part in communication Relevant for the user Ua being in the position near screen 107 information displayed preferentially at output figure As in Im11.Additionally, the most from right to left across the display object of the user Uc walked The transparency of 12c is arranged to high level.Correspondingly, it is possible to prevent and is likely to be not involved in communicate Information relevant for user Uc hinders the visuality of out of Memory.
[2-4. handling process]
Figure 11 is that the primary picture illustrating the image processing apparatus 100 according to first embodiment processes stream The flow chart of the example of journey.
With reference to Figure 11, first image acquisition unit 110 obtains the input captured by imaging device 104 Image (step S102).Then, image acquisition unit 110 is to recognition unit 130 and output image Generation unit 170 exports acquired input picture.
It follows that recognition unit 130 detects facial zone and the hands of existing user in the input image Region, portion (step S104).It follows that recognition unit 130 by be stored in data base 102 The image feature amount of known users in contradistinction to check the characteristics of image of facial zone of each detection Amount identifies unique user (step S106) present in input picture.Additionally, by additionally making With phonetic feature amount and the speech source of detection from the input voice obtained by voice acquiring unit 120 Direction, recognition unit 130 can identify unique user.
It follows that information acquisition unit 150 is from data base 102, termination 105, external information Display information that each user that the acquisitions such as source identify with recognition unit 130 is associated, to be shown (step S108).
It follows that output image generation unit 170 measure in image by recognition unit 130 identify every The time of staying of individual user or translational speed (step S110).Then, time based on the stop measured Between or translational speed, output image generation unit 170 be provided for display by information acquisition unit 150 The transparency (step S112) of the display object of the display information obtained.Additionally, output image produces Unit 170 by use each user facial zone size as with user away from imaging device 104 The corresponding parameter of distance, size and the layer (step S114) of each display object are set.This In, output image generation unit 170 can be according to the face of the user identified by recognition unit 130 Expression, posture or talk situation change transparency or the layer of each display object.Additionally, output figure Other genus of the display position and color etc of the most each display object is set as generation unit 170 Property (step S116).
It follows that output image generation unit 170 is by existing according to the attribute set by display object On input picture, this display object overlapping produces output image (step S118).Then, output figure As generation unit 170 to display device 106 export produced by export image, so that display device 106 show this output image (step S120).
[conclusion of 2-5. first embodiment]
Below the first embodiment of the present invention is described referring to figs. 1 through Figure 11.According to this enforcement Example, in information is mingled in the image of augmented reality in the case of (wherein by weighing over an input image The folded information being associated with each user produces output image), according to each user away from imaging device Distance determine each display information layout in the depth direction.Accordingly, due to achieve with Distance perspective match naturally display, therefore user can easily understand which display information and which User is correlated with.Additionally, along with the user of hope participation communication is directed into the position near screen, phase Prestige makes to communicate more active.
Additionally, according to this embodiment, arrange according to the time of staying or the translational speed of user in image The transparency of each display object.Correspondingly, can present with right to user in more visual mode Communication has the user-dependent information of stronger interest or contributive to communicating with on higher degree User-dependent information.Further, it is also possible to prevent from hindering with the user-dependent information being not involved in communicating The visuality of out of Memory.Additionally, according to this embodiment, the shape of each display object is according to display The acquisition approach of information or for inputting the input medium of information and changing.Therefore, user can be more straight See and understand more accurately the type of display information.
The description ＞ of ＜ 3. second embodiment
In aforementioned first embodiment, though multiple aobvious in the case of display information mixes in the picture Show when information is overlapped, also can present these information in easily understandable manner.In first embodiment In, each display information is substantially in default display position display.In contradistinction to, describe in this part The second embodiment in, determine the display position of each display information adaptively, with suppression multiple The visual reduction that will occur in the case of display information is overlapped.
System configuration according to the second embodiment can be similar to the image procossing system described with reference to Fig. 1 The configuration of system 1.
Exemplary configuration ＞ of ＜ 3-1. image processing apparatus
Figure 12 is to illustrate that the exemplary of image processing apparatus 200 according to the disclosure the second embodiment is joined The block diagram put.With reference to Figure 12, image processing apparatus 200 includes that image acquisition unit 110, voice obtain Take unit 120, recognition unit 130, information acquisition unit 150, weight determining unit 260 and Output image generation unit 270.As the image processing apparatus 100 according to first embodiment, Recognition unit 130 includes image identification unit 134, voice recognition unit 138 and individual's recognition unit 142。
(weight determining unit)
Weight determining unit 260 determines the weight of each user identified by recognition unit 130.At this In embodiment, " weight " of terms user represents similar with user's priority in terms of display general Read, and the display information with the user of bigger " weight " is preferentially disposed near default aobvious Show the position of position.Such as, weight determining unit 260 can according to each user in the input image The time span (that is, the time of staying) existed determines the weight of this user.Alternatively, weight Determine that unit 260 can use the attribute information of each user to determine the weight of this user.More specifically Ground, such as, weight determining unit 260 may determine that the weight of each user so that belongs to specificity Not (such as, women) or year age group (such as, child less than 10 years old or more than 70 years old Old man) the weight of user go above the weight of other users.
Figure 13 is the explanatory diagram of the example illustrating that the weight of weight determining unit 260 determines process.Figure The horizontal axis representing time axle (time T) of 13, and the longitudinal axis represents the time of staying St indicated by dotted line Size with weight W indicated by solid line.In the example in figure 13, the type of weight W according to The attribute of user can be divided into following three classes: for the power of child's (such as, less than 10 years old) Weight Wk, for weight Wf of women, for weight Wm of male.When at time T0Occur When user in the picture stays in the picture, the time of staying, St increased linearly along time shaft Add.From time T0To T3, weight Wm the most linearly increases with time of staying St, then exists Time T3Become steady state value afterwards.From time T0To T3, weight Wf the most linearly increases, and then exists Time T3Become steady state value afterwards.But, in All Time, weight Wf is bigger than weight Wm dW.From time T0To T4, weight Wk linearly increases, then at time T4Become steady state value afterwards. From time T0To T3, weight Wk is more than weight Wf, determines that unit 260 stores such as basis The time of staying of each user and attribute precalculate figure or the computing formula of weight, and according to this Figure or computing formula determine the weight of each user.
As it has been described above, when the weight of the user with particular community is arranged to the weight than other user During bigger value, can preferentially process the user being expected to play an active part in communication, or can attract There is the concern of the user of particular community.Further, it is also possible to by institute's honor in the communications between general public The such as Ms of weight first please or preferential etc the idea introducing system of old man in.Note, by priority treatment User can be the member such as paying Membership Fee.
(output image generation unit)
Output image generation unit 270 is by the input picture that image acquisition unit 110 inputs The overlapping display information obtained by information acquisition unit 150 is to produce output image.More specifically, it is defeated Go out image generation unit 270 and first determine the display letter obtained by information acquisition unit 150 for display The attribute of the display object of breath.Display object attribute include such as the example of Fig. 5 in shape, face The data that color, size, display position, transparency and layer etc. are relevant.For determining the shape of display object The mode of shape, color, size, transparency or layer is similar with the mode of first embodiment.
In this embodiment, output image generation unit 270 determines based on by weight determining unit 260 The weight of each user determine the display position (example of the display information being associated with each user As, the display position of each display object).More specifically, such as, image generation unit 270 is exported Determine the display position of each display object so that the display being associated with the user with greater weight Object is by from default display position displacement lesser extent.Then, output image generation unit 270 By over an input image in the display position determined overlapping each display object produce output figure Picture.
In this embodiment, the weight of user and the concept of the displacement away from default display position are depended on " gravitation " is used for determining display position.Additionally, in display position determines the first example of process, also Introduce the default display position of the display information depending on given user and the display letter of another user The concept " repulsion " of the distance between the default display position of breath.
(1) display position determines the first example of process
Figure 14 A to Figure 14 C is that the display position each illustrating output image generation unit 270 determines The explanatory diagram of the first example processed.In display position determines the first example of process, export image Generation unit 270 determines the display position of the display object for showing each display information so that front The repulsion for each display information and the gravitation stated are in poised state.
With reference to Figure 14 A, it is shown that the display object DO1 of user Ux and the display of another user are right As DO2.As example, the default display position DP1 of display object DO1 and display object DO2 Default display position DP2 between distance d1,2Equal to 120 (units: pixel).Additionally, use Weight W1 of family Ux is equal to 20.
Here, the repulsion R1 acted on display object DO1 is defined as such as distance d1,2With in advance The product of permanent several C (hereinafter referred to as repulsion constant, such as, C=50).Then, repulsion R1=d1,2× C=6000.Meanwhile, the gravitation G1 acted on display object DO1 is defined as such as Weight W1 of user Ux is taken advantage of with the display object DO1 displacement d1's away from default display position DP1 Long-pending.Then, in the poised state (that is, G1=R1) between gravitation G1 and repulsion R1, as The lower displacement d1:d1=G1/W1=R1/W1=6000/20=300 deriving display object DO1.Therefore, Such as, output image generation unit 270 determines on the straight line of link position DP1 and position DP2 (on the direction contrary with position DP2) position NP1 that distance is 300 away from position DP1 makees For showing the new display position of object DO1.Display object DO2 can be determined in a similar fashion New display position.
The example with reference to Figure 14 A describes two overlapped situations of display object.Relatively According to ground, when three or more display objects are overlapped, by considering repulsion during calculating Make a concerted effort, output image generation unit 270 may determine that the display position of each display object.
Referring now to Figure 14 B, it is shown that the display object DO1 of user Ux and another user's Display object DO2 and DO3.Here, by Vd1,2Indicate and originate in lacking of display object DO2 Save at display position DP2 and end at the arrow shown at the default display position DP1 of object DO1 Amount.Additionally, by Vd1,3Indicate and originate at the default display position DP3 of display object DO3 And end at the vector shown at the default display position DP1 of object DO1.Then, act on aobvious Show that the repulsion R1 on object DO1 is represented as vector C × Vd1,2With C × Vd1,3Sum.In balance In state, gravitation G1+ repulsion R1=0.Therefore, output image generation unit 270 can be by inciting somebody to action Repulsion R1 calculates display object DO1 away from default display position DP1 divided by the weight of user Ux Displacement d1.Then, default display position DP1 is added top offset d1 and may determine that display object The new display position NP1 of DO1.
Have been based on single (before synthesis) repulsion along the default display position connecting two display objects Supposition that the straight line put works and the example that describes Figure 14 A and 14B.Alternatively, permissible It is defined as repulsion only such as rising on the predetermined limited direction of vertical direction etc (in image) Effect.
Referring now to Figure 14 C, it is shown that the display object DO1 of user Ux is aobvious with another user's Show object DO2.Here, originate at the default display position DP2 of display object DO2 and terminate Vector Vd at the default display position DP1 of display object DO11,2Point to upper right side.Note, In the example of Figure 14 C, repulsion is defined as working the most in vertical direction.Therefore, repulsion R1 works along from the upwardly extending straight line of position DP1.In this case, display object DO1 Displacement d1 away from default display position DP1 also becomes and is not directed to upper right side and is directed to straight top Vector.As a result, display object DO1 moves up from default display position.Similarly, show right As DO2 moves down from default display position.In this case, each display object and user it Between relative position relation do not change.Therefore, the user of viewing output image can Will be readily understood that the corresponding relation between display object and user.
Although the product of the distance here repulsion being defined as between repulsion constant and display object, but scold Power is not limited to this, and can use another definition of repulsion.Such as, repulsion self can be constant, Or repulsion can square proportional to the distance between display object.Furthermore, it is possible to restriction repulsion Depend on the type of display information and different.
Alternatively, if as replacing the distance used between the default display position showing object Generation and define that the distance between the point (the most immediate point) on the periphery showing object is relative The repulsion answered, then can also eliminate display object may mutually partially overlapping probability.As entering one The replacement of step, if the peripheral at each display object provides relief area and defines and relief area The repulsion that distance between edge is corresponding, then can also create gap between display object.Additionally, The repulsion away from the periphery (window frame) exporting image can also be limited, and by scolding away from this window frame Power calculates the formula representing the poised state acted between gravitation and the repulsion on this display object In.
Above-mentioned example is described, wherein uses the definition of so-called spring system to calculate aobvious Showing the displacement of object, this spring system uses the weight of user and the product conduct of the displacement of display object Gravitation.But, alternatively, it is also possible to use the definition of mass-spring-damper device system.At this In the case of, the acceleration system of display object can be given according to the type of the attribute of user or display information Number (quality) and velocity coeffficient (damping constant).Correspondingly, visual effect can be produced so that The display object of knock against each other swings (and this swing will taper off) along time shaft.
(2) display position determines the second example of process
Figure 15 is to illustrate the second example that the display position of output image generation unit 270 determines process Explanatory diagram.In display position determines the second example of process, export image generation unit 270 Determine the display position of multiple display object so that the gravitation acted on multiple display object is in flat Weighing apparatus state.In this case, the concept of aforesaid repulsion is not used.
With reference to Figure 15, it is shown that the display object of the display object DO5 and user Uy of user Ux DO4.As example, the default display position DP4 of display object DO4 and new display position Distance d4 between NP4 is equal to 40.Meanwhile, the default display position DP5 of object DO5 is shown With distance d5 between new display position NP5 is equal to 80.Additionally, weight W5 of user Ux Equal to 50, and weight W4 of user Uy is equal to 100.Here, when such as showing that position determines place Image distance is lacked by the weight that the size of gravitation is defined as in the first example of reason user like that with display When saving the product between the displacement of display position, act on the gravitation G4 etc. on display object DO4 In W4 × d4=4000.Meanwhile, the gravitation G5 acted on display object DO5 is equal to W5 × d5=4000.It is to say, gravitation G4 and gravitation G5 is in balance on new display position State.Such as, output image generation unit 270 may search for reaching for multiple display objects The display position of the poised state of this gravitation, and using true for the display position that obtains as Search Results It is set to the display position of the plurality of display object.In this case, it is also possible to as display position determines The first example processed make like that the display object that is associated with the user with greater weight from Default display position displacement lesser extent.
(3) restriction in terms of display
Note, when the new display determining the display object processed and determine via above-mentioned display position When position is unsatisfactory for the restriction in terms of display, output image generation unit 270 can change aobvious further Show the display position of object or the display position of display object is set to default display position.Display The example of the restriction of aspect includes following example: display object in screen, and should show object not Should be mutually overlapping with the face of user (the most all users or weight are more than or equal to the user of predetermined threshold) Folded.Such as, when showing that object overlaps on default display position, increasing multiple display object can The probability that energy is overlapped.But, when multiple display objects are overlapped, output image produces Unit 270 controls the size of display information, depth direction according to the method described in first embodiment On layout, transparency etc..Therefore, it can reduce the reduction of the definition of display information.
[example of 3-2. output image]
Figure 16 is to illustrate the output image exported from the image processing apparatus 200 according to this embodiment The explanatory diagram of example.With reference to Figure 16, it is shown that image Im21 is as example in output.At output image Im21 exists three users Ua, Ub and Uc.In the example of Figure 16, user Ua and Ub There is the relatively long time in the input image, and be in communication with each other.Meanwhile, user Uc is temporary transient The user passed by.
Display object 12a and 13a overlaps on the region around user Ua.Display object 12b and 13b overlaps on the region around user Ub.The display object 12c weight being associated with user Uc It is stacked in above user Uc and on display position the most remote for user Uc.These show object not phase The most overlapping.Here, in this embodiment, the position of overlapped display object is from its default display Position is removed so that the display object being associated with the user with greater weight as described above is from it Default display position displacement lesser extent.Therefore, time of staying length and the big user Ua of weight and Display object 13a and 13b of Ub moves seldom from its default display position.In contradistinction to, stop Stay the time short and the display object 12c of user Uc that weight is less moves up to avoid showing object 13a and 13b.Correspondingly, the communication between user Ua and Ub can smoothly continue without by Temporarily the display object of the user Uc of process hinders.Additionally, user can easily understand which is used Family persistently uses system with which kind of degree.
[3-3. handling process]
Figure 17 is the primary picture handling process illustrating the image processing apparatus 200 according to this embodiment The flow chart of example.
With reference to Figure 17, first image acquisition unit 110 obtains the input captured by imaging device 104 Image (step S202).Then, image acquisition unit 110 is to recognition unit 130 and output image Generation unit 270 exports acquired input picture.
It follows that recognition unit 130 detects facial zone and the hand of user present in input picture Region (step S204).It follows that recognition unit 130 by be stored in data base 102 The image feature amount of known users in contradistinction to checks the image feature amount of the facial zone of each detection Identify unique user (step S206) present in input picture.
It follows that information acquisition unit 150 is from data base 102, termination 105, external information The display information to be shown (step that each user that the acquisitions such as source identify with recognition unit 130 is associated Rapid S208).
It follows that output image generation unit 270 measures being identified by recognition unit 130 in image The time of staying (and as required, also measuring translational speed) (step S210) of each user. Then, weight determining unit 260 is based on each user measured by output image generation unit 270 The attribute information of the time of staying and each user determines the weight (step S211) of each user.
It follows that output image generation unit 270 was provided for based on the time of staying or translational speed Transparency (the step of the display object of the display information that display is obtained by information acquisition unit 150 S212).Additionally, output image generation unit 270 is by using the chi of the facial zone of each user Very little as the parameter corresponding with user's distance away from imaging device 104, each display object is set Size and layer (step S214).
It follows that the display position that output image generation unit 270 performs to be described below in detail determines place Reason is with the display position (step S215) determining each display object.Additionally, output image produces single Unit 270 arranges other attribute (step S216) of the color etc. of the most each display object.
It follows that output image generation unit 270 is by existing according to the attribute of set display object On input picture, this display object overlapping produces output image (step S218).Then, output figure As generation unit 270 to display device 106 export produced by export image so that display device 106 Show this output image (step S220).
Figure 18 is to illustrate the example that the display position corresponding with step S215 of Figure 17 determines process The flow chart of property flow process.Flow chart in Figure 18 is corresponding to the display described with reference to Figure 14 A to 14C Position determines the first example of process.
With reference to Figure 18, first each display object placement is lacked by output image generation unit 270 at it Save in display position (step S252).It follows that output image generation unit 270 is according to default aobvious Show that the distance between position calculates repulsion (or the conjunction of multiple repulsion acted on each display object Power) (step S254).It follows that output image generation unit 270 based on the repulsion calculated and The weight of user calculates each display displacement (step S256) to its default display position of image distance.
It follows that output image generation unit 270 determine by by default display position plus being calculated Displacement and whether the new display position of each display object of deriving meets the restriction in terms of display (step S258).Here, if it is determined that new display position meets the restriction in terms of display, the most defeated Go out image generation unit 270 and will show that object placement is in new display position (step S260).With Time, if it is determined that new display position is unsatisfactory for the restriction in terms of display, then export image generation unit 270 will show that object placement is in the another location including default display position (step S262).
Figure 19 is to illustrate that position determines another of process in show corresponding with step S215 of Figure 17 The flow chart of exemplary flow.Flow chart in Figure 19 is corresponding to the display position described with reference to Figure 15 Determine the second example of process.
With reference to Figure 19, first each display object placement is lacked by output image generation unit 270 at it Save in display position (step S272).It follows that output image generation unit 270 determines whether to deposit At any overlapped display object (step S274).Here, if there is no overlapping display Object, then the display position of output image generation unit 270 determines that process terminates.Meanwhile, if deposited At overlapping display object, then process and proceed to step S276.
In step S276, output image generation unit 270 will be determined in step S274 New display position (step S276) is moved to for overlapped display object.It follows that output Image generation unit 270 is based on calculating each away from the default displacement of display position and the weight of user The gravitation (step S278) of display object.Then, output image generation unit 270 determines and is calculated Display object gravitation the most reached poised state (step S280).Here, if it is determined that aobvious Show that the gravitation of object has reached poised state, then the display position of output image generation unit 270 determines Process and terminate.Simultaneously, if it is determined that the gravitation of display object is not up to poised state, then process and carry out To step S282.
In step S282, output image generation unit 270 determines whether to have searched for all full The display position (step S282) of the restriction in terms of foot display.Here, if the most any display position Put the most searched, then process and return to step S276.Meanwhile, if having searched for all full The display position of the restriction in terms of foot display, then output image generation unit 270 will show object again It is arranged in its default display position, and does not consider such as to show the overlap (step S284) of object, so This process of rear termination.
[conclusion of 3-4. the second embodiment]
Below the second embodiment of the present invention is described with reference to Figure 12 to Figure 19.According to this enforcement Example, such as, determine the weight of user according to time of staying of user or attribute information, wherein user Weight is similar to user's priority in terms of display.Determine based on this weight and be associated with each user The display position of display information.Correspondingly, multiple display information can be avoided overlapped, so that The display information of the user must with high priority is not hindered by the display information of other users or interference. As a result, in the case of information is mingled in the image of augmented reality, can be with the side being easier to understand Formula shows information.
Additionally, according to this embodiment, determine the display position of each display information so that with have relatively The display information that the user of big weight is associated is by from its default display position displacement lesser extent.Phase Ying Di, according to this embodiment, the display information that can reduce the user temporarily occurred in the picture may Obstruction the most persistently uses the risk of the communication between the user of this system.Further, it is also possible to excitation user For longer periods use this system.Meanwhile, the display position determined if based on weight is unsatisfactory for display The restriction of aspect, then control the size of display information, depth direction as in the first embodiment On layout, transparency etc. so that multiple display information can be allowed overlapped.Correspondingly, i.e. Make when information is mingled in the image of augmented reality, the notable fall of the definition of information also will not occur Low.
Note, typically use software to realize according to the first and second enforcements described in this specification The series of processes of example.The program constituting the software for realizing this series of processes is stored in the most in advance Within being arranged on each device or outside storage medium in.Additionally, each program when executed by Read in RAM (random access memory), and by such as CPU's (CPU) etc Processor performs.
Although describing preferred embodiment of the present disclosure in detail by reference to accompanying drawing, but the disclosure not limiting In this.For a person skilled in the art, various modification or change are clearly possible, as long as This modification or change are within the technical scope of appended claim or its equivalents.Should Understanding, this modification or change are also within scope of the presently disclosed technology.
The present invention comprises the Japanese earlier patent application submitted for 6th with JIUYUE in 2010 in Japan Office The theme that theme disclosed in JP 2010-198986 is relevant, entire contents is incorporated herein by reference.

Claims (15)

1. an image processing apparatus, including:
Recognition unit, is configured to multiple use present in the input picture that detection is captured by imaging device The facial zone at family, identifies corresponding to the ID of each facial zone, and by each user with Each identified ID is associated；
Information acquisition unit, is arranged to acquisition relevant to each user of described recognition unit identification The display information to be shown of connection, described display information is the attribute information of the user of corresponding facial zone Or the information by user's input of corresponding facial zone；
Weight determining unit, is arranged to determine the power of each user of described recognition unit identification Weight；And
Output image generation unit, is configured in the following manner to produce output image: based on The described weight of each user that described weight determining unit determines determines and is associated with each user The display position of described display information, and on described input picture determined by show in position Overlapping described display information,
Wherein, described output image generation unit determines the described display position of described display information, makes The display information must being associated with the user with the biggest weight displacement away from default display position is the least,
Wherein, described output image generation unit determines that the first of first user shows the display position of information Putting so that repulsion and gravitation are in poised state, described repulsion depends on described in described image Distance between described first display information and the display information of another user of one user, and described Gravitation depends on that the weight of described first user and described first display information are away from default display position Displacement.
Image processing apparatus the most according to claim 1, wherein, described output image produces What unit searches reached the poised state of gravitation for multiple display information shows position accordingly, And the display position obtained as Search Results is defined as the corresponding of the plurality of display information Display position.
Image processing apparatus the most according to claim 1, wherein, described weight determining unit Determine described in each user according to each user time span present in the described input picture Weight.
Image processing apparatus the most according to claim 1, wherein,
Described information acquisition unit obtains the attribute information of each user of described recognition unit identification；With And
The described attribute information that described weight determining unit uses described information acquisition unit to obtain comes really The described weight of fixed each user.
Image processing apparatus the most according to claim 4, wherein, described weight determining unit Determine the described weight of each user so that belong to the weight of the user of particular sex or year age group Go above the weight of other users.
Image processing apparatus the most according to claim 1, wherein, when based on each user When the described display position of the described display information that described weight determines meets the restriction in terms of display, institute State output image generation unit by described display information placement in described display position.
Image processing apparatus the most according to claim 1, wherein, when based on each user When the described display position of the described display information that described weight determines is unsatisfactory for the restriction in terms of display, Described output image generation unit by described display information placement with determined by described display position In another display position including default display position differed.
Image processing apparatus the most according to claim 1, wherein, described output image produces Unit determines phase mutual respect based on the parameter corresponding with each user distance away from described imaging device Before folded which the display information in display information to be superimposed over.
Image processing apparatus the most according to claim 8, wherein, when at described output image In first display information with second display information overlapped time, if with described first display information phase The user of the association distance away from described imaging device is shorter than the use being associated with described second display information The family distance away from described imaging device, the most described output image generation unit is by described first display information It is placed in before described second display information.
Image processing apparatus the most according to claim 8, wherein,
Described recognition unit also identifies the facial zone of each user present in described input picture Size；And
Described output image generation unit uses described of each user of described recognition unit identification The described size in region, portion is as described parameter.
11. image processing apparatus according to claim 8, wherein,
Described output image generation unit measures each user of described recognition unit identification described defeated Enter time span or the translational speed of each user present in image；And
Described output image generation unit is according to for the described user being associated with described display information The described time span measured or described translational speed are arranged on described input picture the institute of overlap State the transparency of display information.
12. image processing apparatus according to claim 11, wherein, described output image produces Raw unit will exist described in the described display information of the most long user in described input picture Transparency is set to the lowest level.
13. image processing apparatus according to claim 11, wherein, described output image produces The described transparency of the described display information of user the lowest for translational speed is set to the lowest by raw unit Level.
14. image processing apparatus according to claim 1, wherein, described output image produces Unit determines, according to each user distance away from described imaging device, the institute being associated with each user State the display size of display information.
15. 1 kinds of image processing methods, including:
The facial zone of multiple users present in the input picture that detection is captured by imaging device, identifies Corresponding to the ID of each facial zone and user is relevant to identified ID Connection；
Obtaining the display information to be shown being associated with the user of each identification, described display information is The attribute information of the user of corresponding facial zone or the information inputted by the user of corresponding facial zone；
Determine the weight of the user of each identification；
The described display being associated with described user is determined based on weight determined by each user The display position of information；And
By on described input picture determined by show in position that overlapping described display information is come Produce output image,
Wherein, the described display position of described display information is determined so that and has the biggest weight The display information that user the is associated displacement away from default display position is the least,
Wherein, the display position of the first display information of first user is determined so that repulsion and gravitation Being in poised state, described repulsion depends on that described in described image described the first of first user shows Show the distance between the display information of information and another user, and described gravitation depends on described first The weight of user and the described first display information displacement away from default display position.
CN201110259511.1A 2010-09-06 2011-08-30 Image processing apparatus and image processing method Expired - Fee Related CN102446210B (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2010198986A JP2012058838A (en) 2010-09-06 2010-09-06 Image processor, program, and image processing method
JP2010-198986 2010-09-06

Publications (2)

Publication Number Publication Date
CN102446210A CN102446210A (en) 2012-05-09
CN102446210B true CN102446210B (en) 2016-12-14

Family

ID=

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101207775A (en) * 2006-12-22 2008-06-25 富士胶片株式会社 Information processing apparatus and information processing method
CN101287056A (en) * 2007-04-12 2008-10-15 奥林巴斯映像株式会社 Image reconstruction device, image reconstruction program, recording medium and image reconstruction method

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101207775A (en) * 2006-12-22 2008-06-25 富士胶片株式会社 Information processing apparatus and information processing method
CN101287056A (en) * 2007-04-12 2008-10-15 奥林巴斯映像株式会社 Image reconstruction device, image reconstruction program, recording medium and image reconstruction method

Similar Documents

Publication Publication Date Title
CN102385697B (en) Image processing apparatus, program and image processing method
EP2426645B1 (en) Presentation of information in an image of augmented reality
CN109618222B (en) A kind of splicing video generation method, device, terminal device and storage medium
CN104104768B (en) The device and method of additional information are provided by using calling party telephone number
WO2013145566A1 (en) Information processing apparatus, information processing method, and program
CN103971391A (en) Animation method and device
EP3547218A1 (en) File processing device and method, and graphical user interface
CN107209631A (en) User terminal and its method for displaying image for display image
CN106341574B (en) Method of color gamut mapping of color and device
CN108628985B (en) Photo album processing method and mobile terminal
US20170371524A1 (en) Information processing apparatus, picture processing method, and program
WO2023142916A1 (en) Image processing method, apparatus and device, and storage medium
CN111796826B (en) Bullet screen drawing method, device, equipment and storage medium
CN112163577A (en) Character recognition method and device in game picture, electronic equipment and storage medium
CN113342435A (en) Expression processing method and device, computer equipment and storage medium
WO2021169991A1 (en) Method and apparatus for configuring theme color of terminal device, and terminal device
CN107609487A (en) A kind of generation method and device of user's head portrait
CN111727438A (en) Object classification for image recognition processing
CN102446210B (en) Image processing apparatus and image processing method
JP7373646B2 (en) Display control system, display control method, and program
WO2014107059A1 (en) User terminal device for generating playable object, and interaction method therefor
CN111445439B (en) Image analysis method, device, electronic equipment and medium
CN106547891A (en) For the quick visualization method of the pictured text message of palm display device
CN110796004A (en) Image identification evaluation method and device, storage medium and electronic equipment
WO2022045832A1 (en) Mixed reality-based content service apparatus and driving method for same apparatus, and computer-readable recording medium

Legal Events

Date Code Title Description
PB01 Publication
SE01 Entry into force of request for substantive examination
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20161214