JP7299687B2 - Image processing device, imaging device, image processing method - Google Patents

Description

The present invention relates to image coding technology.

A dynamic ROI (Region Of Interest) is known as a technique for reducing the bit rate when transferring video while maintaining the image quality of an object that the user is paying attention to. In this technique, a region is set for the detected object. Then, a large amount of code is set for an area where an object that the user pays attention to (area of interest) to improve image quality, and a small amount of code is set for an area where an object that the user does not pay attention to (non-attention area) exists. The bit rate is reduced by lowering the image quality and reducing the amount of data.

However, depending on the ROI settings, the visibility of the object of interest may decrease. In Japanese Patent Laid-Open No. 2004-100003, in order to solve the problem that the visibility deteriorates due to the difference in image quality between the attention area and the background area, the visibility is improved by setting different code amounts for the boundary area between the attention area and the background area. Methods are disclosed to prevent degradation. In addition, Patent Literature 2 discloses the following technique to address the problem of ghosts occurring when the ROI moves when the encoding frequency of the non-ROI is lower than that of the ROI. In other words, Patent Document 2 discloses a method of preventing decrease in visibility by encoding a region of the ROI of the previous frame that does not overlap with the ROI of the current frame with the same code amount as that of the non-ROI to prevent ghosting. there is

JP-A-2009-27457 Japanese Patent Application Laid-Open No. 2013-187769

The methods described in Patent Literature 1 and Patent Literature 2 do not take into account the decrease in visibility when ROIs set with different code amounts overlap each other. In this method, a region is set for the detected object based on the feature value of the object extracted by the dynamic ROI. cannot be set. As a result, the background object is set to the same region and the same code amount as the front object, thereby reducing the visibility of the background object. For example, when a human area with a high code amount passes behind a tree area with a low code amount, even though the person can be seen through the gaps between the branches, trunk, and leaves of the tree, the human The ROI setting of will be the same as the ROI setting of the tree, which will reduce the visibility. The present invention provides an image coding technique that considers the visibility of objects overlapping each other in an image.

According to one aspect of the present invention, the set code amount preset for the attribute of the back object among the objects overlapping each other in the image is set to the attribute of the front object among the overlapping objects. a setting means for setting the set code amount preset for the back object as the code amount for the front object when the set code amount is larger than a preset code amount;
encoding means for encoding the region of the object in front with the code amount set by the setting means for the object.

According to the configuration of the present invention, it is possible to provide an image coding technique that considers the visibility of objects overlapping each other in an image.

FIG. 2 is a block diagram showing a functional configuration example of a dynamic ROI system; 10 is a flowchart of processing performed by the dynamic ROI device A1002; The figure which shows the image D1. The figure which shows the image D101. The figure which shows the image D201. The figure which shows the image D301. The figure which shows the image D401. The figure which shows the object area|region in the image D401. FIG. 2 is a block diagram showing a hardware configuration example of a computer device; 10 is a flowchart of processing performed by the dynamic ROI device A1002; The figure which shows the image D601. The figure which shows the image D701.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. It should be noted that the embodiment described below is an example of a specific implementation of the present invention, and is one of the specific embodiments of the configuration described in the claims.

[First embodiment]
First, a functional configuration example of a dynamic ROI system according to this embodiment will be described using the block diagram of FIG. As shown in FIG. 1, the dynamic ROI system A1000 according to this embodiment includes an image acquisition device A1001 that acquires an image, a dynamic ROI device A1002 that encodes and outputs an image of each frame of the image, and an encoded It has a display device A1003 that decodes and displays an image of each frame.

The image acquisition device A1001 and the dynamic ROI device A1002 are connected via a wireless and/or wired network. Also, the dynamic ROI device A1002 and the display device A1003 are connected via a wireless and/or wired network.

First, the image acquisition device A1001 will be described. The video acquisition device A1001 is a device that acquires a video (moving image) and supplies an image of each frame that constitutes the video to the dynamic ROI device A1002. The acquisition form and supply form of the image by the image acquisition apparatus A1001 are not limited to a specific acquisition form and supply form.

For example, the image acquisition device A1001 may acquire an image held by itself or an external device, and sequentially supply the image of each frame in the acquired image to the dynamic ROI device A1002. Also, the image acquisition device A1001 may capture an image, acquire an image of each frame in the imaged image, and sequentially supply the image to the dynamic ROI device A1002.

Next, the dynamic ROI device A1002 (image processing device) will be described. The dynamic ROI apparatus A1002 encodes the image of each frame supplied from the image acquisition apparatus A1001, and at that time, encodes each object area in the image with an appropriate code amount. Then, the dynamic ROI device A1002 outputs the encoded image of each frame to the display device A1003.

The video acquisition unit A1004 acquires an image of each frame supplied from the video acquisition device A1001.

The area dividing unit A1005 performs area division on the image acquired by the video acquisition unit A1004, detects an object area (object area) in the image, and determines the attributes of the object (object type, etc.) in the area. ). Various methods are known for region segmentation, such as region split, super-parsing, and fully CNN (Convolutional Neural Network) based on Deep Learning. In the present embodiment, it is assumed that fully CNN is used because it can divide regions with high precision, but any method may be used. Since area splitting, super-parsing, fully CNN, and the like are well-known techniques, descriptions of these area division techniques will be omitted. Then, for each object region detected from the image, the region dividing unit A1005 divides the information for specifying the object region in the image (for example, the coordinate positions of the upper left corner and the lower right corner of the object region) and the object region in the object region. Generate object information including the attributes of

The comparison unit A1006 uses the object information generated for the current frame image by the area division unit A1005 and the “past frame object information” recorded in the recording unit A1007 to overlap on the current frame image. determine the fore-and-aft and obscuring relationships between objects The comparison unit A1006 outputs the object information generated for the image of the current frame by the area dividing unit A1005. Output after adding the information to represent.

The operation of the comparison unit A1006 will be described with a specific example. An image D1 shown in FIG. 3 is an image of the (f-2)th frame (f is an integer of 3 or more), and an object region P2 of a person and an object region P1 of a building are detected from the image D1. . For the human object region P2, the attribute "person" of the object in the object region P2 and information for specifying the object region P2 in the image D1 (for example, the coordinate positions of the upper left corner and lower right corner of the object region P2) and object information is generated. Similarly, for the object region P1 of a building, the attribute "building" of the object in the object region P1 and information for specifying the object region P1 in the image D1 (for example, the upper left and lower right corners of the object region P1 coordinate position) and object information is generated. As shown in FIG. 3, in the image D1, there is no situation where one of the person and the building overlaps the other.

The image D101 shown in FIG. 4 is the (f−1)-th frame image (the image one frame after the image D1 in FIG. 3). is detected. For the human object region P102, the attribute "person" of the object in the object region P102 and information for specifying the object region P102 in the image D101 (for example, the coordinate positions of the upper left corner and lower right corner of the object region P102) and object information is generated. Similarly, for the object region P101 of a building, the attribute “building” of the object in the object region P101 and information for specifying the object region P101 in the image D101 (for example, the upper left and lower right corners of the object region P101 coordinate position) and object information is generated. As shown in FIG. 4, in the image D101, the person moves behind the building from the image D1, and part of the person overlaps the building.

When the object information of the person and the building in the image D101 is acquired from the area dividing unit A1005, the comparison unit A1006 compares the object information of the person and the building in the image D101 with the "person and building in the image D1" recorded in the recording unit A1007. By using the "object information of", the front-back relationship and concealment relationship between people and buildings are determined. As shown in FIGS. 3 and 4, the human object region P102 in the image D101 is adjacent to the building object region P101 and is smaller in size than the human object region P2 in the previous frame. This means that a person has moved to the back of a building so that part of the person overlaps with the building (the person is the back object and the building is the front object). Therefore, when the human object region P102 is adjacent to the building object region P101 and is smaller in size than the human object region P2 in the previous frame, the comparison unit A1006 determines that "the person overlaps the building ( A person is an object in the back, a building is an object in the front)”. Then, the comparison unit A1006 adds information (rear information) indicating that the person is "located behind a building" to the object information of the person, and adds information indicating "located in front of the person" to the object information of the building. Adds the information (front information) to represent.

An image D201 shown in FIG. 5 is an f-th frame image (an image one frame after the image D101 in FIG. 4). In image D201, since the person is hidden by the building (the person hidden by the building is indicated by a dotted line), the object area P201 of the building is detected, but the object area of the person is not detected. Similarly, when a person is not completely hidden by a building but is not detected as a person from the image (or when the detection accuracy is less than the threshold), the object region of the person is not detected. Therefore, the object information of the person is not generated, and for the object region P201 of the building, the attribute "building" of the object in the object region P201 and information for specifying the object region P201 in the image D201 (for example, object region P201) are generated. The coordinate positions of the upper left corner and the lower right corner of P201) are generated.

If the object area of the person, which was determined to overlap with the building (located behind the building) in the image D101, cannot be detected from the image D201, the comparison unit A1006 determines that the object area of the person is not detected in the image D201. Concealed (a person is an object in the back, a building is an object in the front)". When the comparison unit A 1006 determines that "the person is hidden by the building (the person is the object behind the building, and the building is the object in front)", the object information of the building is "located in front of the person ( A person is positioned behind)” is added.

As described above, the comparison unit A1006 determines that the size of the object region of one object is reduced over time from the state in which the object region of one object is adjacent to the object region of the other object. Add back information indicating that the object is located behind the other object to the object information of Append. After that, when the object region of one of the objects whose size decreases with the lapse of time was detected up to the immediately preceding frame but is no longer detected from the image of the current frame, the comparison unit A1006 It is determined that the other object is hidden by the other object, and front information representing "located in front of the one object" is added to the object information of the other object.

It should be noted that the method for determining the anteroposterior relation and the concealment relation between objects overlapping on the image of the current frame is not limited to the above method, and various methods can be applied. Further, the information representing the overlapping objects and the objects in the hidden relationship and the management mode thereof are not limited to the above examples.

The recording unit A1007 records the object information of the object in the image of each past frame.

The determination unit A1008 acquires the object information of each object in the current frame from the comparison unit A1006. Then, the determination unit A1008 determines whether or not there is object information to which front surface information is added among the object information acquired from the comparison unit A1006 (whether or not there is an object overlapping in the image of the current frame). judgment). Object information to which front surface information is added is hereinafter referred to as front surface object information.

The determination unit A1009 determines whether or not there is an object whose elapsed time after overlapping exceeds a specified time, among the overlapping objects in the image of the current frame. For example, the determination unit A1009 determines an object (an object serving as the front surface) corresponding to the front object information as the target object. Then, the determination unit A1009 determines whether or not the elapsed time since the front surface information was first added to the object information of the target object is equal to or less than a specified time. In the examples of FIGS. 3 to 5, it is determined whether or not the elapsed time since the person overlapped with the building is less than or equal to a specified time. As a result of this determination, if the elapsed time exceeds the specified time, the determining unit A1009 deletes the front surface information from the front surface object information. The determination unit A1009 measures the elapsed time.

The comparing unit A1010 uses object information of each object in the current frame and a code amount table A1012 in which the set code amount is registered for each attribute of the object, and compares each overlapping object (an object in front and an object in front). object on the back) are compared. The code amount table A1012 registers, for each attribute of an object, a preset code amount that is set in advance as the code amount of an area of an object having that attribute.

The comparison unit A1010 acquires the set code amount corresponding to the "attribute of the object" included in the foreground object information from the code amount table A1012 as the "set code amount for the foreground object". Further, when the foreground information added to the foreground object information indicates that "it is located in front of the object B", the comparison unit A1010 sets the set code amount corresponding to the attribute included in the object information of the object B. is acquired from the code amount table A1012 as the "set code amount for the back object". Then, the comparison unit A1010 compares the "set code amount for the front object" and the "set code amount for the back object". As a result of this magnitude comparison, if “set code amount for front object”>“set code amount for back object”, the comparison unit A1010 deletes the front information added to the front object information. On the other hand, as a result of this size comparison, if "set code amount for front object"<"set code amount for back object", comparison unit A1010 sends front object information to code amount change unit A1011 in the subsequent stage. do.

The operation of the comparison unit A1010 will be described with a specific example. An image D301 shown in FIG. 6 includes a tree object region P301, a bus object region P302, and human object regions P303 and P304. In the code amount table A1012, in the case of FIG. 6, set code amounts for each of the object attributes "tree", "bus", and "person" are registered. Here, the magnitude relationship among the set code amount for tree, the set code amount for person, and the set code amount for bus registered in the code amount table A1012 is: set code amount for tree<set code amount for person<bus setting. It shall be the code amount.

An object region set with a larger code amount is encoded with higher image quality, and an object region with a smaller code amount set is encoded with lower image quality. “Encoding with higher image quality” means “encoding such that the image quality after decoding is higher”, and for example, the quantization step used for encoding is made smaller. On the other hand, "encoding with lower image quality" means "encoding so that the image quality after decoding becomes lower image quality", and for example, the quantization step used for encoding is increased.

In this embodiment, for each object region in an image, a set code amount corresponding to the attribute of the object in the object region is set. It changes according to the attributes of the object being held. In the image D301 of FIG. 6, since none of the tree, bus, and person overlap each other, the set code amount corresponding to the attribute of the object is set in the object region of each object.

An image D401 shown in FIG. 7 is an image several frames after the image D301. Due to the movement of the person from the image D301, the person P403 is hidden behind the tree P401 and the person P404 is hidden behind the bus P402. ing. It is assumed that although the tree P401 is detected from the image D401, the person P403 is not detected, and as a result, the object region P501 in FIG. 8 is detected as the object region of the tree P401. It is also assumed that the bus P402 is detected from the image D401 but the person P404 is not detected, and as a result, the object region P502 in FIG. 8 is detected as the object region of the bus P402. The respective code amounts of such object regions P501 and P502 are set as follows.

The comparison unit A1010 identifies the object (person P404) positioned behind the bus P402 from the front information added to the object information of the bus P402. The comparison unit A1010 compares the set code amount registered in the code amount table A1012 for the bus P402 and the set code amount registered in the code amount table A1012 for the person P404. As a result of this size comparison, it is found that the set code amount of the bus P402, which is the object in front>the set code amount of the person P404, which is the object in the back. Delete the attached front information.

The comparison unit A1010 also acquires the object (person P403) positioned behind the tree P401 from the front information added to the object information of the tree P401. Then, the comparison unit A1010 compares the set code amount registered in the code amount table A1012 for the tree P401 and the set code amount registered in the code amount table A1012 for the person P403. As a result of this size comparison, the set code amount of the tree P401 that is the object in front<the set code amount of the person P403 that is the object in the back. It is sent to the code amount changing unit A1011.

If the foreground information added to the foreground object information received from the comparison unit A1010 indicates that "it is located in front of the object B", the code amount change unit A1011 is included in the object information for the object B. The set code amount corresponding to the attribute is acquired from the code amount table A1012. Then, the code amount changing unit A1011 sets the acquired set code amount as the code amount of the object region corresponding to the front object information. That is, if the set code amount of the object on the back side among the overlapping objects is larger than the set code amount for the object on the front side, the code amount changing unit A1011 changes the set code amount for the object on the back side to the code amount for the object on the front side. Set as quantity.

In the example of FIG. 8, the set code amount corresponding to the person P403 positioned behind the tree P401 is set as the code amount of the object region P501 of the tree P401. Also, the code amount of the object region P502 of the bus P402 does not change. In the object region P501, if the code amount of the tree P401 is set, the visibility of the person P403 is lowered. In the object area P502, even if the code amount of the bus P402 is set, the visibility of the person P404 does not decrease, so the code amount setting is not changed.

The code amount setting unit A1013 acquires from the code amount table A1012 the set code amount corresponding to the attribute in the object information of the object area for which the code amount has not yet been set, and sets the acquired set code amount to the code of the object area. Set as quantity. Also, the code amount setting unit A1013 records the object information of the current frame obtained through the region division unit A1005, comparison unit A1006, determination unit A1008, determination unit A1009, and comparison unit A1010 in the recording unit A1007.

The encoding unit A1014 encodes the image of the current frame, and encodes the object region in the image with the code amount set for the object region in the above process.

The video output unit A1015 outputs the image (encoded image) encoded by the encoding unit A1014 to the display device A1003. Note that the output destination of the encoded image is not limited to the display device A1003, and the output destination may be a memory device inside or outside the dynamic ROI device A1002, or an external device capable of communicating with the dynamic ROI device A1002. It is good as

Next, the display device A1003 will be described. The display device A1003 decodes and displays the encoded image output from the video output unit A1015. The display device A1003 may display the decoded image on a liquid crystal screen or a touch panel screen, or may project it on a projection surface. Although the display device A1003 is separate from the dynamic ROI device A1002 in FIG. 1, the dynamic ROI device A1002 and the display device A1003 may be integrated into one device.

Next, processing performed by the dynamic ROI apparatus A 1002 to encode and output an image of one frame will be described with reference to FIG. 2 showing the flowchart of the processing. The dynamic ROI device A1002 performs processing according to the flowchart of FIG. 2 for each frame image acquired from the image acquisition device A1001. Further, the end condition of the processing according to the flowchart of FIG. 2 is not limited to a specific condition, and may be, for example, "the user turned off the power switch (not shown) of the dynamic ROI apparatus A1002".

In step S201, the video acquisition unit A1004 acquires the image supplied from the video acquisition device A1001. In step S202, the region dividing unit A1005 detects an object region of each object in the image acquired in step S201, and generates object information for each object region.

In step S203, the comparison unit A1006 determines whether or not there are objects overlapping each other on the image acquired in step S201. As a result of this determination, if there are objects overlapping each other on the image acquired in step S201, the process proceeds to step S204. Processing proceeds to step S210.

If the object region of one object is adjacent to the object region of the other object and the size of the object region is smaller than that of the one object region in the previous frame, the comparison unit A1006 determines that the size of the object region is smaller than that of the object region of the other object in the image acquired in step S201. determine that there are objects overlapping each other. At this time, in step S204, the comparison unit A1006 adds to the information of one object back surface information indicating that it is "located behind the other object", and to the information of the other object "is located in front of the one object". Add front information indicating that the

If the object area of the other object, which was determined to overlap with the other object until the previous frame, cannot be detected from the image acquired in step S201, the comparison unit A1006 Assume that there are objects on top of each other. At this time, in step S204, the comparison unit A1006 adds to the object information of one object front surface information indicating that the object is "located in front of the other object".

In step S205, the determination unit A1008 sets the object corresponding to the front object information as the target object, and the determination unit A1009 sets the elapsed time T after the front surface information is first added to the object information of the target object as the specified time θ Determine whether:

As a result of this determination, if the elapsed time T is equal to or less than the specified time θ, the process proceeds to step S207, and if the elapsed time T exceeds the specified time θ, the process proceeds to step S206. In step S206, the determination unit A1009 deletes the front surface information from the front object information.

In step S207, the comparison unit A1010 compares the "set code amount for the front object" and the "set code amount for the back object". As a result of this size comparison, if "set code amount for front object">"set code amount for rear object", the process proceeds to step S206. On the other hand, if the result of this size comparison is "set code amount for front object"<"set code amount for rear object", the process proceeds to step S208.

In step S208, the comparison unit A1010 sends the front object information to the subsequent code amount change unit A1011. If the foreground information added to the foreground object information indicates that "it is located in front of the object B", the code amount changing unit A1011 changes the set code amount corresponding to the attribute of the object B to the foreground object information. It is set as the code amount of the object area corresponding to . The above steps S204 to S208 are performed for each object determined to be overlapping each other.

In step S209, the code amount setting unit A1013 acquires the set code amount corresponding to the attribute in the object information of the object region for which the code amount has not yet been set, from the code amount table A1012, and sets the acquired set code amount to the Set as the code amount of the object area. The code amount setting unit A1013 also records the object information of the current frame at the time of step S208 in the recording unit A1007.

In step S210, the encoding unit A1014 encodes the image of the current frame, and the object region in the image is encoded with the code amount set for the object region in the above process. do. In step S211, the video output unit A1015 outputs the encoded image encoded by the encoding unit A1014 in step S210 to the display device A1003.

As described above, according to the present embodiment, when an object with a high code amount is positioned behind an object with a low code amount, the code amount of the front object is set to the back object. The visibility of the object on the back side can be improved by making the code amount the same as the code amount on the back side.

Note that steps S205 and S206 may be deleted as necessary, and in that case, even an object whose elapsed time after being overlapped exceeds the specified time is treated as a target for processing from step S207 onward.

Also, change the code amount to be set by taking into consideration not only the result of comparing the "set code amount for the front object" and the "set code amount for the back object" but also the size of the difference between them. can be For example, in the case of "set code amount for front object"<"set code amount for back object", the greater the difference between these set code amounts, the larger the code amount is set for the front object.

[Second embodiment]
In each of the following embodiments, including the present embodiment, differences from the first embodiment will be explained, and the same as the first embodiment unless otherwise specified. In the present embodiment, the comparison unit A1010 determines that the "attribute of the object" included in the front object information is the "attribute of the object in which part or all of the object positioned behind the object is visible" (visibility attribute ). Then, if the "attribute of the object" included in the foreground object information is the visibility attribute, the comparison unit A1010 sends the foreground object information to the subsequent code amount changing unit A1011. On the other hand, if the "attribute of object" included in the foreground object information is not the visibility attribute, the comparison unit A1010 deletes the foreground information added to the foreground object information.

The operation of the comparison unit A1010 according to this embodiment will be described with a specific example. An image D601 shown in FIG. 11 includes a building object region P601, a tree object region P602, and human object regions P603 and P604. It is assumed that the set code amount has already been set for each of the building object region P601, the tree object region P602, and the human object regions P603 and P604.

An image D701 shown in FIG. 12 is an image several frames after the image D601. Due to the movement of the person from the image D601, the person P703 is hidden behind the building P701 and the person P704 is hidden behind the tree P702. ing. Here, an object located behind the building P701 cannot be seen through the building P701, but an object hidden behind the tree P702 can be seen through gaps between leaves and branches of the tree. Therefore, in this case, instead of the code amount table A1012, a visibility attribute table in which whether or not the attribute is a visibility attribute is registered for each attribute of an object is used. 11 and 12, the attribute of the building P701 is not registered as the visibility attribute in the visibility attribute table, but the attribute of the tree P702 is registered as the visibility attribute.

Therefore, in the case of this embodiment, the comparison unit A1010 refers to the visual attribute table and determines whether or not the attribute of the object included in the front object information is registered as the visual attribute in the visual attribute table. As a result of this determination, if the attribute of the object included in the front object information is registered as the visibility attribute in the visibility attribute table, the comparison unit A1010 sends the front object information to the subsequent code amount change unit A1011. to be sent out. On the other hand, as a result of this determination, if the attribute of the object included in the front object information is not registered as the visibility attribute in the visibility attribute table, the comparison unit A1010 determines the front surface information added to the front object information. delete.

Next, processing performed by the dynamic ROI apparatus A 1002 according to this embodiment to encode and output an image of one frame will be described with reference to FIG. 10 showing a flowchart of the processing. The dynamic ROI apparatus A1002 performs processing according to the flowchart of FIG. 10 for each frame image acquired from the image acquisition unit A1004. Further, the condition for ending the processing according to the flowchart of FIG. 10 is not limited to a specific condition, and may be, for example, "the user turned off the power switch (not shown) of the dynamic ROI apparatus A1002".

In step S1001, the comparison unit A1010 refers to the visual attribute table and determines whether or not the attribute of the object included in the front object information is registered as the visual attribute in the visual attribute table. As a result of this determination, if the attribute of the object included in the front object information is registered in the visibility attribute table as the visibility attribute, the process proceeds to step S208. On the other hand, if the attribute of the object included in the front object information is not registered in the visibility attribute table as the visibility attribute as a result of this determination, the process proceeds to step S206.

As described above, according to the present embodiment, even if objects overlap each other, if the object behind can be visually recognized through the object in front, the code amount of the object in front is the same as the code amount corresponding to the object in back. By doing so, the visibility of the object on the back side can be improved.

[Third Embodiment]
Although each functional part shown in FIG. 1 may be implemented by hardware, a part may be implemented by software. For example, the recording unit A1007 may be implemented as a memory, the code amount table A1012 and the visibility attribute table may be implemented as data, and other functional units may be implemented as computer programs. In this case, the computer device that has a memory as the recording unit A1007, has the code amount table A1012 and the visibility attribute table as data, and can execute the computer program implemented as other functional units is the dynamic ROI device A1002. applicable to

A hardware configuration example of a computer device applicable to the dynamic ROI device A1002 will be described with reference to the block diagram of FIG. Devices such as PCs (personal computers), tablet-type terminal devices, and smart phones are applicable to such computer devices.

The CPU 901 executes various processes using computer programs and data stored in the RAM 902 and ROM 903 . As a result, the CPU 901 controls the operation of the entire computer apparatus, and executes or controls each of the processes described above as those performed by the dynamic ROI apparatus A 1002 .

The RAM 902 has areas for storing computer programs and data loaded from the ROM 903 and the external storage device 906 . Furthermore, the RAM 902 has an area for storing data received from the outside via an I/F (interface) 907 (for example, images of each frame received from the image acquisition device A 1001). The RAM 902 also has a work area used when the CPU 901 executes various processes. Thus, the RAM 902 can provide various areas as appropriate. The ROM 903 stores setting data, a boot program, and the like of the computer device.

An operation unit 904 includes user interfaces such as a keyboard, a mouse, and a touch panel, and can input various instructions to the CPU 901 by the user's operation.

A display unit 905 is configured by a liquid crystal screen, a touch panel, or the like, and can display the processing result of the CPU 901 in the form of images, characters, or the like. Note that the display unit 905 may be a projection device such as a projector that projects images and characters.

The external storage device 906 is a large-capacity information storage device represented by a hard disk drive. An external storage device 906 stores an OS (operating system) and a computer program for causing the CPU 901 to implement the functions of the functional units except for the recording unit A1007, the code amount table A1012, and the visibility attribute table in FIG. The external storage device 906 also stores the data of the code amount table A1012 and the visibility attribute table, as well as the information known in the above description (specified time, etc.). Computer programs and data stored in the external storage device 906 are appropriately loaded into the RAM 902 under the control of the CPU 901 and are processed by the CPU 901 .

The I/F 907 functions as an interface for performing data communication with external devices. can.

CPU 901 , RAM 902 , ROM 903 , operation unit 904 , display unit 905 , external storage device 906 and I/F 907 are connected to bus 908 . Note that the hardware configuration of the computer device applicable to the dynamic ROI device A1002 is not limited to the configuration shown in FIG.

Further, in each of the above embodiments, the video acquisition device A1001, the dynamic ROI device A1002, and the display device A1003 are separate devices, but two or more of them may be integrated into one device. For example, the dynamic ROI device A1002 may be incorporated into the video acquisition device A1001 as a camera. In this case, the video acquisition device A1001 functions as an imaging device that sets the code amount of each object region in the image captured by itself, encodes the object region according to the code amount, and outputs the encoded data.

A part or all of the embodiments and modifications thereof described above may be appropriately combined. Also, a part or all of the embodiments and modifications thereof described above may be selectively used.

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

A1004: video acquisition unit A1005: area division unit A1006: comparison unit A1007: recording unit A1008: determination unit A1009: determination unit A1010: comparison unit A1011: code amount change unit A1012: code amount table A1013: code amount setting unit A1014: code conversion unit A1015: video output unit

Claims (9)

A set code amount that is set in advance for the attribute of the object in the back among the objects overlapping each other in the image is set in advance for the attribute of the object in the front among the objects that overlap each other in the image. setting means for setting a preset code amount for the back object as the code amount for the front object when the code amount is larger than the code amount;
An image processing apparatus, comprising: encoding means for encoding the area of the object in front with the code amount set for the object by the setting means. If the attribute of the front object among the objects overlapping each other in the image is an attribute preset as an attribute of the object whose back surface is visible, the rear surface of the objects overlapping each other in the image is determined. setting means for setting a code amount preset for an attribute of an object as a code amount for the front object;
encoding means for encoding the region of the object in front with the code amount set by the setting means for the object;
An image processing device comprising : The setting means is configured to set code amounts preset for attributes of objects that do not overlap each other in the image and objects that are on the back of objects that overlap each other in the image. 3. The image processing apparatus according to claim 1 or 2 , characterized in that it sets . Furthermore,
4. An image processing apparatus according to claim 1 , further comprising means for detecting objects overlapping each other in said image. Furthermore,
5. The image processing apparatus according to claim 1 , further comprising means for outputting the image encoded by said encoding means. means for acquiring said image;
An imaging apparatus comprising: the image processing apparatus according to any one of claims 1 to 5 . An image processing method performed by an image processing device,
The setting means of the image processing apparatus presets the code amount preset for the attribute of the back object among the overlapping objects in the image, and the attribute of the front object among the overlapping objects. a setting step of setting the code amount preset for the object on the back side as the code amount for the object on the front side when the code amount is larger than the code amount preset for the object;
An image processing method, wherein the encoding means of the image processing apparatus encodes the area of the object in front with the code amount set for the object in the setting step. An image processing method performed by an image processing device,
If the setting means of the image processing device determines that the attribute of the front object among the objects overlapping each other in the image is an attribute preset as the attribute of the object whose back surface can be visually recognized, in the image a setting step of setting, as a code amount for the front object, a set code amount preset for attributes of a back object among objects overlapping each other;
an encoding step in which the encoding means of the image processing apparatus encodes the area of the object in front with the code amount set for the object in the setting step;
An image processing method comprising: A computer program for causing a computer to function as each means of the image processing apparatus according to any one of claims 1 to 5 .

Images