JP5173377B2 - Image processing method and apparatus, program, and storage medium - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method, and more particularly to an image processing apparatus and an image processing method for performing color matching on moving image data.

  Various color perception models have been proposed to predict how human colors appear. A typical example is a color appearance model (CAM) called CIECAM02. In CIECAM02, parameters of an observation environment such as a white point, adaptation luminance, background luminance, and viewing environment conditions can be set. When color matching is performed using CIECAM02, these observation environment parameters can be set for each input / output, and color matching under different observation environments becomes possible.

  In addition, an image appearance model in which a visual spatial characteristic is incorporated into a color appearance model has been proposed. Furthermore, in recent years, color matching has been performed not only for still images but also for moving images. In order to predict the appearance of a moving image, it is necessary to further consider temporal characteristics of vision.

  For example, Non-Patent Document 1 discloses a technique in which an iCAM (image Color Appearance Model) that is an image appearance model is applied to a moving image. This document describes the following techniques. First, the low-pass image and the Y-low-pass image (absolute luminance image) for the frame from the frame image 10 seconds ago to the target frame image are multiplied by the adaptation weight calculated by the following equation (1). In equation (1), AW indicates the calculated weight, and f indicates the frame number. That is, for example, if there are 30 frames per second, f = 0 indicates the current frame number, and f = -300 indicates the frame number 10 seconds before.

Next, a low pass image and a Y low pass image of the frame of interest are generated by synthesizing the plurality of weighted low pass images and Y low pass images, respectively. Then, by using these in the iCAM framework, moving image reproduction reflecting spatial and temporal visual characteristics is performed.
MD Fairchild and GM Johnson, “Image appearance modeling,”. SPIE / IS & T Electronic Imaging Conference, SPIE Vol. 5007, Santa Clara, 149-160 (2003).

  In a general moving image, there are naturally a plurality of subject objects, and each object has different characteristics. However, according to the technique described in Non-Patent Document 1, uniform appearance processing is performed on the entire frame image. Therefore, since temporal visual characteristics and the like are uniformly reflected on the entire frame image, it has been impossible to perform appropriate processing in consideration of the characteristics of each object.

  This example will be described with reference to FIG. The car A and the car B shown in FIG. 13 are the same object, and these are moving at different speeds in the frame of the moving image. Although the car A and the car B are originally the same object, the colors (color appearances) perceived for each car may not match due to the difference in speed. As described above, the color appearance of an object in a moving image may differ depending on the object characteristics such as moving speed and position.

  As described above, when uniform appearance processing is applied to the entire frame image, the difference in characteristics between the car A and the car B illustrated in FIG. 13 cannot be taken into consideration, and appropriate for each object. Processing results may not be obtained.

  The present invention has been made to solve the above-described problem, and an object thereof is to provide an image processing apparatus and an image processing method for performing color conversion processing according to the feature amount for each object image in a frame image. And

  As a means for achieving the above object, an image processing apparatus of the present invention comprises the following arrangement.

  That is, an image processing apparatus that reproduces a moving image composed of a plurality of time-divided frame images so as to reflect human spatial visual characteristics and temporal visual characteristics, wherein the frame of interest in the plurality of frame images An object extraction unit that extracts an object image from an image, an object feature amount acquisition unit that acquires an object feature amount for the extracted object, and the object feature amount corresponding to each object image with respect to the target frame image Color conversion preparation means for performing color conversion preparation processing to create a color conversion image, frame adaptation time acquisition means for acquiring a frame adaptation time within a predetermined processing period up to the frame of interest, A frame adaptation feature is applied to the frame image of interest based on the frame adaptation time. A calculation unit frame adaptation feature amount to calculate the amount, relative to the frame image of interest, and having a color conversion unit that performs color conversion based on the color conversion image and the frame adaptation feature amount.

  According to the present invention having the above configuration, color conversion processing corresponding to the feature amount can be performed for each object image in the frame image.

  Hereinafter, the present invention will be described in detail based on preferred embodiments with reference to the accompanying drawings. The configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

<First Embodiment>
Apparatus Configuration FIG. 1 is a diagram illustrating a block configuration example of an image processing apparatus according to the present embodiment. In the figure, a CPU 701 manages the operation of the entire apparatus using a RAM 703 as a work memory in accordance with programs stored in a ROM 702 and an external storage device 705 such as a hard disk. The CPU 701 further executes various types of image processing including processing related to color matching in the present embodiment.

  The input unit 704 indicates an input device such as a keyboard and a mouse and an input interface thereof, and receives an input instruction from an operator. A display unit 706 shows a display device such as a monitor and a video interface, and displays a moving image and a user interface screen according to the present embodiment. An output unit 707 indicates an output device such as a printer and an output device interface.

  The moving image input unit 708 is a processing unit that inputs and processes moving image data from a device that handles moving image data such as a video camera, a digital camera, or a video player. For example, generally, moving image data is composed of an image and sound, but first, this is separated into an image signal and an audio signal. When an image signal is received as an analog signal, the image A / D processing unit converts this from analog to digital. When an audio signal is received as an analog signal, audio A / D processing converts from analog to digital. Part, etc. Further, the image processing apparatus includes a synthesis processing unit that compresses and synthesizes the processed digital image signal and digital audio signal. The moving image input unit 708 also includes a decoding processing unit that decodes the object code into an object image when moving image data is sent by an object encoding method such as MPEG4. The image data acquired by the moving image input unit 708 is converted into an image format such as a JPEG or TIFF image, whereby image data for color matching is obtained.

FIG. 2 is a flowchart showing an outline of a moving image reproduction process in the image processing apparatus according to the present embodiment, and details of each process will be described later.

  The moving image data input from the moving image input unit 708 shown in FIG. 1 is first divided into still images at predetermined time intervals in step S201. The divided still images are hereinafter referred to as “frame images”. In addition, a frame image that is a color matching processing target in the present embodiment is referred to as a “target frame image”. That is, in step S201, moving image data is time-divided to generate a frame image of interest.

  Next, in step S202, frame image information for the target frame image is created, and in step S203, an object is cut out from the target frame image. In step S <b> 204, object information necessary for color matching processing that reflects the human spatial visual characteristics and temporal visual characteristics, which are features of the present embodiment, is generated for the extracted object.

  In step S205, a low-pass image for the target frame image is generated according to these object feature amounts. In step S206, color matching processing is performed on the frame image of interest using the generated low-pass image, thereby generating moving image data to be reproduced.

  Hereinafter, the process of each step shown in FIG. 2 will be described in detail.

-Attention frame image division processing (S201)
The moving image data input from the moving image input unit 708 is time-divided at specific or arbitrary time intervals. The divided frame images are held in the ROM 702, the RAM 703, or the external storage device 705. At this time, an address indicating the storage destination of the frame image is stored in the frame image data address information 403, and the data size of the frame image is stored in the frame image size information 404 in the frame image information table shown in FIG. .

● Frame image information creation processing (S202)
Information about one frame image that is time-divided is stored in the ROM 702, the RAM 703, or the external storage device 705 as a frame image information table as shown in FIG.

  As shown in FIG. 3, the frame image information table of this embodiment has a frame sequence number 401 as frame image information identification information indicating number information for identifying frame image information. Furthermore, it has time information 402 indicating a time interval between frames, that is, an interval at which a frame image is drawn. Furthermore, it has frame image data address information 403 indicating the address to the frame image data stored in the storage device, and frame image size information 404 indicating the data length of the frame image data. Further, it has frame low-pass image address information 405 indicating the address of the low-pass image data for the frame image generated in step S205 described later. Further, it has object information table address information 406 indicating the address of the object information table storing the information of the object included in the frame image, and object information size information 407 indicating the data length of the table.

  In this embodiment, the frame sequence number 401 indicates the order in which frame images are reproduced, and is incremented and stored every time a frame image is cut out. The time information 402 stores the division time when the frame image is divided at a specific time interval. If this is an arbitrary time interval, a time processing device (equipped with a standard time) from the time when the previous frame image of the frame image of interest starts to be displayed until the frame of interest image is displayed ( (Internal clock).

Subject object extraction process (S203)
Here, a subject object included in the target frame image is extracted to obtain object image data. As a method of extracting the subject object, a well-known method can be applied. For example, the following technique is applied.

  For example, according to the technique described in Japanese Patent Application Laid-Open No. 2004-248312, the technique of the block matching method is used to detect the motion of a subject included in a moving image composed of a plurality of continuous frame images. Yes. In this block matching method, a frame image is divided into blocks (areas), and a similar position to the previous image is searched for in block units by pattern matching to detect a subject included in a series of images. The detected object is expressed by area boundary information indicating the boundary between the object and the background. Object image data can be generated from this region boundary information.

  There is also known an object encoding method in which a plurality of objects such as a background and a person are separated from a moving image and encoding is performed for each object. This object encoding method is supported by the international standard (ISO / IEC MPEG4) of the moving image encoding method. In the case of such an object encoding method, it is possible to receive an object code separated for each object and decode it to create object image data.

  In the present embodiment, each object extracted from the target frame image by some method is registered as new object information in the object information table shown in FIG. The address indicating the object information is stored in the object information table address information 406 of the frame image information table shown in FIG. 3, and the size of the object information is stored in the object information size information 407.

  As shown in FIG. 4, items described below are stored in the object information table of the present embodiment. First, it has an object number 501 indicating the identification number of the object information, and object classification information address information 502 indicating the address to the object classification information table indicating the classification of the object. Here, the object classification information represents an element of an object existing in the frame image. For example, “car A” shown in FIG. 13 described later is one classification. Details of the object classification information will be described later.

  Furthermore, it has object image position information 503 in which coordinate information of an object extracted as a subject object is stored. This object image position information 503 indicates the relative position of a pixel when the upper left corner of the frame image is (0, 0) and the upper left corner of the object image is (0, 0). Show. These coordinates are used when generating a low-pass image of the object.

  Further, the image processing apparatus is characterized by having object feature amount information 504 indicating a feature amount such as the moving speed and size of the object.

  In this subject object extraction process, the object number 501 and the object image position information 503 are stored in the object information table shown in FIG.

Object information generation process (S204)
FIG. 5 is a flowchart showing the flow of object information generation processing in step S204. First, it is determined whether or not the object image data extracted in the subject object extraction process in step S203 is already registered in an object classification information table described later in a frame image before the frame of interest (S301). This determination can be performed as follows, for example. First, with reference to the frame sequence number 401 of the frame information table, frame information having a frame sequence number that is one smaller than the sequence number of the frame image of interest is acquired. Then, an object image is extracted based on the object image position information 503 by referring to the object information table for the acquired frame information. Then, it is determined whether or not the extracted object image matches the object image cut out in the subject object extraction process in step S203. If they match, it can be determined that the image has already been registered.

  As a method for determining the coincidence of the object image, for example, it is possible to check whether all the pixels constituting the object image coincide with each other, or to perform pattern matching by enlarging, reducing, rotating, etc. the object image. It is also effective to perform. Further, whether or not the object image has been registered may be determined based on the ratio between the matching size of the object image and the original size of the object image. For moving images using object coding, an object signal for identifying an object is acquired, so it is easy to determine whether the objects match.

  If it is determined in step S301 that the extracted object is not registered in the object classification information table, information on the object is registered in the object classification information table as new object classification information in step S302.

  FIG. 6 shows a configuration example of the object classification information table. As shown in the figure, the object classification information table of this embodiment has an object classification number 901 for identifying the object classification. Further, it has a drawing start frame number 902 that indicates the frame number at which the object has started drawing, and a drawing end frame number 903 that indicates the frame number at which the object has finished drawing. Further, it has attention frame object feature amount information 904 indicating the object feature amount in the attention frame (that is, the drawing end frame).

  When newly registering object classification information in the object classification information table, 1 is added to the maximum value of the object classification number 901 and a new object classification number 901 is registered. Then, the target frame number is registered for both the drawing start frame number 902 and the drawing end frame number 903. Then, the address indicating the registered object classification information is stored in the object classification information address information 502 of the object information table shown in FIG.

  On the other hand, if it is determined in step S301 that the object classification information has been registered, the process proceeds to step S303, where the registered object classification information is updated. That is, 1 is added to the drawing end frame number 903 of the object classification information. Further, the address of the already registered object classification information is stored in the object classification information address information 502 of the object information table shown in FIG.

  After completing the new registration process in step S302 or the update process in step S303, the feature amount of the object is calculated in step S304. Here, the “movement speed” of the object will be described as an example of the object feature amount. The object moving speed is based on, for example, the following (2) based on the difference ΔPos in the object image position information 503 and the difference ΔT in the time information 402 between the frame of interest and the immediately preceding frame (frame sequence number −1 of the frame of interest). ).

  In the equation (2), f indicates a frame number. When the object is a new drawing object, V (f) = 0 may be set.

  Here, the “movement speed” of the object has been described as an example of the object feature amount, but “movement acceleration”, “size”, and the like of the object can also be used as other feature amounts. The “movement acceleration” can be calculated by dividing V (f) in the above equation (2) by ΔT, and the “size” can be calculated by calculating the connected component of the object. Further, the object image position information 503 stored in the object information table shown in FIG. 4 may be used as the object feature amount.

  The object feature amount calculated here is stored in the object feature amount information 504 in the object information table shown in FIG. 4 and the target frame object feature amount information 904 in the object classification information table shown in FIG. If the object is not a new object, the attention frame object feature amount information 904 is overwritten and updated.

  As described above, when the object feature amount is calculated in step S304, it is determined in step S305 whether object information has been generated for all object images in the frame. If all creations have been completed, the object information generation process is terminated. If not, the process returns to step S301 to generate the next object information.

  In the object information generation process in step S204, that is, object classification information address information 502 and object feature amount information 504 are stored in the object information table shown in FIG.

● attention frame low-pass image generation processing (S205)
FIG. 7 is a flowchart showing the flow of the attention frame low-pass image generation processing in step S205.

  First, in step S601, a frame low-pass image for the frame image of interest is generated and stored, and the address information is set in the frame image information table shown in FIG. Since this frame low-pass image is treated as an XYZ image, the frame image of interest is converted into an XYZ image and generated by applying a low-pass filter such as a Gaussian filter. In step S601, low-pass image generation processing is performed in accordance with the object feature amount that is a feature of the present embodiment. Details thereof will be described later.

  Next, the frame sequential numbers of the start frame and the end frame that are the generation targets of the weighted low-pass image described later are acquired (S602). In the present embodiment, similarly to the technique described in Non-Patent Document 1, a low-pass image is generated for a frame image that exists within a processing period up to a frame of interest and a frame 10 seconds before that frame. Therefore, the frame sequence number of the frame corresponding to −10 sec from the frame of interest is set as the start frame sequence number for low pass image generation. Also, the frame sequence number of the frame of interest is set as the end frame sequence number that is the target of low-pass image generation. Here, the time information of −10 sec is calculated based on the time information 402 of the frame image information table shown in FIG. At this time, if there is no frame corresponding to exactly −10 sec, the frame closest to −10 sec may be used instead.

  Then, the start frame serial number of the low-pass image generation target set in step S602 is set in the low-pass image generation counter i (S603).

  In steps S604 to S606, a frame low-pass image for one frame image is generated. Hereinafter, the process for this one-frame image will be described.

  First, the frame sequential frame low-pass image indicated by the low-pass image generation counter i and the time information are acquired from the frame image information table shown in FIG. 3 (S604). Next, the relative time of the frame image is calculated based on the time information of the frame of interest and the time information of the frame with the frame sequence number i (S605). This relative time corresponds to the frame adaptation time. For example, if 30 frames are drawn per second, this adaptation time is 0 for the frame of interest and −300 for the frame corresponding to −10 sec. It becomes.

  When the frame adaptation time acquisition process in step S605 is completed as described above, in step S606, a weighted low-pass image to which the frame adaptation time is applied is generated. That is, the frame low-pass image acquired in step S604 is multiplied by a low-pass weight AW that is a frame adaptation feature amount. This AW can be calculated by applying the frame adaptation time acquired in step S605 to the above equation (1). As a result, a weighted low-pass image that reflects visual temporal characteristics is generated.

  Next, it is determined whether or not the low-pass image generation counter i corresponds to a low-pass image generation target end frame (that is, a target frame) (S607). If it is not the end frame, 1 is added to the low-pass image generation counter i (S608), and the process proceeds to step S604. On the other hand, if it is the end frame, the low-pass image for the frame of interest is generated by synthesizing the weighted low-pass images of the frames generated in step S606 (S609).

Details of low-pass image generation processing (S601)
FIG. 8 is a flowchart showing frame low-pass image generation processing of the frame of interest in step S601.

  First, in step S801, a background low-pass image of the frame of interest is generated. This background low-pass image is an image obtained by applying a Gaussian filter, which is a low-pass filter, to the frame image of interest, as in the technique described in Non-Patent Document 1. Of course, the low-pass processing here is not limited to filtering by a Gaussian filter, and low-pass processing based on other low-pass filters or human contrast sensitivity characteristics may be performed.

  Next, in steps S802 and S803, the processing in the object frame image is changed according to the feature amount of the object, which is a feature of this embodiment, to generate a low-pass image of the object.

  In step S802, the object feature amount of one object in the target frame image is acquired. As the object feature amount, attention frame object feature amount information 904 stored in the object classification information table shown in FIG. 6 or object feature amount information 504 in the object information table shown in FIG. 4 is used. Here, it is assumed that the “movement speed” feature amount in the target frame stored in the target frame object feature amount information 904 is acquired.

  Next, in step S803, an object color conversion image, that is, a low-pass image is generated as color conversion preparation processing in the present embodiment. Here, the filter size of the low-pass filter is changed in accordance with the “movement speed” of the object acquired in step S802. For example, it is conceivable to perform a step-by-step change such as preparing filters of three levels such as “large”, “medium”, and “small” and using a larger filter as the moving speed increases. Alternatively, the continuous change may be performed using a function that inputs the moving speed, outputs the filter size, and outputs a smaller value as the moving speed is lower.

  Here, an example is shown in which the “movement speed” of the object is used as the object feature amount used for changing the filter processing, but instead of this, the “movement acceleration”, “size”, “position information”, etc. of the object Other feature quantities may be used. A combination of a plurality of these feature amounts may be used.

  In step S804, the object low-pass image generated in step S803 is combined with the background low-pass image generated in step S801. Specifically, the object region of the background low-pass image is overwritten with the object low-pass image.

  Next, in step S805, it is determined whether or not processing has been completed for all the objects in the frame. If unprocessed objects remain, the process returns to step S802. If the processing for all objects is completed, the process proceeds to step S806.

  In step S806, the background low-pass image obtained by combining all the object low-pass images is stored in the ROM 702, the RAM 703, or the external storage device 705 as a frame low-pass image. Then, the address is stored in frame low-pass image address information 405 of the frame image information table shown in FIG.

  As described above, the low-pass image for the frame of interest is generated by the processing described with reference to the flowcharts of FIGS.

  FIG. 9 schematically shows the above operation. According to FIG. 9, it can be seen that the processing corresponding to the object feature amount (movement speed) is performed on each object of the car A and the car B. That is, the low-pass filter process corresponding to the moving speed is first performed on each object of the car A and the car B to create a low-pass image for each frame. Then, a low-pass image for the frame of interest is generated by synthesizing a plurality of frames of low-pass images based on the weighting of equation (1).

Reproduction image generation process (S206)
FIG. 10 is a flowchart showing the flow of the reproduction image generation process in step S206. Here, an example in which the iCAM framework is used as in Non-Patent Document 1 is shown.

  First, chromatic adaptation conversion processing is performed on the target frame image (XYZ image) and the frame low-pass image generated in step S205 based on the following equations (3) and (4) (S1001).

  If each pixel of the frame image of interest converted by the equation (3) is R, G, B, and each pixel of the low-pass image converted by the equation (3) is Rw, Gw, Bw, RGB after chromatic adaptation Values (Rc, Gc, Bc) are calculated from the following equations (5) to (7).

  Here, D is a parameter representing the degree of adaptation to the device white color, and takes a value from 0 (no adaptation) to 1 (complete adaptation).

  Next, the observation condition coefficient FL is calculated for each pixel (S1002). Here, the observation condition FL is calculated by the following equation (8) based on the Y low-pass image (absolute luminance image) created from the Y value (YA) of the frame low-pass image.

  Next, the RGB values after adaptation (Rc, Gc, Bc) converted in step S1001 are converted into an IPT space (S1003). Here, the IPT space is a rectangular color space in which I represents the degree of light and darkness, P represents a red-green hue, and T represents a yellow-blue hue. Hereinafter, the conversion process to the IPT space will be specifically described.

  First, Rc, Gc, and Bc after adaptation are converted into a cone response space LMS according to the following equations (9) to (11).

  Then, non-linear compression of the cone response is applied according to the following equations (12) to (14), and the FL value calculated by equation (8) is used at that time.

  Thereafter, conversion to the IPT space is performed using the following equations (15) and (16).

  When the conversion to the IPT space is completed as described above, the color in the IPT space is then inversely converted to the RGB space of the monitor that displays the moving image (S1004). For conversion from IPT to monitor RGB, first, reverse conversion from IPT value to adaptive RGB value is performed, converted to XYZ value according to the monitor's observation environment, and further converted to monitor RGB value based on the characteristics of the monitor Is done by doing.

  As described above, according to the present embodiment, a more appropriate appearance process is applied to a moving image by changing the low-pass image generation process based on the feature amount for each object in the frame. Is possible.

Second Embodiment
Hereinafter, a second embodiment according to the present invention will be described. The moving image reproduction process in the second embodiment is almost the same as that in the first embodiment described above, but the attention frame low-pass image generation process in step S205 is different. Therefore, the attention frame low-pass image reproduction process in the second embodiment will be described below, and the other processes are the same as those in the first embodiment described above, and the description thereof will be omitted.

● attention frame low-pass image generation processing (S205)
FIG. 11 is a flowchart showing the attention frame low-pass image generation processing in the second embodiment. In the second embodiment, the timing for performing the process according to the object feature amount is different from the process shown in the flowchart of FIG. 7 in the first embodiment. That is, in the first embodiment, in step S601, a process corresponding to the object feature is performed on the frame of interest. However, as described later, the second embodiment is characterized in that each object feature amount is used not only for the frame of interest but also for the previous frame.

  First, in step S1101, a frame low-pass image for the frame image of interest is generated and stored. Since this frame low-pass image is treated as an XYZ image, the frame image of interest is converted into an XYZ image and generated by applying a low-pass filter such as a Gaussian filter. The created frame low-pass image is stored in the ROM 702, RAM 703, or external storage device 705, and its address is stored in the frame low-pass image address information 405 of the frame image information table shown in FIG. In step S1101, the low-pass image generation process corresponding to the object feature amount is not performed as in step S601 of FIG. 7 described in the first embodiment.

  Next, the frame sequential numbers of the start frame and end frame that are the generation targets of the weighted low-pass image to be described later are acquired (S1102). In the present embodiment, similarly to the technique described in Non-Patent Document 1, a low-pass image is generated for a frame image that exists between a frame of interest and a frame 10 seconds before that frame. Therefore, the frame sequence number of the frame corresponding to −10 sec from the frame of interest is set as the start frame sequence number for low pass image generation. Also, the frame sequence number of the frame of interest is set as the end frame sequence number that is the target of low-pass image generation. Here, the time information of −10 sec is calculated based on the time information 402 of the frame image information table shown in FIG. At this time, if there is no frame corresponding to exactly −10 sec, the frame closest to −10 sec may be used instead.

  Then, the low-pass image generation counter i is set to the start frame serial number of the low-pass image generation target set in step S1102 (S1103).

  In steps S1104 to S1108, a frame low-pass image for one frame image is generated. Hereinafter, the process for this one-frame image will be described.

  First, the frame sequential frame low-pass image indicated by the low-pass image generation counter i and the time information are acquired from the frame image information table shown in FIG. 3 (S1104). Next, the relative time of the frame image is calculated based on the time information of the frame of interest and the time information of the frame with the frame sequence number i (S1105).

  Next, a weighted low-pass image is generated based on the relative time (S1106). Specifically, based on the relative time calculated in step S1105, the low-pass weight AW is calculated from the equation (1), and the low-pass weight AW is multiplied by the frame low-pass image acquired in step S1104. As a result, a weighted low-pass image reflecting visual temporal characteristics is generated, and this low-pass image is hereinafter referred to as a weighted background low-pass image.

  Next, in steps S1107 and S1108, the processing in the frame image is changed according to the feature amount of the object, which is a feature of the second embodiment, to generate a weighted low-pass image.

  In step S1107, the object feature amount of one object in the frame image of frame sequence number i is acquired. As the object feature amount, the object feature amount information 504 stored for each frame in the object information table shown in FIG. 4 is used. Here, it is assumed that the “movement speed” feature amount is used.

  In step S1108, a weight is set for the object image area in the weighted background low-pass image generated in step S1106, and the weighted background low-pass image is updated. For example, a weighting function VW (V) for the moving speed V is defined such that the moving speed V approaches 1 as the moving speed V increases, and approaches 0 as the V decreases, and the moving speed of the object acquired in step S1107 depends on the moving speed. A weight VW is calculated. This weight VW is the object adaptation feature amount in the second embodiment. By multiplying the weight VW by the data of the object image area in the weighted background low-pass image generated in step S1106, a low-pass image weighted according to the feature amount of the object is generated.

  FIG. 12 schematically shows the above operation. According to FIG. 12, a weighted low-pass image for the frame of interest is generated by performing weighting (VW) based on the object feature amount (moving speed) for each object for each object of car A and car B. You can see that

  In this example, the “movement speed” of the object is used as the object feature amount for calculating the weight for the object. Instead, the “movement acceleration”, “size”, and “position information” of the object are used instead. Other feature amounts such as may be used. A combination of a plurality of these feature amounts may be used.

  Then, it is determined whether an object low-pass image has been generated by applying the adaptation time for each object to all objects in the frame (S1109). If the processing for all objects has been completed, the process proceeds to step S1110. If the process has not been completed, the process returns to step S1107.

  In step S1110, it is determined whether or not the low-pass image generation counter i corresponds to an end frame (that is, a target frame) that is a low-pass image generation target. If it is not the end frame, 1 is added to the low-pass image generation counter i (S1111), and the process proceeds to step S1104. On the other hand, if it is an end frame, a plurality of weighted low-pass images generated in step S1108 are combined to generate a low-pass image for the frame of interest (S1112).

  In the second embodiment, the example in which the processing change according to the object feature amount is realized by changing the weight value of the object low-pass image according to the feature amount is described. However, the second embodiment is different from the first embodiment described above. Similarly, the filter size may be changed.

  As described above, according to the second embodiment, in addition to obtaining the same effects as those of the first embodiment described above, not only the attention frame but also the feature amount in the previous frame is used. The temporal visual characteristics can be controlled in more detail.

<Other embodiments>
Although the embodiment has been described in detail above, the present invention can take an embodiment as a system, apparatus, method, program, recording medium (storage medium), or the like. Specifically, the present invention may be applied to a system composed of a plurality of devices (for example, a host computer, an interface device, an imaging device, a web application, etc.), or may be applied to a device composed of a single device. good.

  The present invention also provides a software program that realizes the functions of the above-described embodiments directly or remotely to a system or apparatus, and the system or apparatus computer reads out and executes the supplied program code. Achieved. The program in this case is a computer-readable program that corresponds to the flowchart shown in the drawing in the embodiment.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, or the like.

  Recording media for supplying the program include the following media. For example, floppy disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD- R).

  As a program supply method, the following method is also possible. That is, the browser of the client computer is connected to a homepage on the Internet, and the computer program itself (or a compressed file including an automatic installation function) of the present invention is downloaded to a recording medium such as a hard disk. It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. It is also possible to make it. That is, the user can execute the encrypted program by using the key information and install it on the computer.

  Further, the functions of the above-described embodiments are realized by the computer executing the read program. Furthermore, based on the instructions of the program, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can also be realized by the processing.

  Further, the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, and then executed, so that the program of the above-described embodiment can be obtained. Function is realized. That is, based on the instructions of the program, the CPU provided in the function expansion board or function expansion unit can perform part or all of the actual processing.

It is a figure which shows the block structural example of the image processing apparatus which concerns on one Embodiment which concerns on this invention. It is a flowchart which shows the moving image reproduction process in this embodiment. It is a figure which shows the example of a frame image information table in this embodiment. It is a figure which shows the example of an object information table in this embodiment. It is a flowchart which shows the flow of the object information generation process in embodiment of this invention. It is a figure which shows the structural example of the object classification information table in this embodiment. It is a flowchart which shows the attention frame low-pass image generation process in this embodiment. It is a flowchart which shows the frame low-pass image generation process of the attention frame in this embodiment. It is a schematic diagram which shows a mode that the low-pass image generation according to an object feature-value is performed in the appearance process of this embodiment. It is a flowchart which shows the reproduction | regeneration image generation process in this embodiment. It is a flowchart which shows the flow of the attention frame low-pass image generation process in 2nd Embodiment. It is a schematic diagram which shows a mode that the low-pass image generation according to an object feature-value is performed in the appearance process of 2nd Embodiment. It is a schematic diagram which shows a mode that an object feature-value is not reflected in the conventional appearance process.

Claims (12)

An image processing apparatus for reproducing a moving image composed of a plurality of time-divided frame images so as to reflect human spatial visual characteristics and temporal visual characteristics,
An object extracting means for extracting an object image from the frame image of interest in the plurality of frame images;
An object feature amount acquisition means for acquiring an object feature amount for the extracted object;
Color conversion preparation means for creating a color conversion image by performing color conversion preparation processing according to the object feature amount for each object image with respect to the target frame image;
A frame adaptation time acquisition means for acquiring a frame adaptation time within a predetermined processing period until the frame of interest;
A frame adaptation feature amount calculating unit that calculates a frame adaptation feature amount based on the frame adaptation time for the frame image of interest;
Color conversion means for performing color conversion based on the image for color conversion and the frame adaptation feature amount for the frame image of interest;
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the color conversion preparation unit creates a low-pass image for the object frame image using a low-pass filter corresponding to the object feature amount for each object image. .   The image processing apparatus according to claim 2, wherein the color conversion preparation unit changes a size of the low-pass filter in accordance with the object feature amount. An image processing apparatus for reproducing a moving image composed of a plurality of time-divided frame images so as to reflect human spatial visual characteristics and temporal visual characteristics,
An object extracting means for extracting an object image from the frame image of interest in the plurality of frame images;
An object feature amount acquisition means for acquiring an object feature amount for each of the extracted object images within a predetermined processing period until the frame of interest;
Object adaptation feature amount calculating means for calculating an object adaptation feature amount based on the object feature amount for each object image;
Color conversion means for performing color conversion on the object frame image based on the object adaptation feature for each object image;
An image processing apparatus comprising: The image processing apparatus according to claim 4 , wherein the object adaptation feature amount represents an adaptation weight reflecting a human temporal visual characteristic.   The image processing apparatus according to claim 1, wherein the object feature amount includes information on a moving speed of the object image.   The image processing apparatus according to claim 1, wherein the object feature amount includes information on movement acceleration of the object image.   The image processing apparatus according to claim 1, wherein the object feature amount includes information on a size of the object image.   The image processing apparatus according to claim 1, wherein the object feature amount includes information on a position of the object image in the frame image of interest. An image processing method for reproducing a moving image composed of a plurality of time-divided frame images so as to reflect human spatial visual characteristics and temporal visual characteristics,
An object extraction step for extracting an object image from the frame image of interest in the plurality of frame images;
An object feature quantity acquisition means for acquiring an object feature quantity for the extracted object;
A color conversion preparation step for preparing a color conversion image by performing color conversion preparation processing according to the object feature amount for each object image with respect to the target frame image;
A frame adaptation time acquisition unit for acquiring a frame adaptation time within a predetermined processing period until the frame of interest;
A frame adaptation feature amount calculating unit that calculates a frame adaptation feature amount based on the frame adaptation time with respect to the frame image of interest;
A color conversion step in which color conversion means performs color conversion based on the image for color conversion and the frame adaptation feature amount for the frame image of interest;
An image processing method comprising:   A computer program that, when executed by a computer, causes the computer to function as the image processing apparatus according to any one of claims 1 to 9.   A computer-readable recording medium having recorded thereon the computer program according to claim 11.

Images