JP4670631B2 - Image processing apparatus, image processing method, program for image processing method, and recording medium recording program for image processing method - Google Patents

Description

The present invention relates to an image processing apparatus, image processing method, a recording medium recording a program of a program and an image processing method of the image processing method can be applied to, for example, a video camera, tuner, a DVD (Digital Versatile Disk) recorder or the like . In the present invention, video data is sequentially processed in the order from the center of the screen to the periphery. To simplify the configuration and reduce power consumption.

  2. Description of the Related Art Conventionally, in a video camera that is one of the moving image processing apparatuses, video data obtained as a result of imaging is subjected to white balance adjustment, knee adjustment, noise reduction processing, and the like by an image processing circuit, and then compressed and recorded on a recording medium. Further, video data based on the imaging result is input to an image processing circuit for display, and the resolution and gamma are corrected according to the monitor mechanism used for display and used for the monitor.

  In conventional video equipment, this type of processing may be executed by software processing using a digital signal processor. In processing using such software, one frame of video data corresponding to one frame is handled. If it cannot be processed in this period, video data is processed in real time by increasing the processing speed by increasing the number of digital signal processors or adding dedicated hardware circuits.

  In addition, in this type of processing, there are processes in which the processing time varies depending on the processing target, such as a noise reduction process for adaptively switching the processes, and the process using software is sufficient even in the worst case. A digital signal processor with a high processing speed is adopted so that it can be processed. In this case, the number of digital signal processors is increased or a dedicated hardware circuit is added, and the processing speed is increased and video data is increased. It was processed in real time.

  In recent years, in this type of moving image processing apparatus, the resolution and frame frequency of video data to be processed tend to increase, and accordingly, image processing is required to be executed at high speed.

  With regard to the processing in such an image processing circuit, Japanese Patent Application Laid-Open No. 2004-121958 proposes a device for noise reduction processing.

By the way, in image processing using software, even in the worst case, it can be processed in real time sufficiently. When processing video data in real time by increasing the processing speed, the worst case occurs due to the situation that occurs very rarely. In normal processing other than the above, the operation speed is unnecessarily increased, and there is a problem in that the configuration becomes complicated and the power consumption increases.
JP 2004-121958 A

The present invention has been made in consideration of the above points. When video data is processed in real time, an image processing apparatus, an image processing method, An image processing method program and a recording medium on which the image processing method program is recorded are proposed.

In order to solve this problem, the invention of claim 1 is characterized in that a video data acquisition unit for acquiring video data and one field of the video data or a processing period that is an integral multiple of one field or one frame period of the video data. When applied to an image processing apparatus having a processing unit that repeats data processing of one frame, the processing unit performs data processing on a timer that measures an elapsed time in the processing period and video data of one field or one frame. A data processing unit, wherein the data processing unit processes the video data for each data processing unit in an order from the center to the periphery of the image by the video data, and the elapsed time measured by the timer , in between the processing period, the data processing of the video data of one field or one frame is not completed If, among the processing period, aborted data processing of the video data of one field or one frame.

The invention according to claim 6 is applied to an image processing method that repeats data processing of one field or one frame of video data in a processing period that is an integral multiple of one field or one frame of video data. A time measurement step for measuring an elapsed time in a period; and a data processing step for data processing of the video data of one field or one frame, wherein the data processing step starts from the center of the image by the video data. the order toward the periphery, for each data processing unit, the video data to the data processing, the elapsed time measured by said step of time measurement, in between the processing period, the video data of one field or one frame If the data processing is not completed, between the processing period, the one field or one frame of the bi Abort the data processing of Odeta.

According to a seventh aspect of the present invention, there is provided an image processing method in which data processing of one field or one frame of video data is repeated by a processing period that is an integral multiple of one field or one frame of video data by execution by an arithmetic processing means. The time measurement step for measuring the elapsed time in the processing period, and the data processing step for processing the video data of one field or one frame, the data processing step comprising: , the order toward the periphery from the center of the image by the video data, for each data processing unit, the video data to the data processing, the elapsed time measured by said step of time measurement, in between the processing period, the 1 when the data processing of the video data in the field or one frame is not completed, the processing Between, aborted data processing of the video data of one field or one frame.

The invention according to claim 8 is an image processing method in which data processing of one field or one frame of video data is repeated by a processing period that is an integral multiple of one field or one frame of video data, when executed by arithmetic processing means. The program of the image processing method is applied to a recording medium on which the above program is recorded. The program of the image processing method includes a time measurement step for measuring an elapsed time in the processing period, and data processing for processing the video data of one field or one frame. The data processing step includes: processing the video data for each data processing unit in an order from the center to the periphery of the video data image, and measuring the time measurement step. the elapsed time that, in between the processing period, the one field or one frame If data processing Deodeta is not completed, between the processing period, aborted data processing of the video data of one field or one frame.

According to the configuration of claim 1, data processing of one field or one frame of the video data is performed by a video data acquisition unit that acquires video data and a processing period that is an integral multiple of the period of one field or one frame of the video data. When applied to an image processing apparatus having a repetitive processing unit, the processing unit has a timer for measuring an elapsed time in the processing period and a data processing unit for processing the video data of one field or one frame. and, wherein the data processing unit, the order toward the periphery from the center of the image by the video data, for each data processing unit, the video data to the data processing, the elapsed time measured by the timer, in between the processing stage in the case where data processing of the video data of one field or one frame is not completed, between the processing period If data processing of one field or one frame of video data is terminated, real-time processing that repeats data processing of one field or one frame of the video data in a processing period that is an integral multiple of one field or one frame of video data in, when the processing is not in time, by completing the processing between omitted and the processing stage the processing of low peripheral portion importance, that the amount, which simplifies the structure compared to the conventional, to reduce power consumption Can do.

Thereby claim 6, claim 7, according to the configuration of claim 8, simplified structure as compared with the conventional image processing method capable of reducing power consumption, program and an image processing method of the image processing method It is possible to provide a recording medium on which the program is recorded.

  According to the present invention, when video data is processed in real time, the configuration can be simplified and power consumption can be reduced as compared with the conventional case.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate.

(1) Configuration of Embodiment FIG. 2 is a block diagram showing a video camera according to an embodiment of the present invention. The video camera 1 records an imaging result of a moving image on a recording medium, and reproduces and displays the recorded imaging result.

  That is, in this video camera 1, the lens 2 is configured to change the focus, aperture, and zoom to condense incident light under the control of a controller (not shown), and then subject to the imaging surface of the imaging device provided in the moving image receiver 3. The optical image is formed.

  The moving image receiving unit 3 performs photoelectric conversion processing on the optical image formed on the imaging surface of the image sensor, and acquires the imaging result of the moving image. In addition, the moving image receiving unit 3 performs correlated double sampling processing or the like on the output signal of the imaging device based on the imaging result, and then performs analog / digital conversion processing to output video data.

  The received image processing unit 4 sequentially inputs video data output from the moving image receiving unit 3 to the global data memory 5 and executes processes such as white balance adjustment, knee adjustment, and gamma adjustment. Also, by processing this video data, information necessary for focus control and aperture control of the lens 2 is acquired and notified to a controller (not shown), and the video camera 1 controls the focus and aperture of the lens 2 by control of the controller based on this information. To control.

  In this embodiment, the received image processing unit 4 detects a region where a face is photographed from an image based on video data by face recognition processing using a pattern matching technique, and the luminance level and high frequency of the video data in this region are detected. The signal level of the component is detected. In addition, this detection result is notified to the controller as information necessary for aperture control and focus control. Based on this information, the controller sets the aperture so that the brightness level of the area in which the face is detected becomes a predetermined value. Variable and aperture control. Further, by so-called hill-climbing method, focus control is performed by changing the focus in the direction in which the signal level of the high-frequency component increases in the area where the face is detected.

  As a result, the received image processing unit 4 sequentially processes the video data in real time by data processing in which the time required for processing varies depending on the video data. Note that the received image processing unit 4 may detect the color temperature of the area where the face is photographed and adjust the white balance based on the color temperature.

  The image processing unit 6 performs noise reduction processing on the video data processed by the received image processing unit 4 and held in the global data memory 5. The image processing unit 6 performs the noise reduction process by adaptively switching the process according to the video data to be processed. Thus, the image processing unit 6 sequentially processes the video data in real time by data processing in which the time required for processing varies depending on the video data.

  The display image processing unit 7 processes the video data processed by the image processing unit 6 and stored in the global data memory 5 or the video data decoded by the image compression / decompression unit 8 described later and stored in the global data memory 5. The resolution and gamma suitable for the subsequent moving image display unit 9 are corrected.

  In this processing, the display image processing unit 7 executes resolution conversion processing by two-dimensional filtering processing, and adaptively switches the characteristics of the filter used for the filtering processing according to the video data. As a result, the display image processing unit 7 sequentially processes the video data in real time by data processing in which the time required for processing varies depending on the video data.

  The display image processing unit 7 generates video data for on-screen display under the control of a controller (not shown), combines the generated video data with video data based on the imaging result, and outputs the synthesized video data to the moving image display unit 9. For example, the recording time, remaining battery power, etc. are displayed on the screen.

  The moving image display unit 9 is configured by a liquid crystal display panel, for example, and displays a video image based on the video data processed by the display image processing unit 7. Thus, the video camera 1 is configured to monitor the imaging result acquired by the moving image receiver 3 and the imaging result recorded on the recording medium.

  The image compression / decompression unit 8 compresses the video data processed by the image processing unit 6 and stored in the global data memory 5 by, for example, a moving picture expert group (MPEG) technique, and the encoded data obtained as a result of the processing is compressed. The media recording / playback unit 10 records the encoded data output from the image compression / decompression unit 8 on a recording medium. Here, the recording medium is, for example, an optical disk, a memory card, a magnetic disk, a magnetic tape, or the like.

  On the other hand, the media recording / reproducing unit 10 sequentially reproduces the encoded data of the moving image file recorded on the recording medium under the control of the controller and outputs the encoded data to the image compression / decompression unit 8. The encoded data output from the media recording / reproducing unit 10 is decompressed and stored in the global data memory 5. As a result, in the video camera 1, during reproduction, video data stored in the global data memory 5 is processed by the display image processing unit 7 by the image compression / decompression unit 8 and is displayed on the moving image display unit 9. A monitor image of the recorded imaging result is displayed.

  As a result, in this video camera 1, the received image processing unit 4, the image processing unit 6, and the display image processing unit 7 that access the global data memory 5 to process the video data all process the video data in real time, One frame of video data is processed so as to be processed in the period of one frame and transferred to the subsequent processing.

  In this embodiment, the received image processing unit 4, the image processing unit 6, and the display image processing unit 7 are constituted by a digital signal processor, and this kind of image is obtained by software processing in a processor provided in the digital signal processor. As shown in FIG. 3, each video data is subjected to image processing from the center of the screen toward the periphery as shown in FIG. In addition, when image processing is performed in this way and one frame of video data due to the worst case cannot be processed in a period of one frame, the processing is interrupted and the processing is switched to the next frame. Process with.

  FIG. 4 is a diagram showing the processing order in the conventional configuration in comparison with FIG. 3. In the conventional configuration, these image processing units are configured so that one frame of video data can be processed in a period of one frame. The image processing is performed in the order of raster scanning in units of data processing of one pixel or a plurality of pixels. Note that the processing programs of the processors provided in the image receiving image processing unit 4, the image processing unit 6, and the display image processing unit 7 are provided by prior installation, but are provided by such prior installation. Instead of this, it may be provided via a recording medium such as an optical disk, a magnetic disk, or a memory card, or may be provided by downloading via a network such as the Internet.

  FIG. 5 is a block diagram illustrating a configuration of the image processing unit 6. In the image processing unit 6, the data input unit 13 accesses the global data memory 5 under the control of the processor 14 and inputs the video data to be processed by the amount corresponding to the image processing unit. The local data memory 15 temporarily stores and holds the video data input from the data input unit 13 and outputs it to the local bus BUS under the control of the processor 14. The local data memory 15 acquires and holds video data output to the local bus BUS under the control of the processor 14, and the data output unit 12 stores the video data held in the local data memory 15 under the control of the processor 14. Output to the global data memory 5. As a result, the image processing unit 6 acquires the video data held in the global data memory 5 in units of image processing, holds the video data in the local data memory 15, and processes the video data held in the local data memory 15. The video data stored in the global data memory 5 is processed again by repeating this series of processing.

  In the image processing unit 6, the timer 16 measures the elapsed time in the processing unit period, which is a frame period, and notifies the processor 14 under the control of the processor 14.

  The image processing auxiliary circuit 17 is a circuit that assists the processing by the processor 14, and has a hardware configuration that executes complicated processing in processing by software, and is configured by, for example, a multiplication circuit.

  The processor 14 processes the video data stored in the local data memory 15 by executing a processing program recorded in a memory (not shown) and stores the video data in the local data memory 15. In the processing of the video data, the processor 14 processes the video data at a high speed using the image processing auxiliary circuit 17 as necessary according to the processing program.

  In the image processing circuit 6, the processing of the video data by the processor 14 is a noise reduction process in which the processing is adaptively switched according to the video data to be processed and the noise of the video data is removed as described above. Thereby, for example, processing in which the processing time of video data for one frame varies in accordance with the video data is executed.

  The processor 14 receives a processing start instruction signal S7 from the host controller as an interrupt or polling target signal in a frame period, and executes the processing procedure shown in FIGS. 1 and 6 by this processing start instruction signal S7 to obtain video data. To process.

  That is, when the processing start instruction signal S7 is input (FIG. 6), the processor 14 proceeds from step SP1 to step SP2, and detects the processing start instruction signal S7 by interruption or polling. Thus, the processor 14 starts processing the video data, moves to step SP3, resets the timer 16, and then instructs the timer 16 to start measuring elapsed time in the subsequent step SP4.

  Subsequently, in step SP5, the processor 14 notifies the data input unit 13 of parameters related to the process of reading video data from the global data memory 5 to the local data memory 15, and in step SP6, the parameters notified in step SP5. The data input unit 13 is instructed to read the video data. Here, the parameters are, for example, a read address on the global data memory 5, a write address on the local data memory 15, and a size.

  Subsequently, the processor 14 proceeds to step SP7 and determines whether or not the reading of the video data to the local data memory 15 is completed. If a negative result is obtained here, step SP7 is repeated, whereas at step SP7. If a positive result is obtained, the process proceeds to step SP8. Thus, the processor 14 reads out video data for one image processing unit from the global data memory 5 and stores it in the local data memory 15.

  In the following step SP8, the processor 14 performs image processing on the video data stored in the local data memory 15 and stores it again in the local data memory 15, and in the subsequent step SP9, the video from the local data memory 15 to the global data memory 5 is stored. The data output unit 12 is notified of parameters related to the data writing process. In the subsequent step SP10, the data output unit 12 is instructed to output video data using the parameters notified in step SP9. Here, this parameter is, for example, a read address on the local data memory 15, a write address on the global data memory 5, or a size.

  Subsequently, the processor 14 moves to step SP11 (FIG. 1) and determines whether or not the transfer of the video data to the local data memory 15 is completed. If a negative result is obtained here, the processor repeats step SP11. If a positive result is obtained in step SP11, the process proceeds to step SP12. As a result, the processor 14 performs image processing on the video data for the number of image processing units stored in the local data memory 15 and then transfers the video data to the global data memory 5.

  When the video data is read out from the global data memory 5 and image-processed for one image processing unit in this way, and the image-processed video data is stored in the global data memory 5, the processor 14 in step SP12, the timer 16 The time measurement value by is acquired, and thereby the elapsed time after starting this processing procedure is detected. In the subsequent step SP13, a certain specified time is added to the elapsed time detected in step SP12, and it is determined whether or not the addition result is equal to or shorter than the processing time for one frame. Here, the specified time is the time from when the processing start instruction signal S7 is received until the timer 16 starts time measurement, and the predicted time that is predicted to be necessary for image processing of video data of one image processing unit. The total time such as the time required to complete the processing procedure when the processing at step SP13 and the negative result at step SP13 are obtained. The default value is applied to the estimated time at the start of the processing of this processing procedure, and the maximum time among the times required for this processing is applied when the processing in units of image processing is repeated. Accordingly, the processor 14 confirms that the elapsed time does not exceed the processing time for one frame even when the subsequent image processing unit is processed.

  As a result, when a positive result is obtained in step SP13, the processor 14 proceeds from step SP13 to step SP14, and the number of blocks of the image processing unit that has completed the processing held as the internal information, and 1 of the image processing unit. By comparing with the number of blocks for a frame, it is determined whether or not all video data for one frame has been processed. If a negative result is obtained here, the process returns from step SP14 to step SP5.

  Thus, the processor 14 repeats the processing procedure of steps SP5-SP6-SP7-SP8-SP9-SP10-SP11-SP12-SP13-SP14-SP5 within the range of processing time for one frame, and stores them in the global data memory 5. The processed video data is sequentially processed for each image processing unit.

  As shown in FIG. 7, the processor 14 sets one line as an image processing unit of video data by this repeated processing, and starts data processing from the center line of the image based on the video data. When the video data of the central line is processed, the processing target is switched to the lower line of the central line, and the video data of the lower line is processed. Subsequently, the video data of the upper line of the center line is processed. By repeating this, the processor 14 alternately processes data from the middle line to the upper and lower lines in the order from the center to the periphery of the image.

  Instead of such line-by-line processing, as shown in FIGS. 8 and 9, for example, a rectangular block of 16 pixels × 16 pixels is set as an image processing unit, and the order from the center of the image toward the periphery is set. Alternatively, data processing may be sequentially performed in units of image processing. Further, in this case, as shown in FIG. 8, the processing target may be switched by spiral scanning in an oblique direction so that the four corners of the image are processed last, and as shown in FIG. The processing target may be switched by spiral scanning in the vertical direction and the horizontal direction so as to process both left and right ends.

  As a result, the processor 14 can process the image processing units sequentially at a sufficiently high speed for, for example, a video that is easy to process, and when the processing of all the image processing units is completed, an affirmative is obtained in step SP14. A result is obtained, and the process moves from step SP14 to step SP15. Here, the processor 14 instructs the timer 16 to stop the operation, and the processing procedure is terminated in the subsequent step SP16.

  On the other hand, when the video data to be processed is a video with a large change that requires time for processing, a positive result is obtained in step SP13 before the processing of all image processing units is completed. In this case, the processor 14 proceeds from step SP13 to step SP15, immediately instructs the timer 16 to stop the operation, and ends this processing procedure in the subsequent step SP16. Accordingly, in this case, when the processing time is about to exceed the time of one frame, the processor 14 stops the processing of the remaining image processing units and ends this processing procedure.

  On the other hand, the received image processing unit 4 and the display image processing unit 7 are configured in the same manner as the image processing unit 6 except that the processing contents in the processor 14 are different. Among these, the received image processing unit 4 is the same as the image processing unit 6, and a series of processes such as white balance adjustment, knee adjustment, gamma adjustment, face detection and the like in units of image processing in the order from the center to the periphery of the image. When the processing time is about to exceed the time of one frame, the processing of the remaining image processing units is stopped for the processing with low priority.

  That is, the received image processing unit 4 is controlled by the processor 14 of the data input unit 13 and the data output unit 12 in the order from the center to the periphery of the image in units of image processing by the rectangular area described above with reference to FIG. The video data held in the global data memory 5 is input to the local data memory 15 to execute processing such as white balance adjustment, knee adjustment, gamma adjustment, and face detection.

  At this time, when the processing time is likely to exceed the time of one frame, the received image processing unit 4 stops the processing with low priority among the series of processing such as white balance adjustment. In this embodiment, the face detection process is applied to the low priority process, whereby the white balance adjustment, knee adjustment, and gamma adjustment with high priority are executed in every image processing unit. Corresponding to this, when the face detection process is interrupted in this way, the controller performs aperture control and focus control according to information necessary for aperture adjustment and focus adjustment based on the area where the face detected so far is captured. The processing is executed, and these controls are executed with reference to the region where the face detected in the center of the screen is photographed.

  As a result, in the received image processing unit 4, the time required to process the unprocessed image processing unit by the processing other than the processing related to the face recognition is applied to the specified time provided for the determination processing in step SP 13. The The time required to process an unprocessed image processing unit by this other processing is detected by monitoring the processing time by the timer 16 to detect the processing time required for face recognition and the processing time required for other processing. Thus, it is obtained from the processing time required for the detected other processing and the number of unprocessed blocks.

  On the other hand, the display image processing unit 7 is controlled by the processor 14 of the data input unit 13 and the data output unit 12 in the same order from the center of the image toward the periphery, in units of image processing by lines shown in FIG. Alternatively, the video data held in the global data memory 5 is input to the local data memory 15 in units of image processing based on the rectangular areas described above with reference to FIGS. 8 and 9, and resolution conversion and gamma adjustment processing are executed.

  In this process, when the processing time is about to exceed the time of one frame, the display image processing unit 7 stops the process with the lower priority in the series of processes. In this embodiment, the gamma adjustment process is applied to the low priority process, whereby the resolution conversion process with a high priority is executed in every image processing unit.

  As a result, the display image processing unit 7 also applies the time required to process the unprocessed image processing unit in the resolution conversion process to the specified time provided for the determination process in step SP13.

(2) Operation of Embodiment In the above configuration, in this video camera 1 (FIG. 1), video data obtained by moving images acquired by the moving image receiving unit 3 during recording is stored in the global data memory 5 and then received. The images are sequentially processed by the image processing unit 4 and the image processing unit 6, and then compressed by the image compression / decompression unit 8 and recorded on the recording medium by the media recording / reproducing unit 10. The video data stored in the global data memory 5 is subjected to resolution and gamma correction by the display image processing unit 7 and is displayed by the moving image display unit 9, thereby being used for a monitor.

  On the other hand, at the time of reproduction, the video data recorded on the recording medium is reproduced by the media recording / reproducing unit 10 and then decompressed by the image compression / decompression unit 8 and stored in the global data memory 5. The resolution and gamma of the video data stored in 5 are corrected by the display image processing unit 7 and displayed by the moving image display unit 9, thereby being provided to the monitor.

  In these series of processing, the video data is subjected to white balance adjustment, knee adjustment, and gamma adjustment by processing by the received image processing unit 4, and a region where a face is photographed by face recognition is detected, and aperture control is performed from this region. Information necessary for focus control is acquired, and processing for aperture control and focus control by control of the lens 2 based on this information is executed. Thus, for example, when a large number of people are included in the imaging result, such as in a group photo shooting site, it takes time to perform processing related to face recognition, and thus the received image processing unit 4 according to video data. The processing time at will change.

  Further, in the image processing unit 6, noise reduction processing is executed, and this noise reduction processing is executed by adaptively switching the processing according to the video data, and in this case, the processing is also performed according to the video data. Time will change.

  In the display image processing unit 7, the resolution is converted by a filtering process whose characteristics are switched according to the video data, and in this case also, the processing time changes according to the video data.

  Even if the worst case takes the longest processing time, the received image processing unit 4, the image processing unit 6, and the display image processing unit 7 are configured so that the video data can be sufficiently processed in a period of one frame. The configurations of the image processing unit 4, the image processing unit 6, and the display image processing unit 7 are redundant. In addition, the configuration becomes complicated, and power consumption increases accordingly.

  In particular, in this embodiment, the received image processing unit 4, the image processing unit 6, and the display image processing unit 7 are constituted by a digital signal processor (FIG. 5), except that the program used for execution of the processor 14 is different. By having the same configuration, the received image processing unit 4, the image processing unit 6, and the display image processing unit 7 are configured such that even in the worst case, the video data can be sufficiently processed in a period of one frame. Then, the configuration of the received image processing unit 4, the image processing unit 6, and the display image processing unit 7 becomes extremely redundant. If such a redundancy is avoided, the same digital signal processor cannot be applied to the received image processing unit 4, the image processing unit 6, and the display image processing unit 7.

  For this reason, in this video camera 1, each image processing unit 4, 6, 7 processes video data for one frame in a period of one frame, and a predetermined image in the order from the center to the periphery of the image. Video data is processed for each processing unit (FIGS. 3 and 4, FIGS. 7 to 9). Further, the video data is sequentially processed in this way, the elapsed time is measured by the timer 16 (FIG. 5), and if the processing is not in time, the processing is terminated halfway.

  As a result, even if the image processing circuits 4, 6, and 7 are not configured so that video data can be sufficiently processed in a period of one frame even in the worst case, each image processing circuit 4, 6, and 7 can process one frame of video. With regard to data, even in the worst case, processing can be completed reliably in a period of one frame, thereby simplifying the configuration and reducing power consumption when video data is processed in real time. Can do.

  Specifically, in the received image processing unit 4 that executes processes such as white balance adjustment, knee adjustment, gamma adjustment, and face recognition, when the process cannot be completed in one frame period, the highest priority among these processes The low face recognition process is aborted. As a result, the video data recorded on the recording medium can be processed in real time without any interruption in the process. On the other hand, in the case of face recognition, only the center part of the screen can be processed in the worst case. As a result, even if the processing is interrupted in this way, various processing in the video camera 1 can be prevented from causing any harm.

  On the other hand, in the image processing unit 6 that executes the noise reduction process, when the process cannot be completed in one frame period, the noise reduction process itself is terminated. However, when time is required for noise reduction processing, an image related to video data is complicated, and noise is not noticeable in such an image. Also, in the center of the screen, which is the main part of the image, there is a feature that noise is not noticeable due to noise reduction. As a result, in this case as well, even if the process is interrupted, it can be prevented from causing any harm.

  On the other hand, in the display image processing unit 7 that corrects the resolution and gamma, when the process cannot be completed in one frame period, the gamma correction process is terminated. However, since the video data to be processed by the display image processing unit 7 is video data to be provided to the monitor, even if the gamma correction processing is interrupted in this way, the video data to be recorded on the recording medium, the recording The video data recorded on the medium can be prevented from being affected. Also, the resolution conversion process can display the monitor image without distortion at the correct angle of view by executing the gamma correction process on the entire screen even if the gamma correction process is interrupted in this way. On the other hand, with gamma correction, even if processing is omitted, there is a feature that the change in image quality is not conspicuous in a moving image, and in addition, an image based on video data that requires processing interruption is a complicated image. As a result, the omission of gamma correction is more inconspicuous. Also in this case, since the gamma correction is performed at the center of the screen, which is the main part of the image, the image quality deterioration due to the omission of processing has a feature that is not noticeable. As a result, in this case as well, the processing can be stopped halfway, so that no harm is caused.

(3) Effects of the embodiment According to the configuration described above, video data is sequentially processed in the order from the center of the screen to the periphery. When real-time processing is performed, the configuration can be simplified and power consumption can be reduced as compared with the conventional case.

  More specifically, when this processing is performed by software data processing using a digital signal processor, and various processing circuits are configured by the same digital signal processor, the redundancy of the digital signal processor is reduced. As a result, the configuration can be simplified and the power consumption can be reduced as compared with the prior art.

  In addition, the data processing unit related to these processes is a rectangular area set on the video data line and the video data image, and the data processing unit is sequentially processed to simplify the configuration compared to the conventional case. , Power consumption can be reduced.

  In the above-described embodiment, the case has been described in which the image processing circuits 4, 6, and 7 sequentially process video data in a fixed image processing unit and processing order. Accordingly, the image processing unit and / or processing order may be switched so that the main part is processed at an early stage and the least useful part is finally processed based on the past processing result. That is, for example, when shooting in landscape mode, and when it can be determined that the landscape is shot from the result of face recognition of the previous frame, the four corners of the screen are relatively unusable parts compared to other parts. In this case, as shown in FIG. 9, it is possible to minimize the influence of switching the processing order of each part and canceling the processing in the middle, thereby efficiently processing video data. it can. On the other hand, when shooting in sports mode, and when it can be determined that a sports scene is shot from the result of face recognition processing of the previous frame, the subject often moves in the horizontal direction, It can be said that the upper and lower ends are relatively less useful parts than the other parts, and in this case, the processing order of each part is switched as shown in FIG. The influence can be reduced as much as possible, whereby the video data can be processed efficiently.

  In the above-described embodiment, the case where the present invention is applied to a video camera and video data for one frame is processed in a period of one frame has been described. When video data is processed in a period of one field, the present invention can be widely applied to processing video data for one field and one frame in a period of a plurality of frames and fields. When processing video data for one field and one frame in a period of multiple frames and multiple fields in this way, for example, acquisition of information necessary for focus control and aperture control processing by the above-described face recognition processing Is intermittently executed every other frame. In this case, video data for one frame is processed in a period of two frames.

  In the above-described embodiments, the case where video data is sequentially processed in units of data processing in the order from the center to the periphery has been described. However, the present invention is not limited to this, and processing in units of data processing is omitted during the processing. It can also be widely applied to. Note that the case where the processing is omitted in this way is, for example, the case where the noise reduction processing is omitted for a portion where character information and graphics information are displayed.

  In the above-described embodiments, the case where the data processing unit is set as the image processing unit by a plurality of pixels and the video data is sequentially processed in the data processing unit has been described. However, the present invention is not limited to this, and the data processing unit is described. Can also be widely applied to the case where video data is sequentially processed by setting the pixel unit.

  In the above-described embodiments, the case where the present invention is applied to a video camera and video data is processed by a digital signal processor has been described. However, the present invention is not limited to this, and when processing by a hardware configuration, Also, the present invention can be widely applied to the case where video data is processed only by software, such as a video content display program in a computer.

  In the above-described embodiments, the case where the present invention is applied to a video camera and video data based on the imaging result is processed has been described. However, the present invention is not limited thereto, and is applied to, for example, a television tuner. When processing video data by broadcast waves, it can be widely applied to DVD recorders and the like when processing video data input from various video devices by applying it to a monitor device.

  The present invention can be applied to, for example, a video camera, a tuner, a DVD (Digital Versatile Disk) recorder, and the like.

It is a flowchart which shows the process sequence of the digital signal processor in the video camera which concerns on the Example of this invention. It is a block diagram which shows the video camera which concerns on the Example of this invention. It is a basic diagram with which it uses for description of the process of the video data in the video camera of FIG. It is a basic diagram with which it uses for description of the process of the video data in the conventional video camera. FIG. 3 is a block diagram illustrating a configuration of an image processing unit in the video camera of FIG. 2. It is a flowchart which shows the process sequence of the digital signal processor in the video camera of FIG. It is a basic diagram which shows the process order at the time of setting an image processing unit to a line. It is a basic diagram which shows the process sequence when an image processing unit is set to a rectangular area. It is a basic diagram which shows the process order by the order different from FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Video camera, 3 ... Moving image receiving part, 4 ... Received image processing part, 5 ... Global data memory, 6 ... Image processing part, 7 ... Display image processing part, 8 ... Image compression / expansion part , 9... Movie recording unit, 10... Media recording / playback unit, 12... Data output unit 13... Data input unit, 14.

Claims (6)

A video data acquisition unit for acquiring video data;
An image processing apparatus having a processing unit that repeats data processing of one field or one frame of the video data with a processing period that is an integral multiple of one field or one frame of the video data;
The processor is
A timer for measuring elapsed time in the processing period;
A data processing unit for processing the video data of one field or one frame,
The data processing unit
The video data is processed for each data processing unit according to the order from the center to the periphery of the image by the video data,
If the data processing of the video data of one field or one frame is not completed within the processing period due to the elapsed time measured by the timer, the data processing of the video data of one field or one frame is performed during the processing period. Censored,
The image processing apparatus, wherein the data processing unit is a line of the video data. The processing unit is a digital signal processor;
The image processing apparatus according to claim 1, wherein the data processing in the processing unit is data processing by software in the digital signal processor.   The image processing apparatus according to claim 1, wherein the data processing in the processing unit is processing in which time required for processing changes according to the video data. In an image processing method of repeating data processing of one field or one frame of video data by a processing period that is an integral multiple of one field or one frame of video data,
A time measurement step of measuring an elapsed time in the processing period;
A data processing step of processing the video data of one field or one frame,
The data processing step includes:
The video data is processed for each data processing unit according to the order from the center to the periphery of the image by the video data,
If the data processing of the video data of one field or one frame is not completed within the processing period due to the elapsed time measured in the time measuring step, the video data of the one field or one frame is processed in the processing period. Censored data processing
An image processing method, wherein the data processing unit is a line of the video data. In a program of an image processing method for repeating data processing of one field or one frame of video data by a processing period that is an integral multiple of one field or one frame of video data by execution by an arithmetic processing means,
A time measurement step of measuring an elapsed time in the processing period;
A data processing step of processing the video data of one field or one frame,
The data processing step includes:
The video data is processed for each data processing unit according to the order from the center to the periphery of the image by the video data,
If the data processing of the video data of one field or one frame is not completed within the processing period due to the elapsed time measured in the time measuring step, the video data of the one field or one frame is processed in the processing period. Censored data processing
A program of an image processing method, wherein the data processing unit is a line of the video data. In a recording medium on which a program of an image processing method is recorded that repeats data processing of one field or one frame of video data by a processing period that is an integral multiple of one field or one frame of video data by execution by arithmetic processing means ,
The image processing method program includes:
A time measurement step of measuring an elapsed time in the processing period;
A data processing step of processing the video data of one field or one frame,
The data processing step includes:
The video data is processed for each data processing unit according to the order from the center to the periphery of the image by the video data,
If the data processing of the video data of one field or one frame is not completed within the processing period due to the elapsed time measured in the time measuring step, the video data of the one field or one frame is processed in the processing period. Censored data processing
A recording medium recording a program of an image processing method, wherein the data processing unit is a line of the video data.

Images