JP4259062B2 - Image processing device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an apparatus for processing a digitized image, and more particularly to an image processing apparatus in which a plurality of elemental functional image processing modules are prepared and various image processing functions are provided by combining them. These image processing functions are used in various systems that handle digital images, such as document editors, drawing tools, image transmission apparatuses, and printers.
[0002]
Such an image processing function is often provided as an image processing library file. In that case, the function is used by linking with an application program that wants to use the image processing function.
[0003]
[Prior art]
In an image processing device, a DTP (desktop publishing) system that can input an image, a print system that can output an image, etc., for example, enlargement / reduction, rotation, affine conversion, color conversion, filter for an image to be processed Various image processing such as synthesis is performed.
[0004]
When performing these processes, if the attributes, process contents, procedures, parameters, etc. of the input image are fixed, the process may be performed using dedicated hardware. However, for example, when various images having different color spaces or the number of bits per pixel are input, or when processing contents, procedures, parameters, and the like are changed variously, a more flexible configuration is required.
[0005]
As means for satisfying such a requirement, for example, as described in JP-A-5-260373 and JP-A-7-105020, a programmable module is arranged in a pipeline form or DAG (Directed Acyclic Graph: directed). An example of performing desired processing flexibly by connecting to a non-circular graph) form is disclosed.
[0006]
In the invention described in Japanese Patent Laid-Open No. 5-260373, the content of each arithmetic processing of a plurality of programmable arithmetic processing units and the connection form of each programmable arithmetic processing unit by a network unit can be freely set from the outside through a host control means. Configure as you can. Accordingly, the present invention aims to provide a digital video signal processing apparatus capable of high-speed and advanced arithmetic processing and having a high degree of freedom for function change and system change.
[0007]
In the invention described in Japanese Patent Laid-Open No. 7-105020, the necessary functional modules are connected / initialized in a pipeline form in a desired order to perform processing. Thus, an invention that performs image processing flexibly has been proposed.
[0008]
When the connection is completed, the header of the last module is requested as necessary. This request sequentially goes back the connected modules to reach the first image input module, and returns header information of the image to be read. Each module rewrites the information of the part to be changed and then passes it to the subsequent module.
[0009]
For example, if the size of the input image is 1000 × 1000 pixels and a reduction processing module to 500 × 500 pixels is included in the processing pipeline, the reduction module uses the size information in the header information passed from the previous stage. Change from 1000x1000 to 500x500 and pass to the subsequent stage. Such processing is sequentially repeated, and finally the last module outputs the header information of the image output by the image processing pipeline to the outside.
[0010]
Next, data processing is performed based on the obtained header information and the like. Similarly to the header processing, the data processing requests the last module to output a certain amount of data. Thereafter, the last module requests the preceding module to input image data necessary for it. This request is similarly transmitted retroactively to the previous stage and is transmitted to the image input module. Then, after the necessary image data is read, it is passed to the subsequent stage, and finally outputted from the last processing module while being sequentially processed.
[0011]
When the above processing is performed up to the end of the image or for a necessary portion, the processing pipeline is unnecessary, and the end module is requested to end. This termination request also reaches the foremost image input module by going back to the module connection in the same manner as the header information acquisition and the processing. Then, the image input module releases the resources used for the processing, releases itself, and returns control to the subsequent stage. Similarly, after the control is returned, the process of releasing the resource and itself is performed and the process is returned to the subsequent stage. All operations are finished when the last module resource / release of the last module is finished.
[0012]
The above-described method described in Japanese Patent Laid-Open No. 7-105020 is applied to the field of image processing or the like in a multi-processing pipeline simulation method using a single processor used in an operating system such as UNIX (registered trademark). Is. Particularly in image processing, by limiting the processing unit for one time to a part of the image, for example, one line of the image, it becomes possible to extremely reduce the memory area that each processing module holds for processing. .
[0013]
As a result, it is possible to provide an inexpensive image processing apparatus that can execute complicated processing with a small amount of memory. In addition, when operating on an operating system that supports virtual memory, swap-out due to memory shortage can be minimized, so that processing can be executed at high speed.
[0014]
[Problems to be solved by the invention]
The image processing module constituting the image processing pipeline is a push type that inputs the processing result of the input data to the image processing module connected downstream of the pipeline, or upstream of the pipeline to acquire the data to be processed. Is a pull type that transmits a data request to an image processing module connected to the. A typical image processing pipeline is composed only of image processing modules of the same type so that external control is minimal and connections between the image processing modules are efficient.
[0015]
However, when the pipeline is configured with a pull-type image processing module, push-type processing in which the user directly supplies image information to the pipeline using other applications, particularly when the user There has been a problem that a pull type module cannot directly execute a process of outputting stored image information to a storage or the like using a module that outputs the image information in a certain image format.
[0016]
In this case, in order to give image information to the pull-type image processing module, it is necessary to hold the desired image information in a memory or the like different from the pipeline constituted by the pull-type image processing module. is there. Therefore, there is a problem that extra memory is consumed and a long processing time is required.
[0017]
The present invention has been made in view of the above-described problems. The pull-type image processing module behaves as a push-type image processing module without any changes to the pull-type image processing module. In particular, it is an object to directly output image information to a pull-type image processing module.
[0018]
[Means for Solving the Problems]
The present invention is an image processing apparatus that includes a plurality of image processing modules that are connected in multiple stages and that each perform predetermined image processing on an image to be processed. Of the image processing modules that are connected in multiple stages, , connected are two image processing modules front respectively, are the image processing module subsequent in the case of a subsequent stage, to the front stage image processing modules, an image request means for performing the image request, the image requesting unit to the processing target image received by an image processing means for performing predetermined image processing, and an image output means for outputting the result of said image processing means has processed preceding the image processing module of the rear stage Image request requesting means for requesting the image processing module to make an image request, and an image to be processed is input to the preceding image processing module Once the image request requesting means may request to perform image request to the subsequent image processing modules, the image request means, an image request to stage the image processing module before in response to the request It is characterized by performing.
[0019]
In addition, the present invention is an image processing apparatus that includes a plurality of image processing modules that are connected in multiple stages and that each perform predetermined image processing on an image to be processed, the image processing modules connected in multiple stages. of out, connected are two image processing modules front respectively, are the image processing module subsequent in the case of a subsequent stage, with respect to the pre-stage image processing module, and an image request means for performing the image request, preceding the image processing module is the image processing module of the rear stage, and an image demand request means for requesting to perform the image request, on the input processing target image, image processing for performing predetermined image processing It means and said image processing means and image output means for outputting to an image processing module of the rear stage results were processed, processed image in front the image processing module Is input, the image request requesting unit requests the subsequent image processing module to perform an image request, and the image processing unit applies the processing target image input in response to the request. Predetermined image processing is performed.
[0031]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment when the present invention is applied to an image processing apparatus will be specifically described.
[0032]
As shown in FIG. 1, the image processing apparatus 101 according to the present invention stores a processing storage unit 102 that stores an image processing command given from the host application 100, an image storage unit 103 that stores image information, and a storage in the image storage unit. The image processing unit 104 performs image processing on the processed image in accordance with an image processing instruction stored in the processing storage unit, and a process execution control unit 105 that controls execution of the image processing.
[0033]
The image processing unit 104 includes at least one image processing module 200 that performs various processes as shown in FIG. Furthermore, at least one of the image processing modules 200 is an image processing module provided with an image request request unit. At least one of the image processing modules 200 is an image processing module including an image requesting unit that requests an image in response to an image request.
[0034]
Further, as shown in FIG. 3, the process execution control unit 105 includes a module generation unit 300 that generates the image processing module 200 corresponding to the image processing command stored in the processing storage unit 102 and connects between the image processing modules; It comprises memory securing means 301 for securing and releasing a memory for developing each image processing module and a buffer used by each image processing module. Each image processing module may have means for securing and releasing a memory as a buffer, but will be omitted in the following description.
[0035]
FIG. 4 is a block diagram illustrating the configuration of a pipeline connecting the middle image processing module 400, the previous image processing module 401, and the subsequent image processing module 402 generated by the module generation unit 300. The middle image processing modules 400, 401, and 402 are all pull-type image processing modules. The middle-stage image processing module 400 includes an input buffer 406 for holding image information acquired from the previous-stage image processing module 401, a control unit 407 that controls the image processing module, and image information that has been processed in the middle-stage image processing module 400. An output buffer 408 for outputting to the subsequent image processing module 402 is configured.
[0036]
When the middle image processing module 400 receives the image request 413 from the subsequent image processing module 402, the request is transmitted to the control unit 407, and the control unit 407 sends an image request 412 to the previous image processing module 401.
[0037]
The pre-stage image processing module 401 that has received the image request 412 writes the processed image information in the output buffer 405, so the control unit 407 copies the image information written in the output buffer 405 to the input buffer 406. The control unit 407 writes the result of processing the image information in the input buffer 406 to the output buffer 408, and is copied to the input buffer 409 by the control unit 410 of the post-stage image processing module 402.
[0038]
FIG. 5 is a block diagram illustrating the configuration of a pipeline connecting the middle image processing module 500, the previous image processing module 501, and the subsequent image processing module 502 generated by the module generation unit 300.
[0039]
The image processing modules 500, 501, and 502 are all push type processing modules. The middle image processing module 500 includes an input buffer 506 for holding the image information input from the previous image processing module 501, a control unit 507 for controlling the image processing module, and image information processed in the middle image processing module 500. Is output from the output buffer 508 for outputting to the subsequent image processing module 502.
[0040]
The middle-stage image processing module 500 receives image information from the previous-stage image processing module 501 to the input buffer 506 and receives an image processing request 512 from the control unit 507.
[0041]
When the middle-stage image processing module 500 obtains image information sufficient for processing by one or more input image information to the input buffer 506, the control unit 507 processes the input image information, and the result is output to the output buffer 508. Write. Then, the control unit 507 inputs the image information in the output buffer 508 to the input buffer 509 of the subsequent image processing module 502 and sends an image processing request 513.
[0042]
As described above, the pull-type image processing module returns a processing result in response to the image request, while the push-type image processing module performs processing on the input processing target image.
[0043]
That is, when the pipeline is configured, the pull-type image processing module receives an image processing request from the subsequent stage, whereas the push-type image processing module receives an image processing request from the previous stage. A pipeline cannot be constructed by directly connecting processing modules.
[0044]
In the present invention, the pull-type image processing module can behave like a push-type image processing module, and in particular, the pull-type image processing module can directly output and output image information.
[0045]
FIG. 6 shows a request image processing module 600 provided with an image request request means for requesting an image request to the pull-type image processing module in the subsequent stage, and a request image processing module in response to the image request. It is a block configuration diagram illustrating the configuration of a pipeline when a pull-type requested image processing module 604 provided with an image requesting unit that requests an image is connected.
[0046]
The requested image processing module 600 further includes an input buffer 601 as an image holding unit for holding image information acquired from an image processing module connected to the previous stage or image information held by a user. And the control part 602 which controls a module is also provided. The control unit 602 includes an image request request unit 612 that makes an image request request to a subsequent module, and an image processing unit 613 that processes an image of the processing target image. In addition, the requested image processing module 600 includes an output buffer 603 as an image output unit for outputting processed image information to the requested image processing module 604.
[0047]
When image information is written in the input buffer 601 (608) and a processing request 609 for the written image information is input to the control unit 602, the control unit 602 receives an image request from the request image processing module 604. Whether image information sufficient for processing is stored in the input buffer 601 is determined. When it is determined that the image is stored, the image request request unit 612 is caused to execute an image request request 610 to the request image processing module 604.
[0048]
The image request unit 613 of the request image processing module 604 that has received the image request request 610 sends an image request 611 to the request image processing module 600. When the requested image processing module 600 receives the image request 611, it processes the image information stored in the input buffer 601. Then, the result is written in the output buffer 603 and copied to the input buffer 605 of the requested image processing module 604.
[0049]
When the amount of image information stored in the input buffer 605 reaches the minimum output unit required for the output image format, the requested image processing module 604 writes the output information into the output buffer 607 after the conversion processing to the output image format, and is the output destination. For example, data is copied to the storage 616.
[0050]
In the above description, for the sake of explanation, each processing module has an input buffer and an output buffer. However, as shown in FIG. 7, a request image processing module 700 or a request image processing module 705 having only an input buffer is used. It may be a simple configuration.
[0051]
When the request image processing module 705 receives the image request, the image request unit 708 makes an image request to the preceding module. Thereafter, the acquired processing target image is subjected to image processing by the image processing output unit 709 in the control unit 707. Then, the image processing output unit 709 directly writes the result into an input buffer such as an image processing module in the subsequent stage. Similarly, the requested image processing module may not have an output buffer.
[0052]
As a result of the image request request from the image request request unit 703, an image request is received from the pull-type request image processing module 705 connected to the subsequent stage. Thereafter, the control unit 702 performs image processing on the image information stored in the input buffer 704 by the image processing output unit 709. Similarly, the image processing output unit 709 directly writes the result in the input buffer of the requested image processing processing module 705.
[0053]
In such a configuration, it is not necessary to copy image information from the output buffer to the input buffer of the requested image processing module 705, and an improvement in processing speed can be expected. Further, since no output buffer is required, memory can be saved.
[0054]
In addition, first-in first-out processing is performed on the image information stored in each input buffer, but a queue function may be used instead of the input buffer. Since a general queuing function includes an image information holding function and a first-in first-out processing function, the use of the queuing function makes it easy to implement an image processing module.
[0055]
In the embodiment of the present invention, an image processing module that requests an image request to a subsequent module is connected to the preceding stage of a pull-type image processing module that requires N lines of image information for one image processing. A general operation in this case will be described.
[0056]
First, the memory securing unit 301 secures a memory necessary for generating an image processing module. The module generating unit 300 generates a requested image processing module having an image request request unit and a requested image processing module having an image request unit, which are included in the image processing unit 104, on the memory secured by the memory securing unit 301.
[0057]
Then, the requested image processing module is connected to the preceding stage of the requested image processing module, and the storage that is the image output destination is connected to the subsequent stage to configure a pipeline. When generating the requested image processing module, calculate the minimum capacity of the input buffer of the requested image processing module from the number of lines of image information required for the requested image processing module to output once. By securing it, you may save memory.
[0058]
FIG. 8 is a diagram showing a general processing flow of the requested image processing module that makes an image request request to the subsequent module.
[0059]
(Step 801) Image information is input to the input buffer of the requested image processing module.
[0060]
(Step 802) It is determined whether image information for N lines is stored in the input buffer. If image information for N lines is stored, the process proceeds to step 803; otherwise, the process proceeds to step 806.
[0061]
(Step 803) The requested image processing module sends an image request to the subsequent stage so that the requested image processing module connected to the latter stage executes the output process.
[0062]
(Step 804) The requested image processing module connected in the subsequent stage needs image information for N lines for one output process. Therefore, the requested image processing module receives N image requests from the subsequent stage. Then, image information is output to the subsequent stage for each image request.
[0063]
(Step 805) A standby state waiting for image information of the next line to be input. When image information is input to the requested image processing module, the process proceeds to step 801.
[0064]
(Step 806) If the image information input to the input buffer is the last line of the image, the process proceeds to Step 807. Otherwise, go to step 805.
[0065]
(Step 807) The requested image processing module sends an image request request to the subsequent stage so that the requested image processing module connected to the subsequent stage executes output processing for the remaining lines of the image.
[0066]
(Step 808) Since the image request for the number of lines remaining in the input buffer is received, the requested image processing module outputs image information to the subsequent stage for each image request. If an image request is received that is less than the number of lines remaining in the input buffer, image information for the requested amount is output to the subsequent stage, and then the image request is sent to the subsequent stage again. The requested image processing module sends an image request request until all the image information for the lines remaining in the input buffer is output to the subsequent stage. As a result, the process of receiving the image request from the subsequent stage and outputting the image information is repeated.
[0067]
【The invention's effect】
As described above, according to the image processing apparatus of the present invention, when the image to be processed is inputted, the image request requesting unit requests the subsequent pull type image processing module to make an image request. To do. Then, the image processing means performs predetermined image processing on the input processing target image in response to the image request from the pull-type image processing module on the rear stage side. Further, the image output means outputs the result to the pull-type image processing module on the rear stage side.
[0068]
The pull-type image processing module at the subsequent stage makes an image request at the previous stage in response to an image request from the previous stage. Then, the processing target image acquired after the request is processed and output.
[0069]
As a result, the pull-type image processing module can behave like a push-type image processing module.
[0070]
Therefore, a pull-type image processing module configured in a pipeline shape can perform image processing as performed by a push-type image processing module.
[0071]
Further, as in the past, when a user uses another application to provide image information to a pull-type image processing module, a memory other than the pipeline configured by the pull-type image processing module, etc. In addition, it is not necessary to hold the image information to be given.
[0072]
Therefore, it is not necessary to consume extra memory, and the processing time is shortened.
[Brief description of the drawings]
FIG. 1 is a block diagram of an embodiment of an image processing apparatus according to the present invention.
FIG. 2 is a block diagram of an embodiment of an image processing unit.
FIG. 3 is a block diagram of an embodiment of a process execution control unit.
FIG. 4 is a configuration example of an image processing pipeline using a pull-type image processing module.
FIG. 5 is a configuration example of an image processing pipeline using a push-type image processing module.
FIG. 6 is a configuration example of a pipeline in which an image processing module including an image request requesting unit is connected to a front stage of a pull type image processing module.
FIG. 7 is a block diagram showing another embodiment of the image processing module.
FIG. 8 is a flowchart illustrating a processing example of an image processing module including an image request request unit.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 100 Host application, 101 Image processing apparatus, 102 Processing storage part, 103 Image storage part, 104 Image processing part, 105 Processing execution control part, 200 Image processing module, 300 Module preparation means, 301 Memory securing means, 400,500 Middle stage image Processing module 401, 501 Pre-stage image processing module, 402, 502 Post-stage image processing module, 600, 700 Request image processing module, 604, 705 Request image processing module, 612, 703 Image request request means, 614, 708 Image request means, 616 storage, 403, 406, 409, 503, 506, 509, 601, 605, 701, 706 input buffer, 404, 407, 410, 504, 507, 510, 602, 606, 702, 707 control unit, 405 408, 411, 505, 508, 511, 603, 607 Output buffer.

Claims (4)

An image processing apparatus that includes a plurality of image processing modules that are connected in multiple stages and that each perform predetermined image processing on a processing target image.
Of the connected image processing module in the multiple stages, connected are two image processing modules preceding each subsequent stage the image processing module in the case of a subsequent stage,
The image processing module of the previous stage, an image request means for performing the image request,
Image processing means for performing predetermined image processing on the processing target image received by the image request means;
Image output means for outputting a result processed by the image processing means;
With
The image processing module in the previous stage is
Respect to the rear stage image processing module, and an image demand request means for requesting to perform the image request,
With
When the processing target image is input to the preceding image processing module, the image request requesting unit requests the subsequent image processing module to make an image request, and the image requesting unit responds to the request. the image processing apparatus characterized by performing image request to the pre-stage image processing module of Te. An image processing apparatus that includes a plurality of image processing modules that are connected in multiple stages and that each perform predetermined image processing on a processing target image.
Of the connected image processing module in the multiple stages, connected are two image processing modules preceding each subsequent stage the image processing module in the case of a subsequent stage,
The image processing module of the previous stage, an image request means for performing the image request,
With
The image processing module in the previous stage is
Respect to the rear stage image processing module, and an image demand request means for requesting to perform the image request,
Image processing means for performing predetermined image processing on the input processing target image;
An image output means for outputting the result of said image processing means has processed the image processing module of the rear stage,
With
When a processing target image is input to the preceding image processing module, the image request requesting unit requests the subsequent image processing module to make an image request, and the image processing unit responds to the request. An image processing apparatus that performs predetermined image processing on the processing target image input in step S1. The apparatus according to claim 1,
The image processing module, in accordance with the processing capability of the image processing module of the rear stage storage capacity required for storing the processed image, the image processing which comprises a storage capacity setting means operable to set apparatus. The apparatus according to claim 1,
The image processing module, wherein the image processing module has a queuing function for storing a processing target image.

Images