JP4877995B2 - Image processing apparatus and electronic apparatus using the same - Google Patents

Description

  The present invention relates to an image processing apparatus that performs predetermined encoding processing and decoding processing on imaging data, and an electronic device using the image processing device.

  In recent years, electronic devices having an imaging function such as a digital camera and a mobile phone with a camera have been widely used. In such an electronic device, when image data is stored, encoding processing such as JPEG (Joint Photographic Coding Experts Group) compression is often performed in order to reduce the data capacity. As a method for realizing such an encoding process, in addition to a software encoding method by a CPU [Central Processing Unit] (see FIG. 3A), a dedicated image processing apparatus (peripheral engine such as a JPEG engine) is used. There is a method to be used (see FIGS. 3B and 3C).

As an example of the related art related to the above, an imaging apparatus disclosed and proposed in Patent Document 1 can be cited.
Japanese Patent Laid-Open No. 2004-128583 (particularly FIG. 1)

  Certainly, with the conventional method shown in FIGS. 3A to 3C, the data capacity can be reduced when image data is stored. Therefore, the capacity of a memory (semiconductor memory or the like) that stores the image data can be reduced. Can be used effectively.

  However, in the conventional method shown in FIG. 3A, a program necessary for the encoding process by the CPU is required, and a large load is applied to the CPU when the captured image data is compressed and stored, resulting in a decrease in processing speed. There was a fear.

  On the other hand, in the conventional method of FIG. 3B, since the peripheral engine is mounted between the camera sensor and the CPU, the existing camera interface optimized for the transfer of imaging data is effectively used. Thus, the compressed image data (JPEG code) can be smoothly transferred and stored in the memory. Therefore, with the conventional method of FIG. 3B, it is possible to realize a reduction in programs and a load on the CPU as compared with the conventional method of FIG. However, the above-described conventional peripheral engine can be used only when the image data is compressed and stored, and the image data compressed and stored in the memory is expanded and reproduced and displayed on a display (LCD [Liquid Crystal Display] or the like). In this case, it is necessary to separately install a peripheral engine for decoding processing or to perform software decoding by the CPU.

  If the conventional method of FIG. 3C is used, it is possible to perform both encoding processing at the time of compression and decoding processing at the time of reproduction display using a single peripheral engine. Since the frequency of data input / output between the system and the peripheral engine increases, it is disadvantageous in terms of performance.

  In view of the above-described problems, the present invention provides an image processing apparatus capable of performing both the encoding process and the decoding process of imaging data without causing an increase in load and a decrease in performance in the arithmetic apparatus, and An object is to provide an electronic device used.

  In order to achieve the above object, an image processing apparatus according to the present invention is an image processing apparatus connected between a camera sensor and an arithmetic device, and directly captures image data input from the camera sensor to the arithmetic device. A first signal path to be output, a second signal path to be subjected to predetermined encoding processing on imaging data input from the camera sensor and output to the arithmetic device, and an encoded processing input from the arithmetic device. And a third signal path that performs predetermined decoding processing on the imaged data and outputs the result to the arithmetic unit (first configuration).

  The image processing apparatus having the first configuration includes a fourth signal path for outputting a camera clock input from the arithmetic unit to the camera sensor, a switch for conducting / cutting off the fourth signal path, It is preferable to have a configuration (second configuration).

  The image processing apparatus having the second configuration may have a configuration (third configuration) including a camera clock adjustment circuit that adjusts the camera clock to a desired frequency.

  In addition, the image processing apparatus having the third configuration includes an image scaler that adjusts the scale of imaging data input via the first and third signal paths (fourth configuration). It is good to.

  Further, an electronic apparatus according to the present invention includes a camera sensor, an arithmetic device, and an image processing device having any one of the first to fourth configurations connected between the camera sensor and the arithmetic device. It is set as the structure to have (5th structure).

  In the electronic apparatus having the fifth configuration, the arithmetic unit may be configured to select the first signal path from the camera sensor when previewing the imaging data input from the camera sensor on the display unit. When the input imaging data is stored in the storage unit, the second signal path is selected. When the imaging data stored in the storage unit is reproduced and displayed on the display unit, the third signal path is selected. As described above, a configuration (sixth configuration) may be employed in which an instruction is sent to the image processing apparatus.

  With the image processing apparatus according to the present invention, it is possible to perform both the encoding process and the decoding process of the imaging data without causing an increase in load or a decrease in performance in the arithmetic device.

  First, a first embodiment of an electronic device according to the present invention (generally, an electronic device having an imaging function such as a digital camera or a mobile phone with a camera) will be described in detail with reference to FIG.

  FIG. 1 is a block diagram showing a first embodiment of an electronic apparatus according to the present invention.

  As shown in the figure, the electronic apparatus of the present embodiment includes an image processing apparatus 100, a camera sensor 200, an arithmetic device 300, a display unit 400, and a storage unit 500.

  The image processing apparatus 100 is connected between the camera sensor 200 and the arithmetic unit 300, and is a peripheral engine that performs predetermined encoding processing and decoding processing on the captured image data (in this embodiment, JPEG compression / decompression processing is performed). JPEG engine). The internal configuration and operation of the image processing apparatus 100 will be described in detail later.

  The camera sensor 200 includes a lens group and an imaging device (CMOS [Complementary Metal Oxide Semiconductor] sensor or CCD [Charge Coupled Devices] sensor), and converts an optical image formed on the imaging surface of the imaging device into an electrical signal. Then, various signal processing (automatic gain adjustment processing, correlated double sampling processing, analog / digital conversion processing, etc.) is performed, and desired imaging data DAT1 is generated and output to the image processing apparatus 100. is there.

  The imaging operation of the camera sensor 200 is on / off controlled based on a control signal CTRL1 directly input from the arithmetic device 300. During the imaging operation of the subject, the arithmetic device 300 to the image processing device 100 In synchronization with the camera clock CLK1 input via the, image data DAT1 of SXGA [Super eXtended Graphics Array] size (1280 pixels × 1024 pixels) is output at a frame rate of 30 [fps], for example. At that time, the camera sensor 200 outputs the horizontal synchronization signal Hs, the vertical synchronization signal Vs, and the camera pixel clock CPCLK necessary for constructing the frame to the image processing apparatus 100 together with the imaging data DAT1.

  The arithmetic device 300 is a CPU (digital baseband processor: DBB) having a camera interface and general-purpose input / output ports (GIO [General Purpose Input / Output]), and includes an image processing device 100, a camera sensor 200, and a display unit 400. And means for comprehensively controlling the operation of each part of the electronic device such as the storage unit 500.

  The display unit 400 is a unit (LCD in the present embodiment) that performs subject preview display (real-time display) and captured image reproduction display based on instructions from the arithmetic device 300.

  The storage unit 500 is a unit (semiconductor memory in the present embodiment) that stores a captured image based on an instruction from the arithmetic device 300.

  Next, the internal configuration of the image processing apparatus 100 will be described in detail.

  As shown in FIG. 2, an image processing apparatus 100 according to the present embodiment includes a camera interface 101 (hereinafter referred to as camera I / F 101), a clip line buffer 102, a JPEG encoder 103, and a serial interface 104 (hereinafter referred to as serial). I / F 105), a code buffer 105, a JPEG decoder 106, a frame control unit 107, an output switch 108, and a switch 109.

  The camera I / F 101 receives the imaging data DAT1, the horizontal synchronization signal Hs, the vertical synchronization signal Vs, and the camera pixel clock CPCLK that are input from the camera sensor 200 during the imaging operation of the subject, and outputs them to the clip line buffer 102 and the output. Means for outputting to the switch 108.

  The clip line buffer 102 is means for temporarily storing the imaging data DAT1 input from the camera sensor 200 for each line and outputting the data to the JPEG encoder 103.

  The JPEG encoder 103 performs predetermined encoding processing (JPEG compression processing) on the imaging data DAT1 input from the clip line buffer 102, generates compressed imaging data DAT2 (JPEG code), and outputs this to the output switch 108. Means.

  The serial I / F 104 is a unit that sequentially receives the control signal CTRL2 (operation mode switching signal) input from the arithmetic unit 300 and the compressed imaging data DAT2 that is the object of decoding processing in sequence. In view of the reduction in the number of substrate wirings and the reduction in the number of IC pins, a serial I / F or a high-speed I2C bus may be used as the transmission path for the compressed imaging data DAT2. However, since the compressed image data DAT2 has a larger data amount than the control signals CTRL1 and CTRL2, it is desirable to use a high-speed interface as much as possible.

  The code buffer 105 is a means for buffering the compressed image data DAT2 (JPEG code) having a minimum data amount necessary for the decoding process (JPEG decompression process) in the JPEG decoder 106.

  The JPEG decoder 106 is a unit that performs predetermined decoding processing (JPEG decompression processing) on the compressed imaging data DAT2 input from the code buffer 105, generates decompressed imaging data DAT3, and outputs this to the output switch 108.

  The frame control unit 107 independently generates the horizontal synchronization signal Hs, the vertical synchronization signal Vs, and the pixel clock PCLK necessary for constructing the frame of the expanded imaging data DAT3 independently of the input signal from the camera sensor 200. This is means for outputting this to the output switch 108. With such a configuration having the frame control unit 107, the decompressed imaging data is not affected even if the operation of the camera sensor 200 is stopped during the decoding process of the compressed imaging data DAT2 input from the arithmetic device 300. It is possible to construct a DAT3 frame.

  The output switch 108 includes a first signal path through which the imaging data DAT1 is directly input from the camera I / F 101, a second signal path through which the compressed imaging data DAT2 is input from the JPEG encoder 103, and decompression imaging from the JPEG decoder 106. Along with imaging data obtained through any one of the third signal paths to which the data DAT3 is input (herein referred to as imaging data DAT), a horizontal synchronization signal Hs and a vertical synchronization signal necessary for the frame construction. This is means for outputting Vs and the pixel clock PCLK to the arithmetic unit 300.

  The switch 109 is means for conducting / interrupting the fourth signal path for outputting the camera clock CLK1 input from the arithmetic unit 300 to the camera sensor 200. In this way, the camera clock CLK1 is not directly input from the arithmetic unit 300 to the camera sensor 200, but is once input to the image processing apparatus 100 and used as a drive clock for the image processing apparatus 100, while the camera for the camera sensor 200 is used. If the configuration is such that the availability of the clock CLK1 can be switched arbitrarily, it is possible to drive only the image processing apparatus 100 without driving the camera sensor 200 in the decoding process of the compressed imaging data DAT2 input from the arithmetic unit 300. Become. In other words, when the compressed image data DAT2 is decoded, the image processing apparatus 100 operates to replace the camera sensor 200, so that the camera sensor 200 does not need to consume power unnecessarily.

  Next, the operation of the image processing apparatus 100 having the above configuration will be described in detail.

  First, the preview mode of the image processing apparatus 100 will be described.

  In this case, the image processing apparatus 100 performs the first signal path without performing any image processing on the imaging data DAT1 input from the camera sensor 200 based on an instruction (control signal CTRL2) from the arithmetic apparatus 300. This is passed through the arithmetic circuit 300 via (camera I / F 101 → output switch 108). Therefore, the display unit 300 can monitor the subject in real time.

  Next, the encoding mode of the image processing apparatus 100 will be described.

  In this case, the image processing apparatus 100 is based on an instruction (control signal CTRL2) from the arithmetic apparatus 300 via the second signal path (camera I / F 101 → clip line buffer 102 → JPEG encoder 103 → output switch 108). The image data DAT1 input from the camera sensor 200 is subjected to a predetermined encoding process (JPEG compression process) to generate compressed image data DAT2 (JPEG code), which is output to the arithmetic circuit 300. Accordingly, since the data capacity can be reduced at the time of saving the imaging data, the capacity of the storage unit 500 can be used effectively.

  Next, the decoding mode of the image processing apparatus 100 will be described.

  In this case, the image processing apparatus 100, based on an instruction (control signal CTRL2) from the arithmetic apparatus 300, via the third signal path (serial I / F 104 → code buffer 105 → JPEG decoder 106 → output switch 108). The compressed image data DAT2 (JPEG code) input from the arithmetic unit 300 is subjected to a predetermined decoding process (JPEG expansion process) to generate expanded image data DAT3, which is output to the arithmetic circuit 300. Accordingly, the display unit 300 can reproduce and display the captured image stored in the storage unit 500.

As described above, the image processing apparatus 100 according to the present embodiment is an image processing apparatus connected between the camera sensor 200 and the arithmetic apparatus 300, and uses the imaging data DAT <b> 1 input from the camera sensor 200 as the arithmetic apparatus 300. in a first signal path for outputting directly (101 → 108), a second signal path for outputting the arithmetic unit 300 in the imaging data DAT1 input from the camera sensor 200 performs a predetermined encoding process (101 → 102 → 103 → 108 ) and a third signal path ( 104 → 105 → 106 → 108 ) for performing a predetermined decoding process on the compressed imaging data DAT2 input from the arithmetic device 300 and outputting the result to the arithmetic device 300 It is set as the composition which consists of.

  With such a configuration, the encoding processing of the imaging data DAT1 input from the camera sensor 200 and the compressed imaging data input from the arithmetic device 300 are performed without causing an increase in load or a decrease in performance in the arithmetic device 300. Both DAT2 decoding processes can be performed.

  In particular, the image processing apparatus 100 of the present embodiment is connected between the camera sensor 200 and the arithmetic device 300, and can effectively use the existing camera interface optimized for transferring the imaging data DAT. In addition to the imaging data DAT1 input from the camera sensor 200 and the compressed imaging data DAT2 obtained by the JPEG encoder 103, the decompressed imaging data DAT3 obtained by the JPEG decoder 106 is also received smoothly with the arithmetic device 300. It becomes possible to pass.

  Next, a second embodiment of the electronic device according to the present invention will be described in detail with reference to FIG.

  FIG. 2 is a block diagram showing a second embodiment of the electronic apparatus according to the present invention.

  As shown in the figure, the electronic apparatus of the present embodiment has substantially the same configuration as that of the first embodiment, and is characterized in that a camera clock adjustment circuit 110 and an image scaler 111 are provided in the image processing apparatus 100. Yes. Therefore, the same parts as those of the first embodiment are denoted by the same reference numerals as those in FIG. 1, and detailed description thereof will be omitted. Hereinafter, only the characteristic parts of the present embodiment will be mainly described. To do.

  The camera clock adjustment circuit 110 is a unit that adjusts (multiplies and divides) the camera clock CLK1 input from the arithmetic unit 300 to a desired frequency, generates the camera clock CLK2, and outputs this to the camera sensor 200. Note that a PLL circuit or the like may be used as the camera clock adjustment circuit 110.

  As described above, if the camera clock adjustment circuit 110 is provided in the image processing apparatus 100, for example, it has only a camera interface of VGA [Video Graphics Array] size (640 pixels × 480 pixels), and is about 12 [MHz]. Even when the arithmetic device 300 that can output only the camera clock CLK1 is used, the camera clock adjustment circuit 110 is used to generate the camera clock CLK2 having a higher frequency (for example, 48 [MHz] multiplied by 4). Since this can be supplied to the camera sensor 200, a camera sensor 200 having a larger number of pixels (for example, a camera sensor having an SXGA size) can be connected.

  The image scaler 111 can input the imaging data DAT1 input via the first signal path and the scale of the expanded imaging data DAT3 input via the third signal path to the camera interface of the arithmetic device 300. It is means for adjusting the size (in the above example, VGA size).

  Thus, if the image scaler 111 is provided in the image processing apparatus 100, the preview mode of the image processing apparatus 100 can be used even when the camera sensor 200 having a number of pixels exceeding the processing capability of the arithmetic apparatus 300 is used. In the decoding mode, the imaging data DAT can be passed to the arithmetic device 300 without any trouble. On the other hand, in the encoding mode of the image processing apparatus 100, the compressed image data DAT 2 is generated without performing any scale adjustment on the image data DAT 1 input from the camera sensor 200, and is passed to the arithmetic device 300. Therefore, the capability of the camera sensor 200 can be fully utilized.

  Note that the scale of the image scaler 111 is adjusted not to a size that can be input to the camera interface of the arithmetic device 300 but to a smaller size suitable for output on the display unit 400 (in the above example, a size equal to or smaller than VGA). It does not matter as a configuration. With such a configuration, the resizing process in the arithmetic device 300 can be omitted.

  The configuration of the present invention can be variously modified within the scope of the present invention in addition to the above embodiment.

  The present invention is a technique that can be widely used for all electronic devices having an imaging function, such as a digital camera and a mobile phone with a camera.

These are block diagrams which show 1st Embodiment of the electronic device which concerns on this invention. These are block diagrams which show 2nd Embodiment of the electronic device which concerns on this invention. These are block diagrams which show the prior art example of the electronic device provided with the imaging function.

Explanation of symbols

100 Image processing device (JPEG engine)
DESCRIPTION OF SYMBOLS 101 Camera interface 102 Clip line buffer 103 JPEG encoder 104 Serial interface 105 Code buffer 106 JPEG decoder 107 Frame control part 108 Output switch 109 Switch 110 Camera clock adjustment circuit (PLL circuit)
111 Image Scaler 200 Camera Sensor 300 Computing Device (DBB)
400 Display (liquid crystal display)
500 Storage unit (semiconductor memory)
DAT1 imaging data DAT2 compressed imaging data (JPEG code)
DAT3 Decompressed imaging data Hs Horizontal synchronization signal Vs Vertical synchronization signal CLK1 Camera clock CLK2 Camera clock CPCLK Camera pixel clock PCLK Pixel clock CTRL1 Control signal CTRL2 Control signal

Claims (6)

An image processing device connected between a camera sensor and a computing device,
The image processing device includes a first signal path that directly outputs imaging data input from the camera sensor to the arithmetic device, and a predetermined encoding process performed on the imaging data input from the camera sensor. A second signal path that outputs to the computing device, and a third signal path that performs a predetermined decoding process on the encoded imaging data input from the computing device and outputs the processed data to the computing device ,
The first signal path and the second signal path each include a first camera interface that receives image data input from the camera sensor in common.
Image data transmitted through the image processing apparatus via the first to third signal paths is output to the second camera interface provided in the arithmetic unit. Processing equipment.   2. The apparatus according to claim 1, further comprising: a fourth signal path that outputs a camera clock input from the arithmetic unit to the camera sensor; and a switch that conducts / cuts off the fourth signal path. The image processing apparatus described.   The image processing apparatus according to claim 2, further comprising a camera clock adjustment circuit that adjusts the camera clock to a desired frequency.   The image processing apparatus according to claim 3, further comprising an image scaler that adjusts a scale of imaging data input via the first and third signal paths.   It has a camera sensor, a calculating device, and the image processing apparatus in any one of Claims 1-4 connected between the said camera sensor and the said calculating device, It is characterized by the above-mentioned. Electronics.   When the image data input from the camera sensor is previewed on the display unit and the image data input from the camera sensor is stored in the storage unit so as to select the first signal path when previewing the image data input from the camera sensor on the display unit, When the image data stored in the storage unit is reproduced and displayed on the display unit so as to select the second signal path, the image processing apparatus is instructed to select the third signal path. The electronic device according to claim 5, wherein the electronic device is sent.

Images