JP4882954B2 - Imaging apparatus, information synthesis apparatus, and program - Google Patents

Description

  The present invention relates to an imaging device, an information synthesis device, and a program for synthesizing information on an image.

  In the following, Bayer array image data is referred to as Bayer data, YUV format image data is referred to as YUV data, and JPEG (Joint Photographic Experts Group) format image data is referred to as JPEG data.

Conventionally, there is an image processing technique that uses a part of a memory as a ring buffer and performs enlargement / reduction in accordance with, for example, conversion of Bayer data into YUV data. (For example, Patent Document 1)
In general, in an image processing system mounted on a digital camera, after converting Bayer data obtained from an image sensor by photographing into YUV data, the data is further converted into JPEG data suitable for recording, and the data amount is compressed, and then a recording medium It was supposed to be recorded on a memory card.

  For example, when it is desired to superimpose a character image such as date information on the image to be recorded, the character image is synthesized at the Bayer data stage and then transferred to the ring buffer.

However, if a character image is superimposed at the Bayer data stage, when converting to YUV data, fine lines disappear when the data is reduced, or some lines become thick when the data is enlarged. There is a problem in that the character image portion is deformed with reduction / enlargement and a correct character image cannot be obtained.
JP 2003-018610 A

  In order to avoid the above problems, the use of the memory as a ring buffer is temporarily stopped when combining character images, and the YUV data after reduction / enlargement for one frame is developed on the memory. Then, after superimposing (partially overwriting) a character image in software, it is conceivable to adopt a method of converting the YUV data into JPEG data.

  However, in such a case, although there is no image deterioration as described above, the use area of the memory is greatly increased, and another problem that a lot of time is required for conversion of the image data occurs, which is not realistic.

  The present invention has been made in view of the above circumstances, and the object of the present invention is to synthesize information into image data with high quality without unnecessarily expanding the use area of a memory for performing image processing. It is an object to provide an imaging apparatus, an information synthesis apparatus, and a program that can be executed.

  According to the first aspect of the present invention, a photographing means for photographing the subject to acquire the whole image data, a first storage means for storing the whole image data, and a predetermined area division unit from the first storage means. First image processing means for obtaining partial image data by subjecting the transferred image data to first image processing, and a second storage for holding partial image data obtained by the first image processing means Means for synthesizing information data at a predetermined location in the partial image data held by the second storage means, and a second for the partial image data transferred from the second storage means. A second image processing unit that performs image processing; a transfer control unit that controls a transfer operation of partial image data from the first storage unit to the second image processing unit; and When performing synthesis Characterized in that it comprises a stop control means for stopping temporarily the transfer operation of the partial image data by the transfer control means.

  According to a second aspect of the invention, there is provided the third aspect of the invention according to the first aspect, further comprising third storage means for holding partial image data obtained by the second image processing means, wherein the transfer control means is the first control means. The transfer operation of the partial image data from the storage means to the third storage means is controlled.

  According to a third aspect of the invention, in the first aspect of the invention, the first image processing means expands or reduces the image data transferred from the first storage means in a predetermined area division unit. Partial image data is obtained by performing first image processing including reduction processing.

  According to a fourth aspect of the present invention, in the first aspect of the invention, the first image processing unit is configured to store an image of image data transferred from the first storage unit in a predetermined area division unit. Partial image data is obtained by performing first image processing including color conversion processing.

  According to a fifth aspect of the present invention, in the first aspect of the invention, the second image processing means includes a second processing that includes compression processing for the partial image data transferred from the second storage means. The image processing is performed.

  According to a sixth aspect of the invention, in the first aspect of the invention, the area division unit is obtained by dividing the entire image data by a predetermined number of lines.

  According to a seventh aspect of the invention, in the first aspect of the invention, the transfer control means adjusts a transfer operation in units of the area division.

  The invention according to claim 8 is the invention according to claim 1, wherein the synthesizing means performs the synthesis for each partial image data.

  According to a ninth aspect of the present invention, in the first aspect of the present invention, the stop control means further comprises a determination means for determining the position and size of the information data to be synthesized in the image data synthesized by the synthesis means. Is characterized in that the transfer operation of the partial image data is controlled to stop according to the determination result of the determination means.

  According to a tenth aspect of the present invention, in the first aspect of the invention, the synthesizing unit performs the synthesizing for a predetermined number of partial image data.

  According to an eleventh aspect of the present invention, in the first aspect of the present invention, the stop control means may be configured such that the first image processing means performs the first image processing before obtaining the partial image data. The timing for stopping the data transfer operation is determined.

  The invention according to claim 12 is the invention according to claim 1, further comprising a judging means for judging whether or not the partial image data is partial image data to be synthesized with the information data, and the stop The control means controls to stop the transfer operation of the partial image data in accordance with the determination result of the determination means.

  A thirteenth aspect of the present invention is an information synthesizing device mounted on an image processing apparatus comprising a first storage means for storing the entire image data and a second storage means, from the first storage means. The first image processing is performed on the image data transferred in a predetermined area division unit to obtain partial image data, and the partial image data is transferred to the second storage means. First image processing means, composition means for synthesizing information data at a predetermined location in the partial image data held by the second storage means, and the partial image data transferred from the second storage means Second image processing means for performing second image processing on the image, transfer control means for controlling the transfer operation of partial image data from the first storage means to the second image processing means, and the composition Means of the information data When performing deposition, characterized by comprising a stop control means for temporarily stopping the transfer operation of the partial image data by the transfer control means.

  A fourteenth aspect of the present invention is an information synthesizing device mounted on an image processing apparatus comprising a first storage means for storing whole image data and a second storage means, from the first storage means. The first image processing is performed on the image data transferred in a predetermined area division unit to obtain partial image data, and the partial image data is transferred to the second storage means. First image processing means, composition means for synthesizing information data at a predetermined location in the partial image data held by the second storage means, and the partial image data transferred from the second storage means Transfer of partial image data from the first storage means to the second image processing means to an information synthesizing apparatus comprising a second image processing means for performing second image processing on Transfer control to control operation And the step, when the combining means performs a synthesis of the information data, characterized in that to realize a stop control means for temporarily stopping the transfer operation of the partial image data by the transfer control means.

  According to the present invention, it is possible to synthesize information in image data with high quality without unnecessarily widening a use area of a memory for performing image processing.

An embodiment in which the present invention is applied to a digital camera will be described below with reference to the drawings.
(Constitution)
FIG. 1 shows a main circuit configuration of the entire embodiment. Reference numeral 1 denotes a CPU as a system controller that controls the operation of the entire camera. An image processing IC 2 is connected to the CPU 1 via a control bus CB and a data bus DB.

  The image processing IC 2 performs all processes related to image data from image capturing to recording, which will be described later. The image processing IC 2 is connected to the main memory RAM 3 and the ROM 5 storing program data and fixed data via the data bus DB.

  Further, the image processing IC 2 is directly connected to a CCD (Charge Coupled Device) 4, the ROM 5, an LCD (Liquid Crystal Display) 6, a key input unit 7, and a memory card interface 8.

  The CCD 4 is an image sensor of this camera, and has a color filter with a primary color Bayer array. The LCD 6 is a color liquid crystal display panel with a backlight provided on the back side of the camera, and displays a monitor image, a reproduced image, and the like.

  The key input unit 7 includes a power key, a shutter key, a mode key, a cursor key, a set key, and the like, and outputs a key operation signal corresponding to a user operation to the image processing IC 2.

The memory card interface 8 is a recording medium of the camera, and writes image data to a memory card (not shown) that is detachably attached to the camera at the time of shooting, and stores image data from the memory card at the time of playback. Read.
Necessary power is supplied from the power supply unit 9 to each circuit.

FIG. 2 is a block diagram showing a more detailed functional circuit configuration in the image processing IC 2.
In the figure, reference numeral 11 denotes a DMA (Direct Memory Access) controller (referred to as “DMAC” in the figure).

The DMA controller 11 takes in image data from the CCD 4 via the CCD control unit 13 and writes it in the RAM 3 or reads out the image data written in the RAM 3 and writes it again in the RAM 3 after various processing. Data input / output control at the time of
The memory control unit 12 transmits / receives data to / from the RAM 3, ROM 5, and memory card interface 8 under the control of the DMA controller 11.

Further, an image data transfer unit 17, a JPEG conversion unit 16, an image composition unit 18, and an LCD control unit 19 are connected to the DMA controller 11.
The image data transfer unit 17 performs data transfer control in the image processing unit 14, the pixel number conversion unit 15, the JPEG conversion unit 16, and the image data transfer unit 17.

  The image processing unit 14 converts the Bayer data into YUV data and outputs the YUV data to the pixel number conversion unit 15. The pixel number conversion unit 15 performs enlargement processing or reduction processing of YUV data, and outputs the processed data to the image data transfer unit 17.

  The JPEG conversion unit 16 converts the YUV data into JPEG data by a predetermined data compression process, and outputs the JPEG data to the DMA controller 11. The image synthesis unit 18 synthesizes a character image or the like in the image data and outputs the synthesis result to the DMA controller 11.

  The LCD control unit 19 controls the LCD 6 to control display of images as well as monitor images during photographing. The key input control unit 20 receives a key operation signal input from the key input unit 7 and outputs corresponding key code information to the CPU 1.

A rough flow of image processing at the time of shooting when the recording size of designated image data is small and the capacity of the RAM 3 is sufficient will be described below.
When the photographing is instructed, the CCD control unit 13 takes in image data (Bayer data) from the CCD 4 and stores it in the RAM 3. The Bayer data stored in the RAM 3 is read by the DMA controller 11, converted into YUV data by the image processing unit 14 via the image data transfer unit 17, and output to the pixel number conversion unit 15.

  The pixel number conversion unit 15 converts the YUV data from the image processing unit 14 into a desired pixel number size and outputs it to the image data transfer unit 17. The image data transfer unit 17 stores the YUV data in the RAM 3. .

  Next, the JPEG conversion unit 16 reads the YUV data from the RAM 3, converts it into JPEG data, and stores it again in the RAM 3.

  When image synthesis of a character image or the like is necessary, after the image composition unit 18 performs necessary image composition processing at the stage of YUV data, the JPEG conversion unit 16 converts it into JPEG data.

(First operation example)
Hereinafter, Bayer data of 64 pixels wide × 48 pixels vertical (line) shown in FIG. 3A is obtained by photographing using the CCD 4, and this is doubled both vertically and horizontally as shown in FIG. 3B. A description will be given of a first operation in the case where the YUV data of 128 pixels × 96 pixels (lines) is converted and enlarged and then converted into JPEG data of the same size and recorded on a memory card (not shown) from the memory card interface 8.

  In this case, if necessary, in the image shown in FIG. 3B, the upper left end point of the pixel is the origin coordinate (0, 0) and the position of the coordinates (80, 72) is used as a reference, and an arbitrary rectangular area, for example, horizontal 48 pixels × It is assumed that character image data CI having a size of 16 pixels (lines) in length is synthesized, and the synthesized YUV data is converted into JPEG data.

  The character image data CI may be date (date) information such as “2007.4.1” as shown in the figure, and further, photographing data such as time, shutter speed and aperture (F) value, and the like. Character image data may be used, and various image information other than characters such as symbols and symbols can be considered.

  In the ring buffer operation described later, since the JPEG conversion process basically performs data processing in units of 16 lines, this is referred to as “one belt” as one area division unit.

  Therefore, in the Bayer data before being doubled both vertically and horizontally, 8 lines are read as 1 belt so as to correspond, and transferred to the ring buffer.

  FIG. 4 is a flowchart for performing a presetting process for each parameter, which is executed before the image data transfer unit 17 executes image processing to be described later at the time of image shooting. Is mainly performed by the DMAC 11 in the image processing IC 2.

  At the beginning, the image size ((horizontal) 64 (pixels), (vertical) 48 (pixels)) (hereinafter referred to as (64, 48) in the coordinate data format) taken into the DMAC 11 by the CCD controller 13. (Notation) Bayer data is stored in the address “0x000000000000” of the RAM 3 (step S101).

  Next, the DMAC 11 sets conversion processing from Bayer data to YUV data in the image processing unit 14 (step S102). In addition, the DMAC 11 sets the pixel number conversion unit 15 to perform the enlargement process by “2” (step S103).

  Further, the number of belts “6” and the number of belt lines “16” are determined from the input image size (64, 48) and the output image size (128, 96) (step S104).

The image data transfer unit 17 includes the contents determined above.
Input image address “0x000000000000”,
Input image size (64, 48),
Output image address “0x100000000”,
Output image size (128,96),
Number of belts “6”,
Number of input belt lines “8”, and
Number of output belt lines “16”
Is set (step S105).

At the same time, the JPEG conversion unit 16 also
Input image address “0x100000000”,
Output image address “0x200000000”,
Image size (128,96),
Number of belts “6”, and
Number of beltlines "16"
Is set (step S106).

  Further, the position “5, 6” of the stop belt corresponding to the synthesis position is determined from the reference coordinates (80, 72) and the synthesis size (48, 16) of the character image data to be synthesized (step S107).

According to this determination result, the image composition unit 18 sets the composition data.
Coordinates (80, 72) of the character image composition position;
Composite size (48, 16), and
Number of stop belts “2”
Is set (step S108).
Next, the position “5, 6” of the stop belt is also set in the image data transfer unit 17 (step S109).

  As described above, the operations of the above circuits are started on the assumption that necessary presetting has been completed for the image processing unit 14, the pixel number conversion unit 15, the image composition unit 18, the JPEG conversion unit 16, and the image data transfer unit 17. The setting is performed (step S110), and the processing in FIG. 4 is completed as described above, and the operation proceeds to the recording of the image accompanying the shooting.

  FIG. 5 shows the processing contents executed in parallel by the image data transfer unit 17, JPEG conversion unit 16, and image composition unit 18. First, the Bayer data for one belt (8 lines) at the belt position “1” is transferred from the Bayer data area of the RAM 3 in smaller data units to the image processing unit 14 and converted to 16 lines of YUV data as set above. After the conversion, the data is transferred to the pixel number conversion unit 15 to execute the process of enlarging the number of pixels at a magnification of “2” as set above, and control to transfer to the ring buffer area of the RAM 3 (step S201).

  Thereafter, the image data transfer unit 17 determines whether or not the image data processed by the current image processing unit 14 and the pixel number conversion unit 15 is for a preset stop belt (step S202).

  If it is determined that the belt position “1” is not the set stop belt, the transfer end line position “1” and the processing end signal are issued to the JPEG conversion unit 16 as they are (step S205), and then the currently transferred image data. After confirming that the belt position is not the final belt of the Bayer data (step S206), the process returns to the process from step S201.

The JPEG conversion unit 16 waits to receive a transfer end line position and a processing end signal from the image data transfer unit 17 (step S301).
When the processing end signal is received from the image data transfer unit 17, this is determined in step S301, and the YUV data for 16 lines at the belt position “1” held in the ring buffer is converted into JPEG data in smaller data units. The data is transferred to the unit 16 and converted into JPEG data, and the converted JPEG data is written in the corresponding area in the JPEG data area of the RAM 3 (step S302).

  Thereafter, after confirming that the converted data is not the final belt (step S303), the JPEG conversion unit 16 returns to the processing from step S301 and receives a processing end signal from the image data transfer unit 17. Wait for processing of data for the next belt.

  Thus, by repeatedly executing the processing in the image data transfer unit 17, the image processing unit 14, the pixel number conversion unit 15, and the JPEG conversion unit 16, the ring buffer of the RAM 3 is used at the belt position where there is no character image data to be synthesized. The processing is executed for each belt.

  FIG. 7 shows an example of data area division in the RAM 3 (when the ring buffer is used and when it is not used).

  In FIG. 7A, a Bayer data storage area, a YUV data storage area, a JPEG data storage area, and a display data storage area are provided. Conversion from Bayer data → YUV data → JPEG data was sequentially performed for each “1 belt”, which is one area division unit, and three areas were used simultaneously. That is, all the areas described above are areas for storing data for one image.

  In contrast, in FIG. 7B, a ring buffer area is provided instead of the YUV data storage area. In the same manner as described above, when data conversion is sequentially performed for each "1 belt" which is a unit of area division, the YUV area is generally YUV data area because data generally passes only for each area division unit. Even if prepared, there were many unused areas. In other words, by replacing this with a ring buffer area, it is possible to perform the same process in a smaller area, and as a whole, the capacity of the RAM 3 can be used more effectively. Yes.

  In the above description, the process of conversion from YUV data to JPEG data has been disclosed, but the process flow is the same in the conversion from YUV data to display data.

  The display data referred to here is a thumbnail image.

  Incidentally, when inputting image data into the Bayer data area, when outputting an image from the JPEG data area, and when outputting an image from the display data area, processing is performed for each "1 belt" that is a unit of area division. Instead, data processing is performed for each image.

  This will be described in more detail below.

  FIG. 6 exemplifies a captured image that is enlarged from Bayer data to YUV data and JPEG data when there is no character image data to be synthesized. In the figure, for the sake of easy understanding, a solid rectangular frame is added to the image data. However, in actual data conversion, such a solid rectangular frame does not exist.

  FIG. 6A illustrates the converted image when the entire image is converted from Bayer data to YUV data without using a ring buffer in the RAM 3, and further converted from YUV data to JPEG data for reference. It is.

  When processing is performed without providing a ring buffer in the RAM 3 as described above, as shown in FIG. 7A, when shooting an image, the RAM 3 displays the Bayer data, YUV data, JPEG data, and the LCD 6 obtained by shooting. Display data area is required, and a larger storage capacity of the RAM 3 is required.

  On the other hand, FIG. 6B illustrates the processing order of image data in the ring buffer when a ring buffer having a capacity of two belts is provided in the RAM 3 as shown in FIG. 7B. In the ring buffer, the enlarged YUV data is processed for each belt as shown in FIGS. (B-1) to (B-5) and converted into JPEG data. JPEG data for one frame can be acquired at the time when the process is finished.

  Therefore, as shown in FIG. 7B, the RAM 3 can be configured with a smaller storage capacity by the difference between the YUV data for one frame and the ring buffer capacity for two belts.

Next, the processing in FIG. 5 in the case of combining character images will be described.
The Bayer data shown in FIG. 9A is converted into YUV data on the ring buffer, and the date character image data is synthesized at the positions of the 5 and 6 belts shown in FIG. An image as shown in B) is obtained and converted into JPEG data.

  As shown in FIGS. 10 (1) to (3), the processes up to the fourth belt are converted to YUV data and further converted to JPEG data are the same as in the case of FIG.

  Thereafter, when the image data transfer unit 17 transfers Bayer data of the fifth belt to the ring buffer in Step S301, it is determined in subsequent Step S302 that the Bayer data of this belt is set as a stop belt.

  Having determined this, the image data transfer unit 17 issues a process stop signal to the image composition unit 18 (step S203), and then determines whether or not an image composition processing end signal has been sent from the image composition unit 18. By repeatedly judging (step S204), it waits for the signal to be sent.

  However, the image composition unit 18 waits for the signal to be sent by repeatedly determining whether or not a processing stop signal has been sent from the image data transfer unit 17 at the beginning (step S401). When it is determined that the YUV data has been sent, YUV data for one belt (here, the fifth belt) from the ring buffer area of the RAM 3 is transferred to the image composition unit 18 in smaller data units, and the YUV data The corresponding character image data is synthesized to generate a synthesized image (YUV data), and the synthesized image data is transferred again to the ring buffer area of the RAM 3 (step S402).

  Here, the data transferred from the ring buffer area to the image compositing unit 18 may be data only for a portion where the compositing process is performed. As a result, the processing speed can be further improved.

  After completion of the composition processing, the image composition unit 18 issues an image composition processing end signal to the image data transfer unit 17 (step S403), and then confirms that the belt position is not the final belt (step S404). Then, the process returns to step S401 again, and then the image composition process is waited for.

  The image data transfer unit 17 that has received the image synthesis process end signal from the image synthesis unit 18 determines this in step S204, and again sends the transfer end line position “5” and the process end signal to the JPEG conversion unit 16. After the issuance (step S205), it is confirmed that the belt position of the currently transferred image data is not the final belt of the Bayer data (step S206), and the process returns to the process from step S201.

  When the JPEG conversion unit 16 receives the transfer end line position and the processing end signal from the image data transfer unit 17, the JPEG conversion unit 16 determines this in step S301, and the 16 lines of the belt position “5” after image synthesis held in the ring buffer. The corresponding YUV data is converted into JPEG data and compressed, and the converted JPEG data is written in the corresponding area of the RAM 3 (step S302).

  Thereafter, after confirming that the converted data is not the final belt (step S303), the JPEG conversion unit 16 returns to the processing from step S301 and receives a processing end signal from the image data transfer unit 17. Wait for processing of data for the next belt.

  In the ring buffer, in the ring buffer, as shown in FIG. 10 (4), after the enlarged YUV data of the fifth belt is written by the pixel number conversion unit 15, it is determined that this belt is a stop belt. When the image composition unit 18 performs the image composition processing of the character image as determined by 17, the content held as shown in FIG. 10 (5) is obtained.

  JPEG data is generated from the YUV data after the image synthesis by the JPEG conversion unit 16 and held in the RAM 3 as a part of the JPEG data shown in FIG.

  Thereafter, when the image data transfer unit 17 transfers the Bayer data of the sixth belt to the ring buffer in Step S301, it is determined in subsequent Step S302 that the Bayer data of this belt is set as a stop belt.

  Having determined this, the image data transfer unit 17 issues a process stop signal to the image composition unit 18 (step S203), and then determines whether or not an image composition processing end signal has been sent from the image composition unit 18. By repeatedly judging (step S204), it waits for the signal to be sent.

  The image synthesizing unit 18 waits for a processing stop signal to be sent from the image data transfer unit 17 (step S401), and when it is determined that it has been sent, it corresponds to one belt (here, the sixth belt). The corresponding character image data is combined with the YUV data (step S402).

  After completion of the composition processing, the image composition unit 18 issues an image composition processing end signal to the image data transfer unit 17 (step S403), and then determines that the belt position is the final belt (step S404). Thus, the processing is terminated assuming that the image composition processing for the photographed image has been completed.

  The image data transfer unit 17 that has received the image synthesis process end signal from the image composition unit 18 determines this in step S204, and again sends the transfer end line position “6” and the process end signal to the JPEG conversion unit 16. After the issuance (step S205), it is determined that the belt position of the currently transferred image data is the last belt of the Bayer data (step S206), and the processing related to the transfer of the image with respect to the photographed image is also completed. To finish the process.

  FIG. 8 illustrates the processing in the image data transfer unit 17 at this time. If the 6th belt YUV data is not a stop belt, the YUV data for the 6th belt is actually stopped at the timing when a processing end signal is issued to the JPEG conversion unit 16 as shown in FIG. 8 (2). Since the belt is a belt, the processing end signal is temporarily stopped without being issued, and then the processing end signal is issued after receiving the end signal from the image composition unit 18.

  When the JPEG conversion unit 16 receives the transfer end line position and the processing end signal from the image data transfer unit 17, the JPEG conversion unit 16 determines this in step S301, and the 16 lines of the belt position “6” after image synthesis held in the ring buffer. The corresponding YUV data is converted into JPEG data and compressed, and the converted JPEG data is written in the corresponding area of the RAM 3 (step S302).

  Thereafter, the JPEG conversion unit 16 determines that the converted data is the final belt (step S303), and ends the processing assuming that the conversion processing of the photographed image into JPEG data is completed.

  In the ring buffer during the above process, after the enlarged YUV data of the sixth belt is written by the pixel number conversion unit 15 as shown in FIG. 10 (6), it is determined that this belt is a stop belt. The content held as shown in FIG. 10 (7) is obtained by the image composition unit 18 performing the image composition processing of the character image determined by the unit 17.

  JPEG data is generated from the YUV data after the image synthesis by the JPEG conversion unit 16 and held in the RAM 3 as a part of the JPEG data shown in FIG.

  Thereafter, the image data transfer unit 17, the image composition unit 18, and the JPEG conversion unit 16 recognize that the JPEG conversion of the final belt has been completed, and all the processes in FIG. 5 are completed, and are again stored in the RAM 3. One frame of JPEG data is recorded on a memory card as a recording medium via the memory card interface 8.

  As described above, in this operation example, it is possible to perform high-quality and high-speed synthesis of character image data in image data without unnecessarily widening the use area of the RAM 3.

  In the above-described embodiment, the image processing unit 14 and the pixel number conversion unit 15 enlarge the image size when converting Bayer data into YUV data. However, it is possible to perform the same main process by setting the contents in the pre-setting process shown in FIG.

  In this case, in the process of converting Bayer data to YUV data, enlargement or reduction of the image size is performed at the same time, and then the character image is combined with the YUV data so that the character image is deformed. Can be avoided reliably.

  This is not limited to the enlargement or reduction of the image size, and the same applies when various color processing is performed on the color image data, and a part of the character image has a different color from the other parts. Problems can be avoided reliably.

  In the above operation example, the transfer operation in the image data transfer unit 17 is adjusted in units of a predetermined number of lines, for example, 8 lines or 16 lines, together with the basic pixel block handled by the JPEG conversion unit 16.

  As a result, the processing of the JPEG conversion unit 16 can be executed immediately after character image synthesis, and the control operation related to image transfer in the image processing IC 2 is simplified without being affected by the size of the character image to be synthesized. be able to.

(Second operation example)
Hereinafter, Bayer data of 64 pixels wide × 48 pixels vertical (line) shown in FIG. 3A is obtained by photographing using the CCD 4, and this is doubled both vertically and horizontally as shown in FIG. 3B. A second operation in the case of converting and expanding the YUV data of 128 pixels × vertical 96 pixels (lines) to JPEG data of the same size and recording it on a memory card (not shown) from the memory card interface 8 will be described.

  In this case, if necessary, in the image shown in FIG. 3B, the upper left end point of the pixel is the origin coordinate (0, 0) and the position of the coordinates (80, 72) is used as a reference, and an arbitrary rectangular area, for example, horizontal 48 pixels × It is assumed that character image data CI having a size of 16 pixels (lines) in length is synthesized, and the synthesized YUV data is converted into JPEG data.

  The character image data CI may be date (date) information such as “2007.4.1” as shown in the figure, and further, photographing data such as time, shutter speed and aperture (F) value, and the like. Character image data may be used, and various image information other than characters such as symbols and symbols can be considered.

  In the ring buffer operation described later, since the JPEG conversion process basically performs data processing in units of 16 lines, this is referred to as “one belt” as one area division unit.

  Accordingly, in the Bayer data before being doubled both vertically and horizontally, half of the 8 lines are read as one belt correspondingly and transferred to the ring buffer.

  Note that the pre-setting process of each parameter, which is executed before the image data transfer unit 17 executes image processing to be described later at the time of image shooting, is the same as the contents executed in FIG. 3 in the first operation example. It is basically the same, and illustration and description thereof are omitted.

  Thus, it is assumed that necessary pre-settings for the image processing unit 14, the pixel number conversion unit 15, the image composition unit 18, the JPEG conversion unit 16, and the image data transfer unit 17 have been completed. Move to this operation.

  FIG. 11 shows the processing contents executed in parallel by the image data transfer unit 17, JPEG conversion unit 16, and image composition unit 18. First, the Bayer data for one belt (8 lines) at the belt position “1” is transferred from the Bayer data area of the RAM 3 in smaller data units to the image processing unit 14 and converted to 16 lines of YUV data as set above. After the conversion, the data is transferred to the pixel number conversion unit 15 to execute the process of enlarging the number of pixels at a magnification of “2” as set above, and control to transfer to the ring buffer area of the RAM 3 (step S501).

  Thereafter, the image data transfer unit 17 determines whether the image data processed by the current image processing unit 14 and the pixel number conversion unit 15 is within a preset stop belt range (step S502).

  If it is determined that the belt position “1” is not within the set stop belt range, the transfer end line position “1” and the processing end signal are issued to the JPEG conversion unit 16 as they are (step S506), and the current transfer is performed. After confirming that the belt position of the image data is not the final belt of the Bayer data (step S507), the processing returns to step S501.

The JPEG conversion unit 16 waits to receive a transfer end line position and a processing end signal from the image data transfer unit 17 (step S601).
Accordingly, when the processing end signal is received from the image data transfer unit 17, this is determined in step S601, and the 16-line YUV data at the belt position “1” held in the ring buffer is JPEG converted in finer data units. The data is transferred to the unit 16 and converted into JPEG data, and the converted JPEG data is written into the corresponding area in the JPEG data area of the RAM 3 (step S602).

  Thereafter, after confirming that the converted data is not the final belt (step S603), the JPEG conversion unit 16 returns to the processing from step S601 and receives a processing end signal from the image data transfer unit 17. Wait for processing of data for the next belt.

  Thus, by repeatedly executing the processing in the image data transfer unit 17, the image processing unit 14, the pixel number conversion unit 15, and the JPEG conversion unit 16, the ring buffer of the RAM 3 is used at the belt position where there is no character image data to be synthesized. The processing is executed for each belt.

  Thereafter, the same processing is repeatedly executed to convert up to the fourth belt into YUV data and further into JPEG data as shown in FIGS.

  Thereafter, when the image data transfer unit 17 transfers the Bayer data of the fifth belt to the ring buffer at Step S501, it is determined at Step S502 that the Bayer data of this belt is set within the range of the stop belt.

  Having determined this, the image data transfer unit 17 then determines whether or not the belt is the last belt within the range of the stop belt (step S503).

  Here, the fifth belt is within the range of the stop belt, but since the final belt within the range is six belts, it is determined that the belt is not the final stop belt, and the process returns to step S501 again.

  Accordingly, when the image data transfer unit 17 transfers the Bayer data of the sixth belt to the ring buffer in Step S601, it is determined in Step S602 that the Bayer data of this belt is set as the stop belt range.

  When the image data transfer unit 17 that has determined this determines that the belt is the last belt in the range of the stop belt in the subsequent step S503, it issues a processing stop signal to the image composition unit 18 (step S504). ) By repeatedly determining whether or not an image synthesis processing end signal has been sent from the image synthesis unit 18 (step S505), the system waits for the signal to be sent.

  The image compositing unit 18 waits for a processing stop signal to be sent from the image data transfer unit 17 (step S701), and when it is determined that it has been sent, the stop range belt from the ring buffer area of the RAM 3 described above. YUV data corresponding to two belts (here, 5 belts, 6 belts) is transferred to the image composition unit 18 in finer data units, and the character image data composition processing corresponding to the YUV data is collectively performed. To generate a composite image (YUV data), and transfer the composite image data to the ring buffer area of the RAM 3 again (step S702).

Here, the data transferred from the ring buffer area to the image compositing unit 18 may be data only for a portion where the compositing process is performed.
As a result, the processing speed can be further improved.

  After completion of the composition processing, the image composition unit 18 issues an image composition processing end signal to the image data transfer unit 17 (step S703), and the processing is terminated assuming that the image composition processing for the captured image has been completed. .

  The image data transfer unit 17 that has received the image synthesis process end signal from the image synthesis unit 18 determines this in step S505, and again sends the transfer end line position “6” and the process end signal to the JPEG conversion unit 16. After issuance (step S506), it is determined that the belt position of the currently transferred image data is the final belt of the Bayer data (step S507), and the processing related to the transfer of the image with respect to the photographed image is also completed. To finish the process.

  FIG. 12 illustrates the processing in the image data transfer unit 17 at this time. If the YUV data of the fifth belt and the sixth belt are not stop belts, as shown in FIG. 12 (2), the processing end signal to the JPEG conversion unit 16 is originally issued. The conversion processing in the JPEG conversion unit 16 is stopped without temporarily outputting the end signals for the second belt of the eye and the sixth belt, and then the end signal related to the combination processing for a plurality of belts is received from the image combining unit 18. After that, a processing end signal is issued.

  When the JPEG conversion unit 16 receives the transfer end line position and the processing end signal from the image data transfer unit 17, the JPEG conversion unit 16 determines this in step S 601, and determines the belt positions “5” and “6” after image synthesis held in the ring buffer. The YUV data for 2 belts is converted into JPEG data and compressed, and the converted JPEG data is written in the corresponding area of the RAM 3 (step S602).

  Thereafter, the JPEG conversion unit 16 determines that the converted data is the final belt (step S603), and ends the processing assuming that the conversion processing of the photographed image into JPEG data is completed.

  In the ring buffer, the enlarged YUV data of the fifth belt is written by the pixel number conversion unit 15 as shown in FIG. 13 (4), and the sixth belt as shown in FIG. 13 (5). After the enlarged YUV data is written by the pixel number conversion unit 15, the image data transfer unit 17 determines that this belt is the final belt within the stop belt range, and the image composition unit 18 determines characters for a plurality of belts. By executing the batch image composition processing of the images, the stored contents are as shown in FIG.

  JPEG data is generated from the YUV data after the image synthesis by the JPEG conversion unit 16 and held in the RAM 3 as part of the JPEG data.

  Thereafter, the image data transfer unit 17, the image composition unit 18, and the JPEG conversion unit 16 recognize that the JPEG conversion of the final belt has been completed, and all the processes in FIG. 11 are completed, and are stored in the RAM 3 anew. One frame of JPEG data is recorded on a memory card as a recording medium via the memory card interface 8.

  As described above, in this operation example, the conversion processing in the JPEG conversion unit 16 is temporarily delayed in accordance with the size of the character image to be combined.

  In this way, image composition over a plurality of belts is collectively processed, and during that time, conversion processing in the subsequent stage is temporarily delayed. The alignment of the character images is easy and the processing can be executed efficiently.

  In the above embodiment, the position and size of the character image to be synthesized can be arbitrarily set as described above.

  As a result, not only the date information is simply synthesized as in the present embodiment, but also the message and memo information arbitrarily set by the user is arbitrarily set, so that the user can not only shoot images but also the user. Necessary information can be easily recorded.

  In addition, as shown in FIG. 3, various settings relating to image composition are set in advance, so that processing from when an image is actually captured until the necessary character image is synthesized and recorded with the image data is recorded. Can be greatly simplified, and the load on the DMAC 11 that controls the control operation in the image processing IC 2 and the CPU 1 that controls the entire control operation in the camera can be reduced, and the time to shift to the next shooting can be reduced. Shortening and improving the continuous fire performance.

  In the above-described embodiment, a case has been described in which a single character image is synthesized in one captured image. However, the present invention is not limited to this, and it is also possible to synthesize a character image in a plurality of locations.

  Note that when combining the information image data with the image data, the number of lines in the region division unit of the image data may be determined based on the number of lines of the information image data.

In that case, the number of lines in the information image data and the number of lines in the area division unit are the same, and when the area division is performed, the synthesis position is set so that the synthesis position of the information image fits in the area divided area And may be combined.
This makes it possible to perform image composition more efficiently.

  In the above embodiment, the Bayer data is captured and converted into YUV data and output, and the image is synthesized by the image composition unit, and the YUV data is captured and converted into JPEG data and output. This was done in the JPEG conversion unit. In other words, image conversion is performed not by computer software but by a hardware circuit.

  However, the present invention is not limited to this, and the data may be placed on the RAM 3 and converted from Bayer data → YUV data → JPEG data by software instead of hardware.

  In the above embodiment, in steps S201 and S501, the Bayer data for one belt (8 lines) of the belt to be combined is transferred from the Bayer data area of the RAM 3 to the image processing unit 14 in smaller data units. After being converted into 16 lines of YUV data as set above, it is transferred to the pixel number conversion unit 15 and the process of enlarging the number of pixels at a magnification of “2” is executed as set above, and the ring buffer of the RAM 3 In step S402 and S702, YUV data for a predetermined belt is transferred from the ring buffer area of the RAM 3 to the image composition unit 18 in finer data units in steps S402 and S702. The corresponding character image data is synthesized on the data to produce a synthesized image (YUV data To generate, but is configured to transfer the composite image data again into RAM3 ring buffer area, may not necessarily such a.

For example, the Bayer data for one belt (8 lines) of the belt to be combined is transferred from the Bayer data area of the RAM 3 to the image processing unit 14 in smaller data units and converted to 16 lines of YUV data as set above. Then, after transferring to the pixel number conversion unit 15 and executing the process of enlarging the number of pixels at a magnification of “2” as set above, the image composition unit without transferring the image to the ring buffer area. The data is transferred to 18, the corresponding character image data is combined with this YUV data to generate a composite image (YUV data), and the composite image data is transferred to the ring buffer area of the RAM 3. You may comprise.
As a result, the processing speed can be improved.

  In the above embodiment, the process of generating JPEG data from Bayer data is described in detail. However, the process of generating display data from Bayer data is not described in detail, but display data refers to here. Since it refers to a so-called thumbnail image, the basic processing flow is the same as the flow of generating JPEG data from Bayer data, and detailed description thereof is omitted.

  The difference in the generation process between JPEG data and display data is that when YUV data is generated from Bayer data, the pixel number conversion unit 15 performs enlargement processing or reduction processing. Further, there is a difference between whether data converted into JPEG data by the JPEG conversion unit 16 is stored in the JPEG data area or in the display data area. However, the other points may be the same.

  In the present embodiment, by using a ring buffer, conversion processing from Bayer data to YUV data can be executed in 32 bits, and conversion processing from YUV data to JPEG data can be executed in 16 bits. Since the conversion processing from YUV data to JPEG data is limited to 16 bits or 8 bits, the processing speed of the process before and after that can be changed by having a ring buffer, so compared with the case where no ring buffer is used. And can be processed at high speed.

  In addition, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the invention in the implementation stage. In addition, the functions executed in the above-described embodiments may be implemented in appropriate combination as much as possible. The above-described embodiment includes various stages, and various inventions can be extracted by an appropriate combination according to a plurality of disclosed structural requirements. For example, even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, if the effect is obtained, a configuration from which the constituent requirements are deleted can be extracted as an invention.

1 is a block diagram illustrating a main configuration of an electronic circuit of a digital camera according to an embodiment of the present invention. FIG. 2 is a block diagram showing a circuit configuration in the image processing IC of FIG. 1 according to the embodiment. 6 is a flowchart showing processing contents executed in the image processing chip at the time of prior setting prior to image recording according to the embodiment. The figure explaining the image data size of Bayer data and YUV data concerning the embodiment. 9 is a flowchart showing the processing contents executed in the image processing chip during image recording in the first operation example according to the embodiment. The figure which illustrates the order of the image processing when not synthesizing character image data in the first operation example according to the embodiment. The figure which shows the use area | region of RAM which concerns on the embodiment compared with the case where a ring buffer is not used. The timing chart which shows the control content of the image processing in the case of synthesize | combining character image data in the 1st operation example which concerns on the embodiment. The figure which illustrates the image before and after a synthesis | combination in the case of synthesize | combining character image data in the 1st operation example which concerns on the embodiment. The figure which illustrates the order of the image processing in the case of synthesize | combining character image data by the 1st operation example which concerns on the embodiment. 9 is a flowchart showing the processing contents executed in the image processing chip during image recording in the second operation example according to the embodiment. The timing chart which shows the control content of the image processing in the case of synthesize | combining character image data in the 2nd operation example which concerns on the embodiment. The figure which illustrates the order of the image processing in the case of synthesize | combining character image data by the 2nd operation example which concerns on the embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... CPU, 2 ... Image processing IC, 3 ... RAM, 4 ... CCD, 5 ... ROM, 6 ... LCD, 7 ... Key input part, 8 ... Memory card interface (I / F), 9 ... Power supply part, 11 ... DMA controller (DMAC), 12 ... Memory control unit, 13 ... CCD control unit, 14 ... Image processing unit, 15 ... Pixel number conversion unit, 16 ... JPEG conversion unit, 17 ... Image data transfer unit, 18 ... Image composition unit, 19 ... LCD control unit, 20 ... key input control unit, CB ... control bus, DB ... data bus.

Claims (14)

Photographing means for photographing the subject and obtaining the whole image data;
First storage means for storing the entire image data;
First image processing means for obtaining partial image data by performing first image processing on image data transferred from the first storage means in predetermined area division units;
Second storage means for holding partial image data obtained by the first image processing means;
Combining means for combining information data at a predetermined location in the partial image data held by the second storage means;
Second image processing means for performing second image processing on the partial image data transferred from the second storage means;
Transfer control means for controlling the transfer operation of partial image data from the first storage means to the second image processing means;
An image pickup apparatus comprising: a stop control unit that temporarily stops the transfer operation of the partial image data by the transfer control unit when the combining unit combines the information data. A third storage means for holding partial image data obtained by the second image processing means;
2. The imaging apparatus according to claim 1, wherein the transfer control unit controls a transfer operation of partial image data from the first storage unit to the third storage unit.   The first image processing unit performs first image processing including enlargement / reduction processing on the image data transferred from the first storage unit in a predetermined area division unit, thereby obtaining partial image data. The imaging device according to claim 1, wherein the imaging device is obtained.   The first image processing means performs first image processing including image color conversion processing on the image data transferred from the first storage means in a predetermined area division unit to obtain partial image data. The imaging device according to claim 1, wherein the imaging device is obtained.   2. The imaging apparatus according to claim 1, wherein the second image processing means performs second image processing including compression processing on the partial image data transferred from the second storage means. .   2. The imaging apparatus according to claim 1, wherein the area division unit is obtained by dividing the entire image data by a predetermined number of lines.   The imaging apparatus according to claim 1, wherein the transfer control unit adjusts a transfer operation in units of the area division.   The imaging apparatus according to claim 1, wherein the synthesis unit performs the synthesis for each partial image data. A discriminating unit for discriminating the position and size of the information data to be synthesized in the image data to be synthesized by the synthesizing unit;
The imaging apparatus according to claim 1, wherein the stop control unit controls to stop the transfer operation of the partial image data according to a determination result by the determination unit.   2. The imaging apparatus according to claim 1, wherein the synthesizing unit performs the synthesis for a predetermined number of partial image data.   The stop control means determines a timing for stopping the partial image data transfer operation before the first image processing means performs the first image processing to obtain partial image data. The imaging apparatus according to claim 1. The partial image data further includes a determination unit that determines whether or not the information data is partial image data to be combined.
The imaging apparatus according to claim 1, wherein the stop control unit performs stop control on the transfer operation of the partial image data according to a determination result by the determination unit. An information synthesizing device mounted on an image processing apparatus comprising first storage means for storing whole image data and second storage means,
The first image processing is performed on the image data transferred from the first storage means in a predetermined area division unit to obtain partial image data, and the partial image data is transferred to the second storage means. First image processing means configured to be transferred;
Combining means for combining information data at a predetermined location in the partial image data held by the second storage means;
Second image processing means for performing second image processing on the partial image data transferred from the second storage means;
Transfer control means for controlling the transfer operation of partial image data from the first storage means to the second image processing means;
An information synthesizing apparatus comprising: stop control means for temporarily stopping the transfer operation of the partial image data by the transfer control means when the synthesizing means synthesizes the information data. An information synthesizing device mounted on an image processing apparatus comprising first storage means for storing whole image data and second storage means,
The first image processing is performed on the image data transferred from the first storage means in a predetermined area division unit to obtain partial image data, and the partial image data is transferred to the second storage means. First image processing means configured to be transferred;
Combining means for combining information data at a predetermined location in the partial image data held by the second storage means;
An information synthesizing apparatus comprising: second image processing means for performing second image processing on the partial image data transferred from the second storage means;
Transfer control means for controlling the transfer operation of partial image data from the first storage means to the second image processing means;
A program for realizing stop control means for temporarily stopping the transfer operation of partial image data by the transfer control means when the combining means synthesizes the information data.

Images