JP4190384B2 - Video camera - Google Patents

Description

  The present invention relates to a video camera, and more particularly to a video camera that records and reproduces image data and audio data, for example.

  An example of a conventional signal processing apparatus of this type is disclosed in Patent Documents 1 and 2. In the prior art (1) disclosed in Patent Document 1, the difference between the time stamp added to the video signal and the time stamp added to the audio signal is obtained, and the vertical blanking signal period sent to the display is set as this difference. Accordingly, the video signal and the audio signal can be smoothly reproduced in a synchronized manner.

Further, the prior art (2) disclosed in Patent Document 2 is based on the reproduction time obtained by monitoring the reproduction amount of audio data and the time stamp assigned to the image data, between the audio and the image. It is intended to ensure synchronization.
JP-A-8-140054 [H04N 5/93, G11B 20/10] Japanese Patent Application Laid-Open No. 11-134804 [G11B 20/10, H04N 5/937, 7/08, 7/081]

  However, in the prior art (1), since a time stamp is added to both the video signal and the audio signal at the time of recording, a clock circuit is required to generate the time stamp. Even in the prior art (2), it is necessary to prepare a clock circuit for generating a time stamp in order to add a time stamp to image data at the time of recording. That is, both the conventional techniques (1) and (2) have a problem that the circuit configuration is complicated.

  SUMMARY OF THE INVENTION Therefore, a main object of the present invention is to provide a video camera that can ensure synchronization between a reproduced image and reproduced sound with a simple circuit configuration.

  According to a first aspect of the present invention, there is provided a video camera for recording a moving image signal of a subject photographed by an image sensor and an audio signal captured by a microphone on a recording medium. First image output means for output to the monitor in a raster scanning manner, first detection means for intermittently detecting pixel positions scanned by the first image output means, and pixel positions detected by the first detection means A video camera comprising: assigning means for assigning to the audio signal captured by the microphone at the time of detection by the first detecting means.

  In the first aspect of the invention, the first screen output means outputs the still image signal of each screen forming the moving image signal to the monitor in a raster scanning manner, and the first detection means is scanned by the first image output means. The detected pixel position is detected intermittently. Then, the assigning means assigns the pixel position detected by the first detecting means to the audio signal captured by the microphone at the time of detection by the first detecting means. Therefore, at the time of reproduction, synchronization between the reproduced image and the reproduced audio can be ensured by comparing the pixel position assigned to the audio signal with the pixel position of the still image signal currently reproduced (scanned).

  The invention of claim 2 further comprises encoding means for performing encoding processing for each predetermined amount prior to recording of the audio signal captured by the microphone, and the first detecting means is provided each time a predetermined amount of the audio signal is captured. The video camera according to claim 1, wherein the video camera detects a pixel position.

  According to the second aspect of the present invention, the encoding means performs an encoding process on the audio signal for each predetermined amount prior to recording of the audio signal captured by the microphone, and the first detecting means captures the predetermined amount of the audio signal. Each time a pixel is scanned, the position of the scanned pixel is detected. Therefore, the pixel position at the time when the audio signal is captured is assigned to the audio signal.

  According to a third aspect of the present invention, the second image output means for outputting the still image signal of each image forming the moving image signal read from the recording medium toward the monitor in a raster scanning manner, read from the recording medium Audio output means for outputting an audio signal to a speaker, second detection means for detecting a pixel position assigned to the audio signal output by the audio output means, and second image output means at the time of detection by the second detection means A discriminating unit that discriminates a deviation between the pixel position being scanned and the pixel position detected by the second detecting unit, and the number of still image signals output by the second image output unit according to the discrimination result The video camera according to claim 1, further comprising adjusting means for adjusting the image.

  In the invention of claim 3, the second image output means outputs the still image signal of each image forming the moving image signal read from the recording medium to the monitor in a raster scanning manner, and the audio output means is the recording medium. The audio signal read from is output to the speaker. The second detection means detects the pixel position assigned to the audio signal output by the audio output means. The discriminating unit discriminates a deviation between the pixel position scanned by the second image output unit and the pixel position detected by the second detecting unit at the time of detection by the second detecting unit. The adjusting means adjusts the number of still image signals output from the second image output means according to the determination result of the determining means. Therefore, the synchronization between the sound and the image can be ensured by detecting the synchronization shift between the sound and the image at the pixel position level and adjusting the number of images according to the shift.

According to a fourth aspect of the present invention, a moving image signal is recorded on a recording medium in a state of being encoded for each screen, and the second image output means decodes the encoded still image signal of each screen in response to a decoding command. It includes means, adjusting means for controlling the generation timing of the decoding instructions, a video camera according to claim 3 Symbol placement.

  According to the fourth aspect of the present invention, the moving image signal is recorded on the recording medium in a state of being encoded one screen at a time, and the decoding means decodes the encoded still image signal of each screen in accordance with the decoding command. Then, the generation timing of the decoding instruction given to the decoding unit by the adjusting unit is controlled. Therefore, by controlling the decoding timing of the encoded still image signal, it is possible to adjust the synchronization deviation between the sound and the image and ensure the synchronization.

  The decoding means writes the still image signal decoded in response to the decoding command to the first memory, and reads the still image signal stored in the first memory from the first memory at a predetermined cycle. 5. A video camera according to claim 4.

  In the invention of claim 5, the decoding means writes the still image signal decoded in response to the decoding command to the first memory which is a buffer, and the still image signal written in the first memory is written to the first memory at a constant cycle. Read from. Since the still image signal written in the first memory is reproduced, the cycle of the decoding command given to the decoding means is made shorter or longer than the cycle of reading the still image signal written in the first memory, It is possible to adjust the synchronization with the sound by apparently skipping or stopping the reproduced moving image.

  6. The video camera according to claim 4, wherein the adjusting means extends the decoding command generation cycle when increasing the number of screens, and shortens the decoding command generation cycle when decreasing the number of screens. It is.

  According to the sixth aspect of the present invention, the adjusting means extends the generation cycle of the image decoding command when increasing the number of images, and shortens the generation cycle of the decoding command when decreasing the number of screens. That is, if the generation cycle of the decoding command is longer than the read cycle of the still image signal from the first memory, the still image signal is read from the first memory before the still image signal recorded in the first memory is updated. Therefore, the same image is reproduced and the number of screens increases. As a result, the display of the image is paused and the sound can catch up with the image. On the contrary, if the generation cycle of the decoding instruction is made shorter than the reading cycle of the still image signal from the first memory, the contents of the first memory are overwritten before the still image signal is read from the first memory. The still image signal read from the memory is skipped for one screen. Therefore, the number of screens to be played is reduced. Thereby, the delay of the image with respect to the sound can be recovered.

  The second image output means further includes a writing means for writing the still image signal output from the decoding means to the second memory, and a reading means for reading the still image signal stored in the second memory. A video camera according to any one of claims 4 to 6.

  According to another aspect of the invention, the writing means writes the still image signal output from the decoding means to a second memory such as an SDRAM, and the reading means reads the still image signal written to the second memory. In this way, the still image signal written in the second memory is read and reproduced.

  According to an eighth aspect of the present invention, the second memory includes a first area and a second area, the writing means alternately writes still image signals output from the decoding means to the first area and the second area, and the reading means is stationary. The video camera according to claim 7, wherein image signals are alternately read from the first area and the second area.

  According to an eighth aspect of the present invention, the second memory includes a first area and a second area. The writing means alternately writes the still image signal output from the buffer (first memory) of the decoding means to the first area and the second area, and the reading means alternately writes the still image signal from the first area and the second area. Read to.

  The invention of claim 9 is the video camera according to claim 8, wherein the writing by the writing means and the reading by the reading means are performed in a complementary manner.

  In the invention of claim 9, the writing to the second memory by the writing means and the reading from the second memory by the reading means are performed in a complementary manner. Accordingly, since the overtaking of the read address with respect to the write address in the second memory does not occur, no frame shift appears on the playback screen.

  A tenth aspect of the present invention is the video camera according to any one of the fourth to ninth aspects, wherein the moving image signal is encoded according to the MPEG4 system.

  In the invention of claim 10, the moving image signal is encoded according to the MPEG4 system.

  According to the present invention, since the reproduction timing of the moving image data is controlled based on the pixel position of the moving image data, a clock circuit is not required, and synchronization of the moving image data and the audio data is realized with a simpler circuit configuration. can do.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  A video camera 10 according to an embodiment of the present invention shown in FIG. 1 is intended to ensure synchronization of image (moving image) data and audio data without using a time stamp. In order to synchronize the image and sound during playback, the field number information, the vertical line position information, and the horizontal pixel position information of the through-image data displayed at the time of input of the sound data are added to the sound data at the time of recording. "Synchronized sample information"). At the time of reproduction, the synchronization sample information is extracted from the audio frame, and the synchronization deviation between the audio and the image is detected from the extracted synchronization sample information and the field number, vertical line position, and horizontal pixel position of the currently reproduced image. calculate. When this deviation exceeds the allowable range, the number of screens is adjusted by temporarily stopping the image or skipping the image to correct the deviation.

  In the video camera 10 of this embodiment, since the image and the sound are synchronized with the above-described configuration, a clock circuit is not required unlike the conventional one, and the synchronization between the image and the sound can be realized with a simple circuit configuration.

  With reference to FIG. 1, a video camera 10 includes an image sensor 12. The optical image of the object scene is irradiated onto the light receiving surface of the image sensor 12 through an optical lens (not shown).

  When the operator selects the camera mode with the mode switch 42md provided on the operation panel 42, the CPU 40 executes through image processing. The image sensor 12 outputs a raw image signal of the object scene every 1/30 seconds, and the CDS / AGC / AD circuit 14 performs a series of processes of noise removal, level adjustment and A / D conversion on the output raw image signal. Execute. The CDS / AGC / AD circuit 14 outputs raw image data that is a digital signal. The signal processing circuit 16 performs signal processing such as white balance adjustment, color separation, and YUV conversion on the output raw image data, thereby generating YUV format image data. The generated image data is written into the expanded image recording area of the SDRAM 30 by the memory control circuit 32. The expanded image recording area is divided into two banks A and B, and the first frame of image data is first written in bank A. Banks are switched every 1/30 seconds, the second frame is written to bank B, and the third frame is written to bank A again.

  The video encoder 22 reads the image data written in the bank of the SDRAM 30 through the memory control circuit 32 in a progressive manner every 1/60 seconds. First, image data is read from bank A, and then image data is read from bank B. The image data read from the bank A or bank B of the SDRAM 30 is converted into a composite video signal by the video encoder 22 and reduced to a size of 640 pixels × 240 lines. The reduced composite video signal is given to the LCD 24 by a pseudo interlace scan method. As a result, an image is displayed on the LCD 24 at a rate of 1 field per 1/60 seconds (16.6 milliseconds).

  The video encoder 22 uses the V counter 22v to count how many lines in the current field have been drawn, and counts how many pixels in the current line have been drawn using the H counter 22h. is doing. The maximum value of the V counter 22v is 240, and the maximum value of the H counter 22h is 640.

  When the shutter button 42st provided on the operation panel 42 is operated by the operator, the CPU 40 activates the signal processing circuit 20, and cycles each of an image compression (encoding) command and an audio compression (encoding) command. Thus, the MPEG-4 codec 34 is provided.

  The activated signal processing circuit 20 samples the audio signal input from the microphone 18 at a cycle of 48 kHz, converts it into audio data that is a digital signal, and writes this audio data to the SDRAM 30 through the memory control circuit 32. In the 48 kHz sampling process, about 21 milliseconds of audio data are generated by 1024 samples. As will be described in detail later, the audio data is processed into data called audio frames in units of 1024 samples. The CPU 40 monitors the amount of audio data written to the SDRAM 30, and each time writing of audio data for 1024 samples is started, the count value (vertical line) of the V counter 22v from the video encoder 22 is written at the start timing of writing. Position information) and the count value (horizontal pixel position information) of the H counter 22h are acquired.

  The CPU 40 counts the image field number of the through image displayed on the LCD 24 by the field counter 40f from the start of image display. The count value (field number information) of the field counter 40f is recorded in the table 46t on the RAM 46 in association with the audio data for every 1024 samples together with the vertical line position information and the horizontal pixel position information acquired from the video encoder 22. Is done.

  Since the image field is generated every 1/60 seconds (16.6 milliseconds) and the audio frame is generated every 21 milliseconds, the generation timing of the image field and the audio frame is as shown in FIG. As can be seen from FIG. 2, there is no one-to-one correspondence between the audio frame number and the field number (information) added as synchronization sample information to the audio frame. In the example of FIG. 2, there is no audio frame to which the field number “4” is attached.

  When an image compression command is given, the MPEG-4 codec 34 reads audio data from the SDRAM 30 through the memory control circuit 32 as needed, and performs MPEG-4 compression on the read image data. One frame of image data is compressed by one image compression command. The compressed image data reproduced by the MPEG-4 compression is written again into the SDRAM 30 through the memory control circuit 32.

  The MPEG-4 codec 34 reads audio data from the SDRAM 30 through the memory control circuit 32 as needed when an audio compression command is given. Then, MPEG-4 compression is performed on the read audio data. The audio data of 1024 samples is compressed by one audio compression command. The compressed audio data generated by the MPEG-4 compression is written into the SDRAM 30 again through the memory control circuit 32.

  Unless the shutter button 42st is pressed again, the CPU 40 repeatedly gives an image compression command and an audio compression command to the MPEG-4 codec 34 and keeps the signal processing circuit 20 activated. As a result, the compressed image data and the compressed audio data are accumulated in the SDRAM 30.

  In the memory card 38, an MPEG file is newly created in response to the first operation of the shutter button 42st. The CPU 40 reads the compressed image data and the compressed audio data stored in the SDRAM 30 through the memory control circuit 32 in parallel with the output of the above-described compression processing. The read compressed image data and compressed audio data are given to the memory card 38 through the I / F 36 and recorded.

  When the compressed audio data is read from the SDRAM 30 and recorded on the memory card 38, the audio data is grouped into units of audio frames having an audio data header every 1024 samples, as shown in FIG. When the audio data of 1024 samples is collected into an audio frame, the synchronization sample information corresponding to the audio data of the 1024 samples is identified with reference to the table 46t formed in the RAM 46, and the identified synchronization sample information is included in the audio frame. Added. Although FIG. 3 shows a state in which audio frames are continuous, the MPEG file formed on the memory card 38 is recorded with a mixture of compressed image data and compressed audio data.

  When the recording of the compressed image data and the compressed audio data is completed, information such as the total file size, the total number of frames, the total audio size, and the frame rate is written in the file header of the MPEG file. In addition, the file header includes a table holding information such as which address the image data of which image frame and the audio data of which audio frame are recorded at. By referring to this table, image data and audio data of a desired frame can be read out.

  When the shutter button 42st is operated by the operator, the CPU 40 disables the signal processing circuit 20 and stops outputting the compression command to the MPEG-4 codec 34. Thereby, the generation of the compressed image data and the compressed audio data is completed, and then the writing of the compressed image data and the compressed audio data to the memory card 38 is completed.

  As described above, the audio data recorded in the MPEG file is grouped into units called audio frames every 1024 samples, and in each audio frame, the field number of the image data at the timing when the audio data of 1024 samples starts to be recorded in the SDRAM 30, Information (synchronous sample information) indicating the vertical line position and the horizontal pixel position is added.

  When the operator selects the playback mode with the mode switch 42md and selects the desired MPEG file with the file selection key 42fs, the CPU 40 refers to the above-mentioned table included in the file header and compresses the first frame. Image data and compressed audio data as the first audio frame are read from the memory card 38. The read compressed image data and compressed audio data are written into the SDRAM 30 through the memory control circuit 32.

  The CPU 40 also repeatedly gives the MPEG-4 codec 34 an image expansion (decoding) instruction and an audio expansion (decoding) instruction. An image decompression command is issued every 1/30 seconds (33 milliseconds), and an audio decompression command is issued every 21 milliseconds.

  When an audio decompression command is given from the CPU 40, the MPEG-4 codec 34 reads compressed audio data from the SDRAM 30 through the memory control circuit 32, and performs MPEG-4 decompression on the read compressed audio data. One frame of compressed audio data is expanded by one audio expansion command. The expanded audio data is written into the SDRAM 30 again through the memory control circuit 32. The signal processing circuit 26 reads the decompressed audio data through the memory control circuit 32, converts the read decompressed audio data into an analog signal, and gives the converted audio data to the speaker 28. As a result, the reproduced sound is output from the speaker 28. The compressed audio data is sequentially read from the memory card 38, decompressed by the MPEG-4 codec 34, and output from the speaker 28. As a result, sound is output from the speaker 28 without interruption.

  The MPEG-4 codec 34 reads compressed image data from the SDRAM 30 through the memory control circuit 32 when an image decompression command is given from the CPU 40. The read compressed image data is subjected to MPEG-4 decompression. One frame of compressed image data is expanded by one image expansion command. The decompressed image data is stored in a buffer 34a (first memory) included in the MPEG-4 codec 34. One frame of image data stored in the buffer 34a is read every 1/30 seconds (33 milliseconds), and is recorded in the expanded image recording area (second memory) of the SDRAM 30 through the memory control circuit 32. As described above, the expanded image recording area is divided into two banks A and B, and the expanded image data is first recorded in bank A (first area). The bank is switched every 1/30 seconds, and the next expanded second image frame is recorded in bank B (second area), and the third image frame is recorded in bank A again.

  The video encoder 22 reads the image data expanded from the bank of the SDRAM 30 through the memory control circuit 32 every 1/60 seconds by the progressive scan method. First, the image data recorded in the bank A is read, then the bank is switched and the image data recorded in the bank B is read. The video encoder 22 converts the image data read from the SDRAM 30 into a composite video signal, reduces the image data to a size of 640 pixels × 240 lines, and supplies the reduced image data to the LCD 24 by a pseudo interlace scan method. By this pseudo interlace scan method, an image is displayed on the LCD 24 at a rate of 1 field per 1/60 seconds.

  Further, the video encoder 22 uses the V counter 22v and the H counter 22h at the time of reproduction in the same way as at the time of recording, up to which line in the current field has been drawn, and up to which pixel of the current line. Is counted each time. Further, the field counter 40f of the CPU 40 counts what numbered image field is drawn.

  As shown in FIG. 2, since there is no one-to-one correspondence between audio frames and image fields, a mechanism for ensuring synchronization between the audio frames and the image fields is required. The audio is sequentially reproduced one by one in an audio frame, and is not skipped or stopped midway. On the other hand, for the image, the drawing position of the currently reproduced image data (the value of the field counter 40f, the value of the V counter 22v and the value of the H counter 22h) and the synchronous sample information (image field number information) added to the audio frame. , Vertical line position information and horizontal pixel position information), the deviation between the audio frame and the image field is calculated, and when the deviation is outside the allowable range, it is skipped or stopped to correct the deviation.

  This will be described more specifically with reference to FIG. When an audio frame is read from the memory card 38, synchronous sample information (image field number information, vertical line position information, and horizontal pixel position information) is extracted from the read audio frame. A position on the screen indicated by the synchronous sample information is defined as A point. This point A indicates the pixel position of the image that was being reproduced when this audio frame was sampled. That is, when this audio frame is sampled, it indicates that the Ya pixel of the (Xa + 1) th line of the Xth field has been drawn.

  Further, the CPU 40 acquires the count values of the V counter 22v and the H counter 22h from the video encoder 22 at the timing when the synchronization sample information is extracted from the audio frame. A position indicated by the acquired count value and the count value of the field counter 40f is defined as a B point. That is, at the time when the synchronous sample information is extracted from the audio frame, the Yb pixel of the (Xb + 1) th line of the Xth field is drawn.

  When the A point and the B point coincide with each other, the sound and the image are reproduced at the same timing as when recording. Therefore, the difference in position between the point A and the point B is a synchronization shift between the sound and the image. In the case of the example in FIG. 4, the image reproduction is advanced by (Xb−Xa) line and (Yb−Ya) pixels compared to the audio reproduction. In the example of FIG. 4, the point A and the point B exist on the same image field, but the point A and the point B may exist on different image fields.

  If the synchronization deviation between the sound and the image exceeds the allowable range, the deviation is corrected. That is, when the image is ahead of the sound, the image is stopped for one frame time (1/30 second), and when the image is behind the sound, the image is skipped by one frame.

  When stopping the image, the CPU 40 stops the image decompression command to the MPEG-4 codec 34 generated every 1/30 seconds once. The image data expanded by the MPEG-4 codec 34 is stored in the buffer 34a, and the image data stored in the buffer 34a is written into the bank A or the bank B of the SDRAM 30 through the memory control circuit 32 every 1/30 seconds. When the CPU 40 stops generating the image decompression command once, the decompressed image data stored in the buffer 34a is not updated. Therefore, the same image data as the previous time is written in the bank A or bank B of the SDRAM 30. That is, the same image data is written in bank A and bank B.

  For this reason, the video encoder 22 reads and reproduces the same image data twice in succession although the banks are different. Therefore, the same image as the previous one is displayed on the LCD 24, and the image is apparently stopped for one frame time.

  On the other hand, when skipping an image, the CPU 40 generates an image decompression command to the MPEG-4 codec 34, which is normally generated once every 1/30 seconds, twice during 1/30 seconds. Then, the contents of the buffer 34a read once every 1/30 seconds and written to the bank of the SDRAM 30 are overwritten by the next decompressed image data before being read. Therefore, the overwritten image data in the buffer 34 a is written into the bank A or bank B of the SDRAM 30. The overwritten image data is read by the video encoder 22 and reproduced. Therefore, although continuous image frames are decompressed by the MPEG-4 codec 34, an image is displayed on the LCD 24 with one frame skipped. In this way, the deviation between the sound and the image is corrected, and synchronization is maintained.

  Hereinafter, the operations of the CPU 40 in the image compression task, the audio capture task, the audio compression task, the audio reproduction task, and the image reproduction task will be described using the flowcharts shown in FIGS. Note that the image compression task, the audio capture task, and the audio compression task are executed in parallel at the same time, and the audio reproduction task and the image reproduction task are executed in parallel at the same time. The CPU 40 operates according to a program stored in the flash memory 44.

  The operation of the CPU 40 in the image compression task is shown in the flowchart of FIG. First, in step S1, the CPU 40 determines whether or not a recording start operation has been performed by the operator. If YES is determined in the step S1, then in a step S3, the process waits until 1/30 seconds elapse from the recording start operation. This waiting for 1/30 seconds is for waiting for image data for one frame to be written into the SDRAM 30 by the memory control circuit 32 via the signal processing circuit 16.

  If YES is determined in the step S3, an image compression command is given to the MPEG-4 codec 34 in a step S5. Then, the MPEG-4 codec 34 reads image data for one frame from the SDRAM 30 through the memory control circuit 32 and performs MPEG-4 compression. Then, the compressed image data is written into the SDRAM 30 through the memory control circuit 32 again. The compressed image data written in the SDRAM 30 is sequentially written in the memory card 38 by another task.

  When an image compression command is given to the MPEG-4 codec 34 in step S5, it is determined in step S7 whether 1/30 seconds have elapsed since the generation of the compression command. If YES is determined in the step S7, the process returns to the step S5 to again give an image compression command to the MPEG-4 codec 34.

  If NO is determined in step S7, it is determined in step S9 whether or not a recording stop operation has been performed by the operator. If NO is determined in step S7, the process returns to step S7, and if YES is determined, the image compression task is ended.

  The operation of the CPU 40 in the voice capturing task is shown in the flowchart of FIG. First, in step S11, the CPU 40 determines whether or not a recording start operation has been performed by the operator. When the recording start operation is performed, the sound input from the microphone 18 is converted into sound data by the signal processing circuit 20, and this sound data is written into the SDRAM 30 through the memory control circuit 32. In step S13, the writing start position (address) of the audio data in the SDRAM 30 is acquired and stored in the register W1. The value stored in the register W1 is expressed as “W1”.

  In step S15, a first synchronous sample process is performed. In the first synchronization sample processing, synchronization sample information corresponding to the first 1024 samples (first frame) is acquired, and the acquired synchronization sample information is recorded in the table 46t of the RAM 46 in association with the audio data of the 1024 samples. . That is, at the time when writing of the audio data to the SDRAM 30 is started, the values of the V counter 22v and the H counter 22h are obtained from the video encoder 22, and the value of the image field number counted by the CPU 40 is obtained from the field counter 40f. To do.

  In step S17, the current write end position (address) of the audio data in the SDRAM 30 is acquired and stored in the register W2. The value stored in the register W2 is expressed as “W2”.

  In step S19, the data amount of the audio data written in the SDRAM 30 is calculated from the value stored in the register W1 and the value stored in the register W2. In step S21, it is determined whether or not the amount of audio data written in the SDRAM 30 is 1024 samples.

  If NO is determined in step S21, the process returns to step S17 to acquire the write end position again. On the other hand, if YES is determined in the step S21, the first synchronous sample process is performed in a step S23. In the first synchronous sample processing, as described above, synchronous sample information corresponding to 1024 samples of audio data written to the SDRAM 30 is acquired and recorded in the table 46 t formed in the RAM 46.

  In step S25, in order to calculate audio data for the next 1024 samples, the write start position (W1) is updated with the address (W2 + 1) next to the current write end address. In step S27, it is determined whether or not the operator has performed the recording end operation by operating the shutter button 42st. If NO is determined in step S27, the process returns to step S17. If YES is determined in the step S27, the voice capturing task is ended.

  The operation of the CPU 40 in the voice compression task is shown in the flowchart of FIG. First, when the operator performs a recording start operation, the CPU 40 determines that in step S31. In step S33, the process waits until a 1024 sample period (21 milliseconds) elapses from the recording start operation. This standby is performed in order to wait for the voice of 1024 samples input from the microphone 18 to be written into the SDRAM 30 through the signal processing circuit 20 and the memory control circuit 32.

  If YES is determined in the step S33, the CPU 40 gives an audio data compression command to the MPEG-4 codec 34 in a step S35. As a result, audio data for one frame, that is, 1024 samples is taken into the MPEG-4 codec 34 and subjected to MPEG-4 compression. The compressed audio data is written into the SDRAM 30 again through the memory control circuit 32.

  When the audio data compression command is issued, in step S37, the synchronization sample information corresponding to the 1024 samples of audio data is acquired with reference to the table 46t formed in the RAM 46. In step S39, the acquired synchronous sample information is added to the audio data. As a result, one audio frame is formed on the SDRAM 30. The audio frame formed in this way is read from the SDRAM 30 as appropriate by another task and written to the memory card 38 through the I / F 36.

  In step S41, it is determined whether or not 1024 sample periods (21 milliseconds) have elapsed since the generation of the voice compression command in step S35. If YES is determined in the step S41, the process returns to the step S35 to again give an audio compression command to the MPEG-4 codec 34. If NO is determined in step S41, it is determined in step S43 whether or not the operator has operated the shutter button 42st to perform the recording end operation. If NO is determined in step S43, the process returns to step S41. If YES is determined in the step S43, the voice compression task is ended.

  The MPEG file including the audio data and the image data is formed on the memory card 38 by the image compression task, the audio capture task, and the audio compression task described above. Next, an audio reproduction task and an image reproduction task will be described.

  The operation of the CPU 40 in the sound reproduction task is shown in the flowchart of FIG. First, in step S51, it is determined whether or not the voice stop flag is in an on state. If the voice stop flag is in an on state, the process waits until the voice stop flag is turned off. When the voice stop flag is turned off, it is determined whether or not the position specifying flag is turned on in step S53.

  The audio stop flag and the position specifying flag are set to an on state in an image reproduction task described later. In the image reproduction task, audio is not reproduced when frame-by-frame reproduction or skip reproduction is performed. Therefore, during frame-by-frame playback or skip playback, the audio stop flag is turned on to stop audio playback. In addition, when the image is reproduced from the first frame, the sound may be reproduced from the first audio frame. However, when the image is reproduced from the middle image after frame-by-frame reproduction or skip reproduction, the image is reproduced. An audio frame corresponding to the image frame must be identified and reproduced from the identified audio frame. Therefore, when performing frame-by-frame playback or skip playback, the position specifying flag is set to the on state to notify the image playback task that it is necessary to specify the audio frame to be played back.

  If YES is determined in the step S53, an audio frame corresponding to the Fp-th image frame is specified with reference to the reproduction image frame number Fp counted by the CPU 40 using the register Fp in the image reproduction task in a step S55. To do. Note that the value stored in the register Fp is represented by “Fp”. Since the audio frame holds not the image frame number but the image field number, the audio frame holding the (Fp × 2-1) number field is the audio frame corresponding to the Fp image frame. As specified. In step S57, since the audio frame has been specified, the position specifying flag is set to the off state.

  In step S59, the MPEG-4 codec 34 is provided with a decompression command for the audio frame specified in step S55. If NO is determined in step S53, the first audio frame is expanded.

  In step S 61, synchronization sample information (image field number information, vertical line position information, and horizontal pixel position information) is acquired from the audio frame that is the target of the decompression command and recorded in the RAM 46. In step S63, the synchronous sample information acquisition flag is set to an on state in order to notify the image reproduction task of acquisition of the synchronous sampling information.

  In step S65, it is determined whether 21 milliseconds, which is the time taken to reproduce one audio frame, has elapsed since the audio frame decompression command was issued in step S59. If YES is determined in the step S65, the process returns to the step S51. On the other hand, if NO is determined in step S65, the process proceeds to step S67.

  In step S67, it is determined whether or not a reproduction end operation has been performed by the operator, or whether or not the reproduction of all data (audio frames and image frames) has been completed. When NO is determined in the step S67, the process returns to the step S65, and when YES is determined, the sound reproduction task is ended.

  The operation of the CPU 40 in the image reproduction task is shown in the flowcharts of FIGS. First, in step S71 in FIG. 9, 1 is stored in the register Fp and initialized in order to count the image frames to be reproduced. In step S73, it is determined whether or not an operation for frame advance reproduction or skip reproduction has been performed by the operator. If YES is determined in the step S73, an audio stop flag is set to an on state in order to notify the audio reproduction task that the frame advance reproduction or the skip reproduction is performed in a step S87. In step S89, the position specifying flag is set to ON to notify the audio playback task that the audio frame to be played back needs to be specified because the image is played back halfway by frame-by-frame playback or skip playback. To do.

  In step S91, frame advance playback or skip playback processing is performed. A detailed description of the frame advance playback and skip playback processing will be omitted. In the frame advance playback and skip playback processing, an image frame expansion command corresponding to frame advance playback or skip playback is issued, and the image frame counter Fp is updated.

  If NO is determined in step S73, the voice stop flag is set to an off state in step S75. Thereby, the playback of the audio frame is started in the audio playback task.

  Next, in step S77, whether or not the synchronization sample information acquisition flag is in an on state, that is, whether or not the timing for confirming the difference between the image and the audio has been acquired from the audio frame in the audio reproduction task. Judging.

  If NO is determined in step S77, it is determined in step S93 whether 1/30 second has elapsed since the previous image frame decompression command was generated. Since the image frame is reproduced every 1/30 seconds, if 1/30 seconds have not elapsed, the process returns to step S73.

  If YES is determined in the step S93, a decompression command for the Fp image frame is generated to the MPEG-4 encoder 34 in a step S95. As a result, the Fp image frame is expanded and reproduced. In step S97, the value stored in the counter Fp is incremented by one.

  On the other hand, if “YES” is determined in the step S77, confirmation and correction of the deviation between the image and the sound are performed. First, in step S79, the synchronization sample information acquisition flag is set to an off state.

  In step S81, a second synchronous sample process is performed. In this second synchronous sample processing, the values of the V counter 22v and H counter 22h are acquired from the video encoder 22 and recorded in the RAM 46, and the value of the image field number counted by the CPU 40 is acquired from the field counter 40f. Record in RAM 46.

  In step S83, the value indicated by the field number information, the value indicated by the vertical line position information, and the horizontal pixel position information extracted from the audio frame and recorded in the RAM 46 in step S61 (FIG. 8) of the previous audio reproduction task. And the value of the field counter 40f, the value of the V counter 22v, and the value of the H counter 22h acquired in the second synchronous sample processing in step S81 of this image reproduction task and recorded in the RAM 46, Calculate how many fields, how many lines, and how many pixels the playback timing of the image is shifted.

  In step S85, it is determined whether or not the timing difference between the sound and the image is within an allowable range. If YES is determined in the step S85, the process proceeds to the above-described step S93 to determine whether or not 1/30 second has elapsed from the previous image frame expansion. If the determination result is YES, the MPEG-4 codec 34 is determined in a step S95. Generates an image frame decompression instruction.

  On the other hand, if NO is determined in step S85, it is determined in step S101 in FIG. 10 whether or not the difference between the sound and the image is due to the delay of the image. If YES is determined in the step S101, an image skip process is performed in order to recover the image delay. That is, in step S103, the Fp image frame decompression command is issued to the MPEG-4 codec 34 without waiting for the elapse of 1/30 seconds from the generation of the previous image frame decompression command. In step S105, the value stored in the register Fp is incremented by 1, and the process returns to step S73 in FIG.

  The decompressed image data decompressed by the MPEG-4 codec 34 and accumulated in the buffer 34a is read every 1/30 seconds and written to the bank (bank A or bank B) of the SDRAM 30 through the memory control circuit 32. The decompressed image data written in the bank is read out by the video encoder 22 and reproduced.

  However, in step S103, since the image decompression command is given to the MPEG-4 codec 34 without waiting for 1/30 seconds, the contents of the buffer 34a are updated twice during 1/30 seconds. Therefore, the decompressed image data is written into the bank of the SDRAM 30 in one frame. Since the video encoder 22 reads the decompressed image data as it is, the image displayed on the LCD 24 is skipped by one frame. In this way, the image can catch up with the sound.

  On the other hand, if NO is determined in step S101, the display of the image is stopped for one frame in order to correct the preceding of the image. That is, in step S107, it waits for 1/15 seconds (twice 1/30 seconds), and proceeds to step S103.

  The contents of the buffer 34a of the MPEG-4 encoder 34 are read every 1/30 seconds and written to the bank of the SDRAM 30. However, since 1/30 seconds elapse without the MPEG-4 encoder 34 being given an image decompression command, the contents of the buffer 34a are not updated even after 1/30 seconds. Therefore, the same decompressed image data as in the previous time is read from the buffer 34a and written in the bank of the SDRAM 30, and the contents of the bank A and the bank B become the same. Therefore, since the video encoder 22 reads out and reproduces the same decompressed image data as in the previous time, although the bank is different, the image displayed on the LCD 24 is apparently stopped. Since the sound is played while the image is stopped, the sound catches up with the image.

  As described above, in the video camera 10 of this embodiment, when recording an image and sound, information on the image field number, vertical line position, and horizontal pixel position corresponding to the sound frame is stored in the sound frame. Append. Then, when reproducing, the audio is obtained from the image field number information, vertical line position information and horizontal pixel position information extracted from the reproduced audio frame, and the field number, vertical line position and horizontal pixel position of the currently reproduced image. And the synchronization deviation of the image is calculated. When the deviation exceeds the allowable range, the deviation is corrected by temporarily stopping the image or skipping the image. Therefore, unlike the prior art, a clock circuit for time stamp is not required, and synchronization between sound and image can be realized with a simple circuit configuration.

It is an illustration figure which shows the whole structure of one Example of this invention. It is an illustration figure which shows the generation | occurrence | production timing of an audio | voice frame and an image field. It is an illustration figure which shows the data structural example of an audio | voice frame. It is an illustration figure which shows the calculation method of the deviation | shift between an audio | voice and an image. It is a flowchart which shows operation | movement of CPU in an image compression task. It is a flowchart which shows operation | movement of CPU in an audio | voice capture | acquisition task. It is a flowchart which shows operation | movement of CPU in an audio | voice compression task. It is a flowchart which shows operation | movement of CPU in an audio | voice reproduction | regeneration task. It is a flowchart which shows operation | movement of CPU in an image reproduction task. FIG. 10 is a flowchart following the flowchart of FIG. 9.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Video camera 16, 20, 26 ... Signal processing circuit 22 ... Video encoder 30 ... SDRAM
32 ... Memory control circuit 34 ... MPEG-4 codec 38 ... Memory card 40 ... CPU
44 ... Flash memory 46 ... RAM

Claims (10)

In a video camera for recording a moving image signal of a subject photographed by an image sensor and an audio signal captured by a microphone on a recording medium,
First image output means for outputting a still image signal of each screen forming the moving image signal to a monitor in a raster scanning manner;
First detection means for intermittently detecting a pixel position scanned by the first image output means; and a pixel position detected by the first detection means is captured by the microphone at the time of detection by the first detection means. A video camera comprising assigning means for assigning to a recorded audio signal. Encoding means for performing an encoding process for each predetermined amount prior to recording of the audio signal captured by the microphone;
2. The video camera according to claim 1, wherein the first detection unit detects the pixel position each time the predetermined amount of the audio signal is captured. Second image output means for outputting a still image signal of each image forming a moving image signal read from the recording medium toward the monitor in the raster scanning mode;
Audio output means for outputting an audio signal read from the recording medium toward a speaker;
Second detection means for detecting a pixel position assigned to the audio signal output by the audio output means;
A discriminating unit for discriminating a deviation between a pixel position scanned by the second image output unit at the time of detection by the second detection unit and a pixel position detected by the second detection unit; and the second image output unit The video camera according to claim 1, further comprising an adjusting unit that adjusts the number of screens of the still image signal output by the unit according to a determination result of the determining unit. The moving image signal is recorded on the recording medium in a state of being encoded screen by screen,
The second image output means includes decoding means for decoding the encoded still image signal of each screen in response to a decoding command;
It said adjusting means controls the generation timing of the decoded instruction, claim 3 Symbol placement of the video camera.   The decoding means writes a still image signal decoded in response to the decoding command to a first memory, and reads the still image signal stored in the first memory from the first memory at a predetermined period. 4. The video camera according to 4.   The video camera according to claim 4 or 5, wherein the adjusting means extends the generation cycle of the decoding command when increasing the number of screens, and shortens the generation cycle of the decoding command when decreasing the number of screens.   5. The second image output means further includes a writing means for writing the still image signal output from the decoding means to a second memory, and a reading means for reading the still image signal stored in the second memory. 7. The video camera according to 6. The second memory includes a first area and a second area;
The writing means alternately writes the still image signal output from the decoding means to the first area and the second area,
The video camera according to claim 7, wherein the reading unit reads the still image signal alternately from the first area and the second area.   The video camera according to claim 8, wherein writing by the writing unit and reading by the reading unit are performed in a complementary manner.   The video camera according to claim 4, wherein the moving image signal is encoded according to an MPEG4 system.

Images