JPH0310119B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a musical note display device that reproduces an audio signal recorded on a recording medium and can display the audio signal as an arrangement of musical notes.

If you can easily obtain music scores for songs recorded on recorded recording media, it would be meaningful when appreciating songs recorded on recorded recording media, and it would also be useful to enjoy music recorded on recorded recording media while playing an instrument. By recording a composed piece of music on a recording medium, there is an advantage that the musical score for the piece of music that has been composed can be easily obtained. It has been desired to develop a musical note display device that can display musical notes at the corresponding position using musical score-like display figures.

By the way, there is a display device that can display the height of an audio signal reproduced from a recorded recording medium using a note-like display figure on a musical staff-like display figure, so that a musical score-like display can be performed on the display. In this case, it is desired to make the playback speed of the recorded recording medium faster than the normal playback speed so that a musical score corresponding to the audio signal on the recorded recording medium can be created within a short time. , when the playback speed of the recorded recording medium is different from the normal playback speed, the pitch of the reproduced audio signal is the original pitch of the audio signal that is being recorded and played back on the recorded recording medium. Even if the playback speed of the recorded recording medium is simply made faster than the normal playback speed, a correct musical score corresponding to the audio signal on the recorded recording medium can be created within a short time. It is not possible.

The present invention performs an FFT calculation using an analog-to-digital conversion means for converting an audio signal reproduced from a recording medium into a digital signal, and a certain number of data of the digital signals obtained by the above-mentioned analog-to-digital conversion means. , a means for performing frequency analysis by performing a power spectrum calculation based on the calculation result, a means for determining the pitch of the sound based on the frequency analysis result, and a means for determining the pitch of the sound based on the result of the frequency analysis; means for displaying the judgment result of the above-mentioned pitch judgment means in a musical note shape in a portion corresponding to the height of the note;
The playback speed of the recording medium is increased so that the pitch of the audio signal with the time axis compressed is the same as the pitch of the audio signal with the time axis not compressed. The above-mentioned problems can be solved by providing a musical note display device comprising means for changing and setting the sampling period in an analog-to-digital conversion means in accordance with the playback speed of a recording medium used for signal playback. Hereinafter, the specific contents of the musical note display device of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is an overall configuration diagram showing the musical note display device of the present invention focusing on its functions, and FIGS. 6 and 7 show the case where the audio signal source is a disk-shaped recording medium, and FIG. 3 is a block diagram of different embodiments of the musical note display device of the present invention in the case of a tape-shaped recording medium; FIG.

The content shown in the overall configuration diagram of FIG. 1 is explained by the explanation of the embodiment of the musical note display device of the present invention shown in FIGS. 6 and 7, which is made with reference to each figure. It will be clearly understood.

SGd in Figure 6 is a disc-shaped recording medium (e.g. disc record, compact disc, etc.)
An audio signal source (for example, a record player) that reproduces an audio signal from a
SGt is a tape-like recording medium (e.g. magnetic tape,
An audio signal source (e.g., tape recorder, PCM encoder, etc.) that reproduces an audio signal from an audio signal source (e.g., tape recorder, PCM encoder, etc.);
The SGd is capable of reproducing audio signals while rotating a disk-shaped recording medium at a specific number of rotations among multiple types of predetermined rotational speeds.
The audio signal source SGt is capable of reproducing the sampling periodic signal recorded in one of the plurality of recording traces on the recording medium as well as the audio signal originally recorded and reproduced on the recording medium. From the audio signal source _SGt in _FIG . The sampling periodic signal outputted from the audio signal source SGt via the line _l2 is
After the waveform is shaped and amplified by the sampling periodic signal generation circuit SPG, it is sent to an analog-digital converter ADC (hereinafter, analog-digital will be referred to as AD) and a central control unit (microcomputer).
given to the CPU.

The audio signal reproduced from the recording medium at the audio signal source SGd shown in FIG. 6, or
The audio signals reproduced from the recording medium at the audio signal source SGt shown in FIG.
It is applied to the ADC, converted into a digital signal by the AD converter ADC, stored in the main memory MM via the central control unit CPU, and used for FFT calculation. , for example, depending on the type of instrument,
By being able to change the frequency characteristics of the input signal,
This is to enable good frequency analysis of various musical instruments, and the anti-aliasing filter AAF is used to limit the frequency band of the signal so that so-called aliasing noise does not occur. It is a low pass filter.

It is well known that if the cutoff frequency of the anti-aliasing filter AAF mentioned above is fc, then the sampling frequency fs in the AD converter ADC should be 2fc or more.
When frequency analysis is performed by FFT calculation using the digital signal output from the converter ADC, the frequency interval f of the spectrum obtained as the analysis result is the frequency interval f used to obtain the digital signal.
It is also well known that f=fs/N depending on the sampling frequency fs of the AD converter ADC and the number N of digital signal data used when performing the FFT operation.

For example, an anti-aliasing filter
When the AAF cutoff frequency fc is 20KHz, the sampling frequency fs in the AD converter ADC is 40KHz (fs =
2fc), and if it is desired to obtain a spectrum at a frequency interval f of 10 Hz as an analysis result by FFT calculation, the number N of digital signal data to be used for FFT calculation is 4000.

In the FFT calculation described above, the more data N used for the calculation, the longer the time required for frequency analysis. In order to shorten the required time, it is sufficient to use components with performance that allows high-speed FFT calculations.
In the embodiment shown in FIG.
Furthermore, in the embodiment shown in FIG. 7, the FFT calculation is performed in an FFT calculation section FFTD, which is composed of a microprocessor dedicated to FFT calculation that can be directly connected to the central control unit CPU.

The operating section OPa in the embodiment shown in FIG. 6 and the operating section OPb in the embodiment shown in FIG. 7 include input switches (for example,
It is equipped with a push button switch or key), but
In the embodiment shown in FIG. 6, the audio signal source SGd has a predetermined sampling period according to the rotational speed of the recording medium.
In order to perform the AD conversion operation in the AD converter ADC, you must operate the input switch for setting the sampling period, and arrange the notes that correspond to the pitch of the audio signal reproduced from the recording medium. The user operates an input switch to select a predetermined item from one page after another (pages of a score) or an input switch for turning the page. In an embodiment, an input switch or page for selecting a predetermined one from successive successive pages of musical scores (successive pages of musical scores) corresponding to the pitch of the audio signal reproduced from the recording medium. The feed input switch is operated.

Now, the musical note display device of the present invention displays the pitch of an audio signal reproduced from a recording medium in a musical note-shaped figure at a corresponding position in a staff-shaped figure, and displays the pitch on the display surface of a display CRT. For example, it is possible to obtain an advantage by reproducing pattern data from a recording medium at a speed higher than the normal reproduction speed, such as the one shown in FIG. It is created based on the input audio signal and stored in memory so that a desired page of musical score or successive pages of musical score can be displayed on the display surface of the display CRT. The pitch of the audio signal is
It is necessary to convert the input audio signal into a digital signal, perform frequency analysis using the digital signal by FFT calculation, and determine the height of the input audio signal.

By the way, the relationship between the strength of overtones relative to the fundamental tone and the way the fundamental tone and overtones are combined differs depending on the type of instrument, so it is difficult to determine the pitch of a sound based on the results of frequency analysis. The following must be done:

That is, a to d in Fig. 5 are diagrams showing the frequency analysis results of sounds of various musical instruments {a in Fig. 5 is the sound of a flute, b in Fig. 5 is the sound of a clarinet, and c in Fig. 5 is The sound of the violin (the G string of the violin), d in Figure 5 is the E string of the contrabass
However, for example, in wind instruments such as the flute and clarinet shown in Figure 5 a and b, the fundamental tone is stronger than any overtone (the piano, guitar, etc. also exhibit the same tendency as wind instruments). ),Also,
Frequency analysis results show that violins and double basses have overtones that are stronger than the fundamental tone.

As can be seen from the frequency analysis results of various musical instrument sounds illustrated in a to d of FIG. 5, the frequency components that make up the musical instrument sounds are Not only are there cases where the fundamental tone is the strongest, but there are also cases where the overtones are stronger than the fundamental tone, as shown in Figure 5 c and d.
Even if we frequency-analyze the input audio signal and extract the largest frequency component among the frequency components obtained as a result of the analysis, it does not necessarily mean that it is the fundamental tone, so we analyzed the frequency of the input audio signal. In order to determine the pitch of the fundamental tone of the input audio signal based on the result, and to display the note corresponding to the pitch of the fundamental tone at a predetermined position on the staff on the display surface, the pitch of the input audio signal must be determined. It is necessary to determine the fundamental tone of the input audio signal by looking at the relative relationship between the intensities of each frequency component obtained as a result of frequency analysis.

Therefore, in the musical note display devices shown in FIGS. 6 and 7, the input audio signal is converted into a digital signal by an AD converter ADC, and after performing FFT calculation using the digital data obtained, the power spectrum is After performing the calculation and obtaining each frequency component of the input audio signal, the frequency of the frequency component showing the largest spectral value among all the frequency components obtained by the frequency analysis means is Among frequency components existing in a low frequency region, a frequency component exhibiting a smaller spectral value within a predetermined range than the spectral value of the frequency component exhibiting the largest spectral value is detected, If the corresponding frequency component is not detected by the detection means, the frequency of the frequency component showing the largest spectral value as described above is
The pitch (fundamental tone) of the input audio signal is determined, and if a frequency component is detected by the detection means, the frequency component indicating the lowest frequency among the detected frequency components. The fundamental tone of the input audio signal is determined by determining the frequency of the input audio signal as the pitch (fundamental tone) of the input audio signal.

The fact that the fundamental tone of an input audio signal can be correctly found by the above-described method of determining the fundamental tone of an input audio signal will be explained with reference to a to d of FIG. 5.

If the frequency analysis results for the input audio signal indicate that the fundamental tone is the strongest, as shown in a and b in FIG. 5, it is natural that the fundamental tone is determined to be the fundamental tone of the input audio signal. In other words, the fundamental tone has the lowest frequency among the many frequency components obtained as a result of frequency analysis, so in this case, the frequency region lower than the fundamental tone, which has the largest spectral value, contains the frequencies generated by frequency analysis. component does not exist, and therefore, among the many frequency components obtained as a result of frequency analysis, the frequency value of the fundamental tone that exhibits the largest spectral value is determined as the fundamental tone.

In addition, if the result of frequency analysis of the input audio signal is that the spectral value of the overtone is larger than the spectral value of the fundamental tone, as shown in c and d in Figure 5, the frequency analysis result for the input audio signal is The largest of the many resulting frequency components {violin G shown in Figure 5c
In the case of a line, the third harmonic (3rd harmonic), d in Figure 5
The spectrum of the frequency component that exhibits the largest spectral value among the frequency components that exist in a frequency region lower than the frequency of the double overtone (second harmonic) in the case of the double bass E line shown in The frequency of a frequency component that exhibits a smaller spectral value within a predetermined range than the fundamental tone is determined as the fundamental tone.

The above-mentioned "predetermined range" in the case of the G string sound (open string sound) of the violin shown in Figure 5 c may be about 10 dB, and the contraband range shown in Figure 5 d The above-mentioned "predetermined range" in the case of the sound of the E string of the bass (open string) may be about 30 dB.

In the case where the frequency analysis result of the sound of a normal musical instrument shows that the spectral value of the overtone is larger than the spectral value of the fundamental tone, the spectral value of the harmonic with the largest spectral value and the spectral value of the fundamental tone are calculated. The level difference is within 10 dB, and the sound of the E line of the contrabass shown in d in FIG. 5 is an exception.

Therefore, when determining the fundamental tone, the frequency value of the frequency component with the largest spectral value is, for example, 100
Hz or less, there is a frequency component in the frequency region lower than the frequency of the frequency component with the largest spectral value that shows a level difference of, for example, within 30 dB compared to the spectral value of the frequency component with the largest spectral value. In cases other than the above, that is, the frequency value of the frequency component with the largest spectral value is, for example,
In the case of 100Hz or more, there is a frequency component in a frequency region lower than the frequency of the frequency component with the largest spectral value that shows a level difference of, for example, within 10 dB compared to the spectral value of the frequency component with the largest spectral value. If we check whether there is a fundamental tone in the input audio signal, we can always correctly judge the fundamental tone in the input audio signal using the above-mentioned judgment criteria.

By the way, if the note display device is, for example, note name F ₂
If the device is configured to display pitches up to _F5 , the highest pitch _F5 among the pitches to be displayed as described above will be input. Even when the pitch name is F5, the display device must display a display corresponding to the note name _F5 . However, as mentioned above, when determining the pitch (fundamental tone) of the input audio signal, frequency analysis is used. The resulting spectral value or frequency value of the frequency component that shows the largest spectral value is required, so the note name _F5 note in the note display device is based on the result of frequency analysis of the input audio signal. In order to be judged,
The high-order harmonic (at least third-order harmonic) component of the fundamental tone frequency 698.46 Hz of pitch name F ₅ must also be obtained as a correct value through frequency analysis.

Therefore, in a display device that also displays the pitch name _F5 , as in the example described above, anti-aliasing technology that limits the frequency band of the input audio signal is used.
As a filter AAF, its cutoff frequency fc is
If the frequency value of pitch name F ₅ is set to at least three times the frequency value of 698.46Hz, and if the playback speed of the audio signal from the recording medium is K times the normal playback speed, then , it goes without saying that the cut-off frequency fc described above should be further K times the value described above.

When configuring an actual device, it is desirable to use an anti-aliasing filter AAF whose cutoff frequency fc is set as high as other conditions allow (the anti-aliasing filter AAF's cutoff frequency Frequency fc and AD converter
The sampling frequency fs in the ADC, the frequency interval f of the spectrum obtained as a frequency analysis result,
The relationship with the number of pieces of data N, the relationship with the time required for FFT calculation, etc. have already been described).

In Figures 6 and 7, the digital signal output from the AD converter ADC is the main memory MM.
As mentioned above, it is stored in the central controller CPU and used for FFT calculations.
performs predetermined control operations according to the program stored in the read-only memory in the main memory MM, and also performs calculation operations in the FFT calculation unit FFTA and FFT calculation unit FFTD, as well as the video display processor VDP. Via
Provides data to the video RAM VRAM.

The CRT in Figures 6 and 7 is a display. In the following explanation, the display uses a cathode ray tube, and the display screen of the display CRT shows the pitch of the input audio signal. The display is
For example, this is done as shown in FIG.

Video display processor VDP is
It operates as an interface between the video ram V-RAM connected to it via the data bus 4 and the central control unit CPU, and also operates as an interface between the video ram V-RAM and the central control unit CPU. Therefore, the video display processor VDP is configured such that the image content is determined and can generate a composite video signal according to a predetermined standard method. , Nikkei McGraw-Hill, “Nikkei Electronics” March 30, 1981
It is possible to use the video display processor (VDP) manufactured by Texas Instruments, Inc., introduced in the Japanese issue (pages 156 to 164).・A processor is said to be used.

Figure 2 shows a video ram V.
FIG. 2 is a diagram showing an example of a memory map of RAM,
In the video RAM memory map shown in Figure 2, 1024 bytes from addresses 0 to 1023 are used as the sprite generator table SGT, and 768 bytes from addresses 1024 to 1791 are used as the pattern name table PNT. used,
Furthermore, 128 bytes from address 1792 to address 1919 are used as the sprite attribute table SAT,
Furthermore, 32 bytes from address 1920 to address 1951 are used as color table CT, and
96 bytes from addresses 1952 to 2047 are unused, and 2048 bytes from addresses 2048 to 4095 are used as the pattern generator table PGT.

The pattern generator table PGT is
For example, one specific pattern can be stored in 8 pixels x 8 pixels using 8 bytes each, so 8 x
It can store 256 types of patterns with 8 pixels.
The pattern information stored in this pattern generator table PGT is stored in the central controller
The pattern generator table is transferred from the read-only memory ROM in the initial state of the device by the operation of the CPU.
Of course, the PGT may be a read-only memory.

As mentioned above, each 8-byte storage area in the pattern generator table PGT individually stores specific patterns of 8×8 pixels. To enable a specific pattern to be specified by a pattern name given to each storage area in which a specific pattern is stored. Pattern generator table PGT of the example shown in the second diagram
Now, 256 types of patterns can be specified using 256 pattern names from pattern name #1 to pattern name 255.

Next, the pattern name table PNT is used to store information indicating which pattern name in the pattern generator table PGT each of the display sections assumed on the display surface of the display CRT corresponds to. It has a storage capacity corresponding to the total number of displayed display sections.

In the example shown in Figure 3, the total number of sections set on the display screen is {(32 columns) x (24 rows)} = 768, and since 1 byte is used as the amount of information indicating one section, as described above Pattern name table like PNT
is said to have a storage capacity of 768 bytes.

As mentioned above, the required number of patterns are stored in the pattern generator table PGT in the video RAM VRAM, and the required pattern names assigned to each pattern are as follows. Pattern name table PNT
When the video display processor VDP is stored in each section of the display screen in
Information stored in the pattern name table PNT in the video RAM VRAM described above,
Generates a composite video signal according to a specific standard format whose image content is defined by the information stored in the pattern generator table PGT and, if necessary, the information stored in the color table CT. A specific pattern is displayed on the display surface of the display CRT.

The discussion so far has focused on pattern generators and
The specific pattern stored in the table PGT is 768 on the display screen.
This relates to the case where a pattern is displayed in a display mode in which it is displayed on a specific section of a section, a so-called graphic mode. Since the position of the pattern is specified by the table PNT, when an attempt is made to move one pattern on the display screen, the pitch of pattern movement is one section (distance of 8 pixels) on the display screen.

Therefore, in order to reduce the pitch of movement of the pattern on the display surface and make the pattern move smoothly, it is necessary to use the sprite generator table.
The pattern stored in the SGT is moved within the display screen by one pixel pitch by changing the coordinates.

The patterns stored in the sprite generator table SGT may be 8 pixel x 8 pixel sprite data or 16 pixel x 16 pixel sprite data.
Each pattern stored in the sprite generator table SGT is individually assigned a sprite name such as #0, #1...#N, but the sprite surface corresponding to the pattern with each sprite name is The smaller the numerical value indicated by the sprite name, the higher the priority.

In the memory map of the video RAM VRAM illustrated in FIG. 2, 1024 bytes from address 0 to address 1023 are assumed to be used as the sprite generator table SGT, as described above. In this example, if the pattern is 8 pixels x 8 pixels, 128 patterns (sprite names #0 to #127) can be stored, and if the pattern is 16 pixels x 16 pixels, 32 Patterns (sprite names #0 to #31) can be stored. It goes without saying that if 2048 bytes are allocated to the sprite generator table SGT in the video RAM VRAM, the number of patterns stored in the sprite generator table SGT will be twice that of the previous case. Not even.

The sprite attribute table SAT uses 4 bytes for each sprite, and contains the sprite's position (1 byte each for specifying vertical and horizontal positions), display sprite name (1 byte), color code, and Since the end code (1 byte) of the displayed sprite is set, if 128 bytes are used as the sprite attribute table SAT, information for 32 sprites is stored in this sprite attribute table SAT.

The position of the sprite is determined by 256 pixels (8 pixels x 32 sections) in the horizontal direction (X direction) and 192 pixels (8 pixels x 24 sections) in the vertical direction (Y direction) on the display screen.
49152 The coordinates of the pixel, the vertical position (a number that indicates the number of pixels in the vertical direction), and the horizontal position (the number that indicates the number of pixels in the horizontal direction) are stored in the sprite attribute table. It is determined by writing to SAT (the base point of the sprite is set to the upper left corner of the sprite), and the movement of the sprite is 1
This can be done at pixel pitch.

In the musical note display device of the present invention, the pattern generator table PGT and the sprite
Multiple types of patterns are stored in the generator table SGT, and the selection of the pattern to be displayed on the display surface of the display CRT and the specification of pattern movement are performed in the pattern name table PNT and sprite attribute table SAT. Based on the written data, a display as shown in FIG. 4, for example, is displayed on the display surface of the display CRT.

In FIG. 4, which shows the display mode on the display surface of a display CRT, figures such as staff notation, treble clef, bass clef, and other musical symbols are prepared by being stored in advance in read-only memory in main memory MM. At the start of the operation of the musical note display device, various patterns stored in the read-only memory in the main memory MM are stored in the central controller CPU and the video display processor VDP. The pattern in the video RAM VRAM is
Generator table PGT and sprite
It is transferred to and stored in the generator table SGT and used for display operations on the display surface of the display, and the musical note-shaped mark S shown on the musical staff in FIG. The display corresponds to the pitch of the audio signal.
This is what is displayed on the CRT display screen.

The central control unit CPU performs the operations shown in the flowchart shown in Figure 8, creates the data necessary to display the pitch of the input audio signal, and stores the data one after another in the main memory. Store it in MM (or video RAM V RAM),
When displaying music scores on the display CRT,
The above-mentioned main memory MM (or video RAM V RAM) is used to display successive pages of the musical score or desired pages on the display surface of the display.
The data stored in the memory is applied to the video display processor VDP and the video RAM VRAM, so that the pitch of the sound is displayed on the display screen using musical notes as shown in the fourth figure.

In the flowchart shown in Fig. 8, the power is turned on at the start to start the musical note display device, initialization (system initialization) is performed at step 1, and random data is stored in the AD converter ADC and the main memory MM. access memory,
The video, RAM, VRAM, etc. are cleared, and the video, display, processor, VDP, etc.
The registers in the main memory are set, and the storage area in the video RAM VRAM is used for which table, the operating mode is set, etc.
From read-only memory in MM to pattern/generator/table PGT, sprite/generator/table SGT, etc.
Transferring predetermined types of pattern information (for example, various graphic information shown in FIG. 4) via the video display processor VDP, and
Sprite names, X and Y coordinates, and color data are transferred to the sprite attribute table SAT.

The central control unit CPU executes each step from step 2 to step 7, or steps 2, 3, 8 to 12, unless an interrupt {step 13 or steps 13A to 15} shown in the flowchart of FIG. 8 occurs. Although each control operation in each step is repeatedly executed, in the embodiment shown in FIG. When an interrupt occurs at each sampling period set by the input switch of the operation unit OPa, that is, at a specific sampling period set in the counter provided inside the central control unit CPU, In the embodiment shown in FIG. 7, when an interrupt occurs every period of the sampling period signal generated by the sampling period signal generation circuit SPG based on the sampling period signal obtained from the audio signal source SGt, the central The control device CPU interrupts the control operation that was being executed at that time, performs the control operation shown in step 13 or steps 13A to 15, and when the control operation is completed, the control operation that was previously interrupted due to the occurrence of an interrupt is executed. Continue execution of the control action.

That is, the central control unit CPU performs the control operations of steps 2 to 7, or steps 2, 3, and 8 to 12 at times other than when it performs control operations for the AD converter ADC.

Now, when an interrupt occurs every sampling period of the AD converter ADC, the central control unit CPU
A start pulse for the AD conversion operation is given to the ADC, and in step 13 or 13A, the AD converter ADC performs the AD conversion operation and stores the output digital signal in the main memory MM.

In the flowchart shown in Figure 8, two types of interrupt routines, A and B, are shown because a predetermined number of digital signals used for FFT calculation are sequentially processed on the time axis. When the oldest digital signal is discarded every time the latest digital signal is obtained (as is the case with routine A), the signal used for FFT calculations is When a predetermined number of digital signals are all new (in the case of routine B), which of the two digital signals are used to perform the FFT operation? Because it may be done.

In interrupt routine A, audio signal sources SGd, SGt
In step 13, the audio signal is
AD conversion is performed by the AD converter ADC, and in the interrupt routine B, the audio signal is converted to
AD conversion is performed by the AD converter ADC, and then in step 14 it is checked whether the AD conversion operation in the AD converter ADC has been performed a predetermined number of times (whether a predetermined number of digital data has been obtained). If YES, proceed to step 15, change the address where the digital data is stored, and return.

In step 2, perform a switch scan,
Check the input conditions set on the operation unit OPa (or OPb) and proceed to step 3. In step 3,
It is checked whether or not to create pitch data (musical score data) for the audio signal. If YES, proceed to step 4; if NO, proceed to step 8.

In step 4, using the predetermined number n of digital data stored in the main memory MM,
An FFT calculation is performed, and n/2 spectrum data resulting from the calculation are stored in the main memory MM.

FFT calculation is an FFT calculation unit that operates at high speed.
This is performed by the FFTA (Figure 6) or the FFT operation unit FFTD (Figure 7) which operates at high speed.

In step 5, a power spectrum calculation is performed on the spectrum data obtained in step 4, and the calculation result is stored in the main memory MM. Next, in step 6, pitch analysis is performed.
That is, in step 6, first find the largest spectral value, and then find a spectrum that exists in a frequency region lower than the frequency of the spectrum showing the largest spectral value and is larger than the largest spectral value. also determines the frequency value of the spectrum that shows the lowest frequency value among the spectra that have small spectrum values within a predetermined range, and determines this as the pitch (fundamental tone) of the input audio signal, and If there is no spectrum that satisfies the above conditions, the frequency value of the spectrum having the largest spectral value is determined to be the pitch (fundamental tone) of the input audio signal, and the process proceeds to step 7.

In step 7, the pitch data obtained in step 6 is stored in the main memory MM (or video ram V.
Store in RAM. Storing the pitch data in a free storage area of the video RAM VRAM is a desirable implementation since a low-cost dynamic RAM is used as the video RAM VRAM. It is a mode.

The AD conversion operation of the audio signal performed by the interrupt routine A or B, and the operation of creating the pitch data of the audio signal and the storage operation in the memory performed in steps 2 to 7 are performed by the audio signal source SGd. ,
This is done quickly for audio signals that are reproduced at high speed from a recording medium in SGt.

In the musical note display device of the present invention, the audio signal reproduced at high speed from the recording medium is AD converted at a sampling period determined according to the reproduction speed of the recording medium, so that the digital signal is The pitch data of the audio signal obtained based on the result of FFT calculation using is the same as the pitch data obtained from the audio signal reproduced from the recording medium at normal playback speed, In the musical note display device of the invention, the playback speed of the recording medium in the audio signal sources SGd and SGt is increased, and the pitch data of the audio signal can be stored one after another in the memory as a musical score for each page in a short time. can.

When all the pitch data for the audio signals from the audio signal sources SGd and SGt have been stored in the memory, the determination result in step 3 becomes NO, and the process proceeds from step 3 to step 8. Check the feed and if NO, return to step 2,
Further, if the determination result in step 8 is YES, the process advances to step 9.

In step 9, check whether the page feed is plus 1. If YES, proceed to step 11, and if NO, proceed to step 11.
If so, proceed to step 10. In step 10, it is checked whether the page turn is minus 1 or not, and if YES, the process advances to step 11.

In step 11, data to be written in the pattern name table PNT and data to be written in the sprite attribute table SAT are created in correspondence with the data value indicating the pitch read from the memory, and then in step 12, the data to be written in the sprite attribute table SAT is created. video data
The data is transferred to the video RAM VRAM via the display processor VDP, and the video display processor VDP creates a composite video signal using the data written to the video RAM VRAM as described above. and display it
The pitch is sent to the CRT, and the pitch is displayed on the display screen of the CRT in the form of an arrangement of musical notes, for example, in a pattern as shown in FIG. That is, the fourth
The illustrated display shows one page of the score, but the display content changes to display the next page of the score each time the page turn switch (input switch) on the operation sections OPa and OPb is operated. .

In the embodiment shown in FIG. 6, the recording medium in the audio signal source SGd is a disk-shaped recording medium rotating at a constant angular velocity, and the reproduction speed of the recording medium in the audio signal source SGd (relative line between the recording medium and the reproduction element The operation unit OPa sets the sampling period for AD conversion operation according to the playback speed of the recording medium at the audio signal source SGd, depending on how many times the normal playback speed of the recording medium is. In contrast, in the embodiment shown in FIG. 7, the audio signal source SGt sends out a sampling period signal whose period changes in accordance with the playback speed of the recording medium. In the embodiment shown in FIG. 7, it is not necessary to set the sampling period for the AD conversion operation in the operation unit OPb.

As is clear from the detailed explanation above, in the musical note display device of the present invention, an audio signal that is reproduced at high speed so that it can be reproduced in a short time by an audio signal source is converted into a digital signal and subjected to FFT calculation. and perform pitch judgment to determine the pitch of the audio signal.
This is to display an arrangement of musical notes, but the sampling period of the AD conversion operation for the audio signal is changed at the same rate as the time axis compression rate of the audio signal reproduced by the audio signal source. Since this is set, the pitch of the audio signal with the time axis compressed is made the same as the pitch of the audio signal with the time axis not compressed.

That is, the frequency components of the FFT obtained by the fast Fourier transform FFT operation used in the configuration of the musical note display device of the present application were used to generate the digital signal on which the FFT operation was performed.
The following formula f=
fs/N {f is the frequency interval of frequency components obtained by FFT calculation, and the frequency components obtained by FFT calculation are as follows, assuming that the number N of digital signal data used for FFT calculation is constant: A.D.
and the musical note display device of the present invention. Since the data of the digital signal used for FFT calculation is always constant, by changing the sampling frequency fs (the sampling period is the reciprocal of the sampling frequency fs) used for AD conversion, , the frequency component obtained by the FFT operation changes according to the sampling frequency fs used for AD conversion (the sampling period is the reciprocal of the sampling frequency fs). In the display device, even if the playback speed of the recording medium is increased and the pitch of the playback signal increases as described above, the pitch obtained from the frequency analysis results does not become high. The sampling frequency used for AD conversion of the audio signal at the same rate as the time axis compression rate of the audio signal
fs (the sampling period is the reciprocal of the sampling frequency fs) and using a certain number of data of the digital signal obtained from the AD conversion operation described above.
The frequency component obtained by FFT calculation is changed according to the sampling frequency fs used for AD conversion (the sampling period is the reciprocal of the sampling frequency fs),
The pitch of the audio signal when the time axis is compressed by increasing the playback speed of the recording medium can be displayed in the same state as the pitch of the audio signal when the time axis is not compressed. Therefore, with the musical note display device of the present invention, even a long piece of music can be stored in the memory as successive pages of musical score data within a short time, and the musical scores stored in the memory can be displayed one after another on a display CRT. Can be done. So,
By using the musical note display device of the present invention, it is possible to quickly create and display the musical score of a song recorded on a recorded recording medium.

[Brief explanation of drawings]

Fig. 1 is a functional block diagram showing the overall configuration of the musical note display device of the present invention, Fig. 2 is an example of a memory map of a video ram, Fig. 3 is an explanatory diagram of the divisions of the display surface, and Fig. 4 is a display. A top view of the surface display pattern,
FIG. 5 is a spectral distribution diagram of musical instrument sounds, FIGS. 6 and 7 are block diagrams of different embodiments of the musical note display device of the present invention, and FIG. 8 is a flowchart. SGd, SGt...Audio signal source, SPG...Sampling period signal generation circuit, OPa, OPb...Operation unit, FFTA
...FFT calculation unit, FFTD...FFT calculation unit, MM
...Main memory, GEQ...Graphic equalizer, AAF...Anti-aliasing
Filter, ADC...AD converter, CPU...Central control unit, VDP...Video display processor, VRAM...Video RAM, CRT...
...display.

Claims (1)

[Claims] 1. Analog-to-digital conversion means for converting an audio signal reproduced from a recording medium into a digital signal;
FFT is performed using a certain number of digital signal data obtained by the analog-to-digital conversion means described above.
means for performing calculations and power spectrum calculations based on the calculation results to perform frequency analysis; means for determining the pitch of a sound based on the frequency analysis results; and displaying a musical staff on the display surface of the display. Means for displaying the judgment result of the above-mentioned sound pitch judgment means in a musical note shape in a part of the figure corresponding to the pitch of the sound; and means for compressing the time axis by increasing the playback speed of the recording medium. The playback speed of the recording medium used to play back the audio signal was set so that the pitch of the audio signal in the state where the time axis was not compressed was displayed in the same state as the pitch of the reference sound signal in the state where the time axis was not compressed. A musical note display device comprising: means for changing and setting a sampling period in an analog-to-digital converting means according to 2. Analog-to-digital conversion means for converting the audio signal reproduced from the recording medium into a digital signal;
FFT is performed using a certain number of digital signal data obtained by the analog-to-digital conversion means described above.
means for performing calculations and power spectrum calculations based on the calculation results to perform frequency analysis; means for determining the pitch of a sound based on the frequency analysis results; and displaying a musical staff on the display surface of the display. means for displaying the judgment result of the above-mentioned sound pitch judgment means in a note-shaped figure in a part of the figure corresponding to the pitch of the sound; The pitch of the audio signal with the time axis compressed by increasing the playback speed of the recording medium is the same as the pitch of the audio signal with the time axis not compressed. and means for setting the sampling period in the analog-to-digital conversion means using the sampling period signal obtained by the reproduction means so that the musical notes can be displayed in the same state as the height of the note.