JP5248265B2 - Music score recognition apparatus and computer program - Google Patents

Description

  The present invention relates to a score recognition apparatus and a computer program, and is particularly suitable for use in recognizing a score on paper.

Conventionally, a score on a sheet is read using a scanner or the like, and staff data, musical notes, and other various symbols are recognized from the read score data, and performance data such as MIDI file data, display / Generation of data for printing is performed.
Specifically, when recognizing various musical symbols, the staff and paragraphs are recognized, and then the staff is erased, and then a reading label having an arbitrary rectangular shape is set for each symbol. The label data and the data registered in the dictionary in advance are subjected to pattern matching (see Patent Document 1). Here, in the case of a monochrome image (monochrome binary image), a label refers to a lump of black pixels that are four-connected or eight-connected, for example.

Japanese Patent Laid-Open No. 11-161738

By the way, in addition to the so-called staff notation, there are tablatures. In tablature, numbers are generally written on a horizontal line corresponding to a string (in the following description, this line is also called a staff if necessary). Therefore, it is necessary to be able to accurately recognize such tablature.
However, the staff is erased when recognizing the musical score as described above. Therefore, the number written on the staff of the tablature may be different from the original form by erasing the staff. Then, for example, even if the numbers in the plurality of labels to be recognized are different, the difference between them becomes small, and there is a possibility that they are erroneously recognized as the same number.
As described above, according to the conventional technique, if the staff is erased, there is a possibility that the numbers written on the tablature cannot be accurately recognized.

  The present invention has been made in view of such a problem, and is capable of recognizing the numbers written on the tablature more accurately than before even when the staff is erased. Objective.

  The score recognition apparatus of the present invention erases a horizontal line corresponding to a chord in a tablature from an acquisition unit that acquires an image including sheet music score information from an image reading unit and an image acquired by the acquisition unit. An erasing unit, a setting unit that sets a label for a graphic included in a tablature image among images acquired by the acquiring unit after the horizontal line is deleted by the erasing unit, and a setting by the setting unit A recognition unit for recognizing to which number the figure belongs by matching a figure included in the labeled label with a preset dictionary, and reading at least some pixel values of the pixels in the label Which of the reading means and the condition that the pixel value read by the reading means is set in advance according to the number recognized by the recognition means By another, characterized by having a discriminating means for discriminating whether belongs to which numbers figures included in the label.

  The computer program according to the present invention includes an acquisition step of acquiring an image including information on a sheet music score from an image reading unit, and an erasure that deletes a horizontal line corresponding to a string in a tablature from the image acquired by the acquisition step. A setting step for setting a label for a graphic included in a tablature image among the images acquired by the acquisition step after the horizontal line is deleted by the deletion step, and the setting step. A step of recognizing which figure the figure belongs to by matching a figure contained in the label with a preset dictionary and reading at least some pixel values of the pixels in the label And the pixel value read by the reading step is recognized by the recognition step. By determining whether to correspond to any of the preset condition depending on the number, characterized in that to execute a determining step of determining whether belong to which numerical figures included in the label to the computer.

  According to the present invention, the figure included in the label is assigned to which number by determining which of the preset conditions the pixel value of the pixel in the label corresponds to the number recognized by matching. It was made to determine whether it belongs. Therefore, erroneous recognition of numbers in the tablature can be reduced more than before, and the tablature can be recognized more accurately than before even if the horizontal lines corresponding to the strings (so-called staves) are deleted.

(First embodiment)
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram illustrating an example of a configuration of a score recognition apparatus.
In FIG. 1, the score recognition apparatus 100 includes a CPU 1, ROM 2, RAM 3, HDD 4, FDD 5, MIDI interface circuit 6, scanner interface circuit 7, printer interface circuit 8, keyboard interface circuit 9, CRT interface circuit 10, and bus 15. doing. The scanner 11, printer 12, keyboard 13, and CRT 14 are connected to the score recognition apparatus 100.

As shown in FIG. 1, a score recognition apparatus 100 of this embodiment is obtained by adding a MIDI interface circuit 6 to a general computer system such as a personal computer.
The CPU 1 is a central processing unit that performs overall control of the score recognition apparatus 100 by developing a program stored in the ROM 2 on the RAM 3 and executing the program. The musical score recognition apparatus 100 has a built-in timer circuit that interrupts the CPU 1 at a predetermined cycle set in advance. The RAM 3 is used as an image data buffer, a work area, etc. in addition to a program area. An HDD (hard disk drive) 4 and an FDD (flexible disk drive) 5 store programs, image data, performance data, and the like.

  The CRT 14 displays video information output from the CRT interface circuit 10 based on the control of the CPU 1. Information input from the keyboard 13 is taken into the CPU 1 through the keyboard interface circuit 9. The printer 12 prints print information output from the printer interface circuit 8 based on the control of the CPU 1.

  The scanner 11 optically scans a score printed on paper and converts it into binary, gray scale, or color image data. As the scanner 11, an arbitrary type such as a flat bed type, a handy type, or a feeder type can be used. Image information read by the scanner 11 is taken into the RAM 3 or the HDD 4 via the scanner interface circuit 7. The MIDI interface circuit 6 is a circuit that transmits / receives MIDI data to / from an external MIDI device such as a sound module. The bus 15 connects each circuit in the score recognition apparatus 100. In addition, the score recognition apparatus 100 may include a pointing device such as a mouse, a serial interface circuit such as RS232C, and the like.

FIG. 2 is a main flowchart for explaining an example of processing of the CPU 1. It should be noted that here, description will be made mainly on matters necessary for describing the embodiment of the present invention, and detailed explanation of other matters will be omitted.
First, in step S1, the CPU 1 performs an image capturing process. In this image capturing process, a process of capturing image data of a score read by the scanner 12 into the RAM 3 is performed.
Next, in step S2, the CPU 1 performs a staff recognition process. In the staff recognition processing, staff scanning start position detection processing, staff shift amount detection processing, and the like are performed. The outline of the staff scanning start position detection process is as follows. At a certain position in the horizontal (x) axis direction, the width of the black pixel and the width of the white pixel are obtained in order, and arranged in a staff based on the obtained width Detecting the position where it is coming in consideration of some error. Then, in order to remove the influence of the additional line (the horizontal line added to describe the note protruding from the staff), the condition that there is a white pixel width larger than the distance between the staffs on both sides of the staff array. Add. The midpoint of each black run in the horizontal (x) direction where black-and-white pixels are arranged that meet this condition is set as the staff scanning start position. Here, in the case of a monochrome image (monochrome binary image), the run means, for example, a length in which black pixels or white pixels are connected vertically or horizontally.

  Next, an outline of the staff shift amount detection process will be described. First, from the staff scanning start position (5 black pixel positions), the position is changed to the left and right one dot at a time, and when the number of black pixels becomes less than a certain number (for example, 3 or 4) out of 5 points In addition, the number of black pixels is checked by shifting the five points up and down, and the y coordinate is shifted in a direction in which the ratio of black pixels increases in the vertical (y) direction. The shift amount from the staff scanning start position is defined as the staff shift amount. Then, the staff is detected by scanning to the position where the number of black pixels becomes zero in the left-right direction (horizontal direction) of the staff scanning start position.

  Next, in step S3, the CPU 1 performs a paragraph recognition process. In the paragraph recognition process, a paragraph recognition process and a bracket recognition process are performed. In the paragraph recognition process, the staff is detected in the entire image, a set of staffs in which the left ends of the staffs are almost in the same place is searched, and it is inspected whether the ends of the staff are connected with black pixels. And recognize paragraphs. When staffs are connected with brackets, there may be notes that cross between staffs, so it is better to process staffs connected with brackets in one unit. . The brackets are recognized within a predetermined range to the left of the paragraph line. Here, the paragraph line refers to a line that connects the left ends of the staffs of a plurality of parts that are proceeding simultaneously, and the paragraph refers to a lump of a plurality of staffs that are connected by paragraph lines. When there is one part, a block corresponding to a plurality of part paragraphs is called a paragraph. Also, the brackets are music symbols that are often used for piano scores, in particular, parentheses indicated by “[” for collecting two or three staffs.

Next, in step S4, the CPU 1 performs part recognition processing. In the part recognition process, each part is recognized based on the results recognized in the staff recognition process and the paragraph recognition process, and it is determined whether each part is a tablature. If the result of this determination is a tablature, processing rectangle determination processing, staff erasing processing, music symbol recognition processing, recognition result verification processing, and the like are performed. On the other hand, in the case of other than tablature, music symbols in each part are recognized by a known method.
In the process rectangle determination process, a rectangle that is fairly wide including the found staff is taken and is used as a recognition process rectangle. The size of the rectangle is not less than the maximum area where the music symbols related to the staff are present, and is adjusted so that the necessary music symbols are not erased by correcting the staff inclination. Subsequent recognition is performed within this rectangle.

In the staff erasing process, a portion of the obtained staff whose line width is equal to or smaller than a threshold (a portion not overlapping with a music symbol) is erased.
In the music symbol recognition process, a rectangular label is set so as to touch the outer edge of continuous black pixels for the musical score image data from which the staff has been deleted, and the black pixel data in the rectangular label is registered in the dictionary in advance. Various music symbols are recognized by performing pattern matching with the recorded data. By this music symbol recognition processing, the numbers written on the tablature are recognized. In pattern matching, in addition to template matching, the size of a rectangular label and a weight direction index can be used, or a peripheral feature can be used.
In the recognition result verification process, it is verified whether or not the number recognized in the music symbol recognition process is correctly recognized. Details of this recognition result verification processing will be described later.

Next, in step S5, the CPU 1 performs a combination process. In the merging process, the score recognized by the staff recognition process, the paragraph recognized by the paragraph recognition process, the contents of the part recognized by the part recognition process, etc. are combined to generate music score data. Do.
Next, in step S6, the CPU 1 performs other processing. In other processing, processing such as generating performance data such as MIDI file data is performed.
As described above, actually, various processes are executed in addition to the above process, but processes other than the recognition result verification process in step S4 can be realized by a known technique. Detailed description is omitted.

Next, an example of the recognition result verification process will be described with reference to the flowchart of FIG.
First, in step S11, CPU1 discriminate | determines what the number recognized as having been described on the staff of a tablature by a music symbol recognition process. In this embodiment, “2”, “3”, “5”, and “7” are assumed to have high recognition reliability by the music symbol recognition process (pattern matching), and the recognized number is “2”. "," 3 "," 5 "or" 7 ", the processing according to the flowchart of FIG. 3 is not performed.
As a result of the determination, if the recognized number is “1” or “4”, the process proceeds to step S21 described later.
On the other hand, if the recognized number is “0”, “6”, “8” or “9”, the process proceeds to step S12. In step S12, the CPU 1 performs image scanning processing. In the image scanning process, the pixel values of the pixels in the rectangular label set in the music symbol recognition process are sequentially read, and based on the read pixel values, whether or not there is a gap in the graphic in the rectangular label, If there is no gap, the distance between the inner edges of the numeral in the horizontal direction is detected. Details of the image scanning process will be described later.

FIG. 4 is a diagram illustrating a first example of a graphic in the rectangular label 401. Specifically, FIG. 4A shows “8”, FIG. 4B shows “0”, and FIG. 4C shows “6”.
As described above, in this embodiment, since the music symbol recognition process is performed after the staff erasure process is performed, as shown in FIGS. 4A and 4C, “8”, Of “6”, the portion that overlaps the staff may be a white pixel. Therefore, for example, as shown in FIGS. 4A to 4C, the difference between “0”, “6”, and “8” becomes small. In the above-described pattern matching, these are accurately distinguished and recognized. May be difficult to do. Therefore, in this embodiment, the number recognition rate is improved by using the gap between the figures in the rectangular label 401 and the distance between the inner edges of the figure.

Returning to the description of FIG. 3, in step S <b> 13, the CPU 1 determines whether there is a gap in the graphic in the rectangular label 401 based on the result of the image scanning process in step S <b> 12. In the example shown in FIG. 4, it is determined that there is no gap in FIGS. 4A and 4B, and there is a gap in FIG. 4C.
As a result of the determination, if there is no gap, the process proceeds to step S17 described later. On the other hand, if there is a gap, the process proceeds to step S14. In step S14, the CPU 1 determines whether or not the gap length is equal to or greater than a threshold value. As a result of this determination, if the gap length is not equal to or greater than the threshold, it is determined that the gap is caused by a factor such as blurring, and that the number is not originally a gap, and will be described later in step S17. Proceed to

  On the other hand, if the gap length is greater than or equal to the threshold, the process proceeds to step S15. In step S15, the CPU 1 reads “the position of the gap in the rectangular label 401” stored in the RAM 3 by the image scanning process. Further, the HDD 4 stores a table in which the gap positions and numbers are associated with each other, and the CPU 1 collates the read gap positions with the table to determine the numbers. For example, when the gap is located only in a predetermined area on the upper right side, it is determined as “6”. Further, when the position of the gap is only in a predetermined area on the lower left side, it is determined as “9”.

And it progresses to step S16 and CPU1 performs another process and complete | finishes the process by the flowchart of FIG.
As described above, if it is determined in step S13 that there is no gap, and if it is determined in step S14 that the gap length is not greater than or equal to the threshold, the process proceeds to step S17. In step S17, based on the result of the image scanning process in step S12, the CPU 1 has the maximum value of “the distance between the inner edges in the horizontal direction” of the figure in the rectangular label 401 from the center in the vertical direction of the number. It is determined whether the distance is within the threshold. As a result of this determination, if the maximum value of the “distance between the inner edges in the horizontal direction” of the graphic in the rectangular label 401 is within a certain threshold from the center in the vertical direction of the number, the process proceeds to step S18. In step S18, the CPU 1 determines that the figure in the rectangular label 401 is “0”. Then, the process proceeds to step S16, and after other processes are performed, the process according to the flowchart of FIG. 3 is terminated.

  On the other hand, when the maximum value of the “distance between the inner edges in the horizontal direction” of the graphic in the rectangular label 401 is not within a certain threshold from the center in the vertical direction of the number, the process proceeds to step S19. In step S <b> 19, the CPU 1 determines whether or not the number of figures in the rectangular label 401 that have the same value as the “internal edge distance in the horizontal direction” is equal to or greater than a threshold value. As a result of this determination, if the number of figures having the same value as the “inner edge distance in the horizontal direction” of the graphic in the rectangular label 401 is equal to or larger than the threshold value, the process proceeds to step S18 described above, and the CPU 1 proceeds to step S18. It is determined that the figure in 401 is “0”. In the example shown in FIG. 4B, it is determined that the figure in the rectangular label 401 is “0” through the processes of steps S17, S19, and S18. Then, the process proceeds to step S16, and after other processes are performed, the process according to the flowchart of FIG. 3 is terminated.

On the other hand, if the number of figures in the rectangular label 401 that have the same value as the “maximum distance between the inner edges in the horizontal direction” is not equal to or greater than the threshold value, the process proceeds to step S20. In step S20, the CPU 1 determines that the figure in the rectangular label 401 is “8”. In the example shown in FIG. 4A, it is determined that the figure in the rectangular label 401 is “8” through the processes of steps S17, S19, and S20. Then, the process proceeds to step S16, and after other processes are performed, the process according to the flowchart of FIG. 3 is terminated.
Note that the processing in step S19 may not be performed in consideration of noise included in the image. That is, when it is determined in step S17 that the maximum value of the “inner edge distance in the horizontal direction” of the graphic in the rectangular label 401 is not within a certain threshold from the vertical center of the number, The process may advance to step S20 without performing the process of S19, and it may be determined that the figure in the rectangular label 401 is “8”.

  As described above, if it is determined in step S11 that the number recognized by the music symbol recognition process is “1” or “4”, the process proceeds to step S21. In step S21, the CPU 1 performs a width determination process. In the width determination process, the pixel values of the pixels in the rectangular label set in the music symbol recognition process are sequentially read, and the width of the number in the rectangular label (the distance between the outer edges in the horizontal direction) is detected based on the read pixel value. To do. Details of the width determination process will be described later.

FIG. 5 is a diagram illustrating a second example of a graphic in the rectangular label 401. Specifically, FIG. 5A shows “1”, and FIG. 5B shows “4”.
As described above, in the present embodiment, the music symbol recognition process is performed after the staff erasing process is performed. When the staff is positioned in the white portion of “4”, FIG. As shown, the staff of that part may not be erased. For this reason, as shown in FIGS. 5A and 5B, the difference between “1” and “4” becomes small, and it is difficult to accurately distinguish and recognize these in the above-described pattern matching. There is a case. Therefore, in the present embodiment, the recognition rate of numbers is improved as compared with the prior art by using the horizontal width of the figure in the rectangular label 401.

  Returning to the description of FIG. 3, in step S <b> 22, the CPU 1 determines whether or not the maximum value of the horizontal width of the graphic in the rectangular label 401 is equal to or greater than the threshold based on the result of the horizontal width determination processing in step S <b> 21. As a result of the determination, if the maximum value of the horizontal width of the graphic in the rectangular label 401 is equal to or greater than the threshold value, the process proceeds to step S23. In step S23, the CPU 1 determines that the figure in the rectangular label 401 is “4”. In the example illustrated in FIG. 5B, it is determined that the figure in the rectangular label 401 is “4” through the processes of steps S <b> 22 and S <b> 23. Then, the process proceeds to step S16, and after other processes are performed, the process according to the flowchart of FIG. 3 is terminated.

  On the other hand, if the maximum value of the horizontal width of the graphic in the rectangular label 401 is not equal to or greater than the threshold value, the process proceeds to step S24. In step S24, the CPU 1 performs a protrusion determination process. When the process proceeds to step S24, there is a high possibility that the figure in the rectangular label 401 is “1”. However, there is a possibility that there is an error in the above-described music symbol recognition process, and that what was recognized as “1” is a vertical line. Therefore, in the present embodiment, information on the projection of the figure in the rectangular label 401 is obtained by the projection determination process. Details of this protrusion determination processing will be described later.

  Next, in step S25, the CPU 1 determines whether or not the projection of the figure in the rectangular label 401 is at a reasonable position based on the result of the projection determination process in step S24. As a result of the determination, if the projection of the figure in the rectangular label 401 is at a reasonable position as “1”, the process proceeds to step S26. In step S26, the CPU 1 determines that the figure in the rectangular label 401 is “1”. In the example shown in FIG. 5A, it is determined that the figure in the rectangular label 401 is “1” through the processes of steps S22, S24, S25, and S26. Then, the process proceeds to step S16, and after other processes are performed, the process according to the flowchart of FIG. 3 is terminated.

On the other hand, if the projection of the figure in the rectangular label 401 is not at a valid position as “1”, the process proceeds to step S27. When the process proceeds to step S <b> 27, there is a possibility that the figure in the rectangular label 401 is not “1” but connected to the tail candidate line above or below the rectangular label 401.
FIG. 6 shows a tablature (FIG. 6 (a)) in which the numeral 1 is displayed in the vicinity of the note tail, and a tablature (in FIG. b)).
As shown in FIG. 6 (a), the vertical line next to the stem should be recognized as “1”, but as shown in FIG. 6 (b) above or below the tail. A vertical line should be recognized as connected to the tail.

  Returning to the description of FIG. 3, the projection of the figure in the rectangular label 401 is not at a valid position as “1”, and when the process proceeds to step S <b> 27, the CPU 1 determines the threshold value from the figure in the rectangular label 401 and the rectangular label 401. The positional relationship with the vertical line within the distance is detected. In step S <b> 28, the CPU 1 determines whether or not the deviation of the horizontal position between the figure in the rectangular label 401 and the vertical line at a distance within the threshold from the rectangular label 401 is equal to or greater than the threshold.

  If the result of this determination is that the displacement in the horizontal direction between the figure in the rectangular label 401 and the vertical line at a distance within the threshold from the rectangular label 401 is greater than or equal to the threshold, the process proceeds to step S29 as described above. The CPU 1 determines that the figure in the rectangular label 401 is “1”. In the example shown in FIG. 6A, it is determined that the figure in the rectangular label 401 is “1” through the processes of steps S28 and S29. Then, the process proceeds to step S16, and after other processes are performed, the process according to the flowchart of FIG. 3 is terminated.

  On the other hand, when the horizontal position shift between the figure in the rectangular label 401 and the vertical line at a distance within the threshold from the rectangular label 401 is not greater than or equal to the threshold, the figure in the rectangular label 401 is not “1”. , It is determined to be connected to the tail candidate line above or below the rectangular label 401. In step S30, the CPU 1 connects the figure (vertical line) in the rectangular label 401 to the tail candidate line above or below the rectangular label 401, and uses the connected vertical line as a new tail candidate line. Set again. In the example shown in FIG. 6B, it is determined through the processes of steps S28 and S30 that the figure in the rectangular label 401 is connected to the tail. Then, the process proceeds to step S16, and after other processes are performed, the process according to the flowchart of FIG. 3 is terminated.

Next, an example of the image scanning process in step S12 in FIG. 3 will be described with reference to the flowchart in FIG.
First, in step S101, the CPU 1 sets a starting point for scanning a pixel as an initial position. In the present embodiment, the starting point is set at the center of the rectangular label 401 in the horizontal direction where the staff was present in the rectangular label 401. However, the initial position of the starting point is not limited to this.

  Next, in step S102, the CPU 1 determines whether or not the set start point is a black pixel (the pixel value of the start point is 1). If the result of this determination is that the starting point set in step S101 is a black pixel, processing proceeds to step S103. In step S103, the vertical position of the start point set in step S101 is shifted, and the start point position is reset. Specifically, for example, the vertical position of the start point set in step S101 is shifted to a position that is closest to the start point set in step S101 and has not yet been set. Then, the process returns to step S102, and steps S102 and S103 are repeated until the set start point is a white pixel (the pixel value of the start point is 0).

When the starting point is a white pixel as described above, the process proceeds to step S104. In step S104, the CPU 1 scans pixels in the horizontal direction (right direction and left direction) using the set start point as a base point, and reads the pixel value of each pixel.
Next, in step S <b> 105, the CPU 1 determines whether all lines of the rectangular label 401 have been scanned. As a result of the determination, if all the lines of the rectangular label 401 have not been scanned, the process proceeds to step S106. In step S106, the CPU 1 shifts the position of the start point in the vertical direction and resets the position of the start point.

More specifically, for example, the vertical position of the currently set start point is shifted to the position that is closest to the currently set start point and has not yet been set. However, when the start point is shifted to a black pixel, the start point is shifted to a position other than the position where the black pixel exists.
Then, the process returns to step S104, the pixels are scanned in the horizontal direction (right direction and left direction) using the reset start point as a base point, and the pixel value of each pixel is read. Steps S104 to S106 are repeated until all the lines of the rectangular label 401 are scanned.

  If it is determined in step S105 that all the lines of the rectangular label 401 have been scanned, the process proceeds to step S107. In step S107, the CPU 1 determines whether there is a gap in the graphic in the rectangular label 401 based on the arrangement of each white pixel in the rectangular label 401 (that is, the pixel value). For example, when there is no black pixel (a pixel having a pixel value of 1) in the right direction or the left direction with the start point as a base point, it is determined that there is a gap between figures in the rectangular label 401. As a result of the determination, if there is no gap in the graphic in the rectangular label 401, the process proceeds to step S108. In step S108, the CPU 1 obtains the distance between the inner edges in the horizontal direction of the figure in the rectangular label 401 and the position thereof based on the arrangement (that is, the pixel value) of each white pixel in the rectangular label 401, and determines them. Are stored in the RAM 3 in association with each other. In the example shown in FIGS. 4A and 4B, for example, the portion indicated by the arrow is the distance between the inner edges in the horizontal direction of the graphic in the rectangular label 401. And the process by the flowchart of FIG. 7 is complete | finished.

  On the other hand, if there is a gap in the graphic in the rectangular label 401, the process proceeds to step S109. In step S109, the CPU 1 obtains the position and length of the gap between the figures in the rectangular label 401, and stores them in the RAM 3 in association with each other. For example, when a state where there is no black pixel (a pixel having a pixel value of 1) in the right direction or the left direction from the start point is continuous in the vertical direction, the number of pixels continuous in the vertical direction is It becomes the length of the gap. Further, a portion where the white pixel of the length exists in the vertical direction is a gap position. In the example shown in FIG. 4C, a portion indicated by an arrow is a gap. Then, the process proceeds to step S108 described above.

Next, an example of the width determination process in step S21 of FIG. 3 will be described with reference to the flowchart of FIG.
First, in step S201, the CPU 1 sets a starting point for scanning a pixel as an initial position. In the present embodiment, the starting point is set at the center position of the rectangular label 401. However, the initial position of the starting point is not limited to this.
Next, in step S202, the CPU 1 scans the pixels in the horizontal direction (right direction and left direction) using the set start point as a base point, and reads the pixel value of each pixel.

  Next, in step S203, the CPU 1 determines whether or not there is a black pixel in the pixels scanned in the horizontal direction in step S202. If it is determined that there is a black pixel, the process proceeds to step S204. In step S204, the CPU 1 obtains the “lateral width (distance between outer edges) at the location scanned in step S202” of the graphic in the rectangular label 401 and the position where the distance between the outer edges exists, and correlates them with each other. Store in the RAM 3. Note that these can be obtained based on the number of horizontal black pixels scanned in step S202 and the positions of these black pixels. In addition, it can be obtained based on the number of scanned white pixels in the horizontal direction and the positions of the white pixels. In the example shown in FIG. 5, for example, the white arrow portion is the horizontal width of the figure in the rectangular label 401.

  Next, in step S205, the CPU 1 determines whether all the lines of the rectangular label 401 have been scanned. As a result of the determination, if all the lines of the rectangular label 401 have not been scanned, the process proceeds to step S206. In step S206, the CPU 1 shifts the position of the start point in the vertical direction and resets the position of the start point. More specifically, for example, the vertical position of the currently set start point is shifted to the position that is closest to the currently set start point and has not yet been set. Then, the process returns to step S202, and the pixels are scanned in the horizontal direction (right direction and left direction) using the reset start point as a base point, and the pixel value of each pixel is read. Steps S202 to S206 are repeated until all the lines of the rectangular label 401 are scanned. Thus, when all the pixels in the rectangular label 401 are scanned, the processing according to the flowchart of FIG.

Next, an example of the protrusion determination process in step S24 of FIG. 3 will be described with reference to the flowchart of FIG.
First, in step S301, the CPU 1 separates the rectangular label 401 into three regions in the vertical direction. FIG. 10 is a diagram illustrating an example of how the rectangular labels 401 are separated. In the example shown in FIG. 10, the rectangular label 401 is divided into three regions 401a to 401c in the vertical direction.

Next, in step S302, the CPU 1 calculates an average value of the horizontal width (distance between outer edges) of the figure in the rectangular label 401 for each of the three regions 401a to 401c separated in step S301. This calculation can be performed based on the result of step S204 in FIG. And the process by the flowchart of FIG. 9 is complete | finished.
As described above, in step S25 of FIG. 3, it is determined whether or not the projection of the figure in the rectangular label 401 is at a reasonable position based on the result of the projection determination process in step S24. .

  The processing in step S25 will be specifically described. For example, whether or not the difference between the average value of the lateral width in the relatively upper region 401a and the average value of the lateral width in the relatively central region 401b is greater than or equal to a threshold value. If it is equal to or greater than the threshold value, it can be determined that the projection of the figure in the rectangular label 401 is at an appropriate position as “1”. Further, the difference between the average value of the lateral width in the relatively upper region 401a and the average value of the lateral width in the relatively central region 401b is equal to or greater than the threshold value, and the width of the relatively lower region 401c When the difference between the average value and the average width value in the relatively central area 401b is equal to or greater than the threshold value, it is determined that the projection of the figure in the rectangular label 401 is at a reasonable position as “1”. You can also.

  As described above, in the present embodiment, the pixel value in the rectangular label 401 is read, and which of the conditions set in advance according to the number on the tablature recognized by the dictionary matching is determined. By determining, it is determined which number the figure in the rectangular label 401 corresponds to. Therefore, even if the staff is erased, it is possible to recognize the tablature (number) more accurately than before without changing the dictionary matching method.

  In the present embodiment, in steps S17 and S19 of FIG. 3, whether the graphic in the rectangular label 401 is “0” or “8” is determined based on the distance between the inner edges of the graphic in the rectangular label 401. I decided to distinguish. However, instead of (or in addition to) the distance between the inner edges in the horizontal direction of the graphic in the rectangular label 401, the distance between the outer edges in the horizontal direction of the graphic in the rectangular label 401 (or from the end of the rectangular label 401 to the Whether the figure in the rectangular label 401 is “0” or “8” may be determined based on the shortest distance in the horizontal direction to the outer edge of the figure.

  In the present embodiment, in the image scanning process in step S12 of FIG. 3, after all the lines of the rectangular label 401 are scanned, the gap between the figures in the rectangular label 401 and the distance between the inner edges in the horizontal direction are once set. (See steps S104 to S109 in FIG. 7). However, this is not always necessary. For example, in the image scanning process, only the gap length and position of the graphic in the rectangular label 401 are obtained, and the distance between the inner edges in the horizontal direction of the graphic in the rectangular label 401 is determined between steps S13, S14 and S17. The position where the distance between the inner edges exists may be obtained. In this case, in the image scanning process, scanning in the horizontal direction (right direction and left direction) is terminated when a black pixel (a pixel having a pixel value of 1) exists, and the starting point is moved in the vertical direction. Can do. Also, when obtaining the distance between the inner edges in the horizontal direction of the figure in the rectangular label 401, when there are black pixels (pixels having a pixel value of 1) on both the left and right sides, the horizontal direction (right direction and left direction) is determined. The scanning can be finished and the starting point can be moved up and down.

  Further, as in the present embodiment, when the gap length of the graphic in the rectangular label 401 is not greater than or equal to the threshold value, the process is performed assuming that there is no gap in the graphic (steps S14 and S17 to S20 in FIG. 3). However, it is not always necessary to perform the process of step S14.

  Further, in the present embodiment, in step S28 of FIG. 3, when the positional deviation in the horizontal direction between the figure in the rectangular label 401 and the vertical line at a distance within the threshold from the rectangular label 401 is equal to or greater than the threshold. In step S29, it is determined that the figure in the rectangular label 401 is “1”, but this is not necessarily required. For example, you may make it discard the recognition result of the number on the tablature recognized by dictionary matching.

  Further, between steps S17 and S19 in FIG. 3 and step S18, it is determined whether or not YES is determined in step S102 in FIG. 7, and if YES is determined in step S102 in FIG. It progresses to step S20, and when it determines with NO by step S102 of FIG. 7, you may make it progress to step S18.

  In the present embodiment, in step S302 of FIG. 9, the average value of the horizontal width (distance between outer edges) of the figure in the rectangular label 401 is calculated for each of the three regions 401a to 401c separated in step S301. . However, this is not always necessary, and the maximum value of the horizontal width (distance between outer edges) of the figure in the rectangular label 401 may be calculated for each of the three regions 401a to 401c separated in step S301.

  In this embodiment, “2”, “3”, “5”, and “7” are assumed to have high recognition reliability by the music symbol recognition process (pattern matching), and the recognized number is “2”. "," 3 "," 5 "or" 7 ", the processing according to the flowchart of FIG. 3 is not performed. However, this is not always necessary. For example, “2”, “3”, “5”, and “7” may also be determined in step S15.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. In the first embodiment described above, an example has been described in which whether or not there is a gap in the graphic in the rectangular label 401 is determined by performing only horizontal scanning. However, in this case, when the figure in the rectangular label 401 has a gap in the oblique direction, the gap cannot be detected. FIG. 11 is a diagram showing an example of a figure having a gap in an oblique direction in the rectangular label 401. In the example shown in FIG. 11, there is a gap between figures in the upper left direction toward the page. Therefore, in the present embodiment, it is possible to determine the presence / absence of a gap even in such a figure having a gap in an oblique direction. As described above, the present embodiment is different from the first embodiment described above mainly in part of the process (image scanning process in step S12 in FIG. 3) when determining the presence or absence of a gap. Therefore, in the description of the present embodiment, the same parts as those in the first embodiment described above are denoted by the same reference numerals as those in FIGS.

FIG. 12 is a flowchart illustrating an example of the image scanning process in step S12 of FIG.
First, in step S401 in FIG. 12A, the CPU 1 sets a starting point for scanning a pixel as an initial position. Also in the present embodiment, the starting point is set at the center of the rectangular label 401 in the horizontal direction where the staff was present in the rectangular label 401. However, the initial position of the starting point is not limited to this.

  Next, in step S402, the CPU 1 determines whether or not the set start point is a black pixel (the pixel value of the start point is 1). As a result of this determination, if the starting point set in step S401 is a black pixel, the process proceeds to step S403. In step S403, the vertical position of the starting point set in step S401 is shifted in the same manner as described in step S102 in FIG. 7, and the starting point position is reset. Then, the process returns to step S402, and steps S402 and S403 are repeated until the set start point is a white pixel (the pixel value of the start point is 0).

When the starting point is a white pixel as described above, the process proceeds to step S404. In step S404, the CPU 1 scans the pixels leftward (here, leftward toward the paper surface of FIG. 11) with the set start point as a base point, and reads the pixel value of each pixel. At this time, the CPU 1 turns on the left direction flag stored in the RAM 3 or the register.
Next, in step S405, the CPU 1 determines whether there is a gap in the graphic in the rectangular label 401 based on the arrangement of each white pixel (pixel value of the pixel) obtained by scanning the pixel. The determination is performed in the same manner as described in step S107. As a result of the determination, if there is no gap in the graphic in the rectangular label 401, the process proceeds to step S415 to be described later.

On the other hand, if the result of this determination is that there is a gap in the figure in the rectangular label 401, the process proceeds to step S406. In step S406, the CPU 1 turns off the left direction flag turned on in step S404.
Next, in step S407, the CPU 1 scans the pixel in the right direction with the set start point as a base point, and reads the pixel value of each pixel. At this time, the CPU 1 turns on the right direction flag stored in the RAM 3 or the register.
Next, in step S408, the CPU 1 determines whether or not there is a gap in the graphic in the rectangular label 401 based on the arrangement of each white pixel (that is, the pixel value) obtained by scanning the pixel. The determination is performed in the same manner as described in step S107. As a result of the determination, if there is no gap in the graphic in the rectangular label 401, the process proceeds to step S428 in FIG.

On the other hand, if the result of this determination is that there is a gap in the graphic in the rectangular label 401, the process proceeds to step S409. In step S409, the CPU 1 turns off the right direction flag turned on in step S407.
Next, in step S410, the CPU 1 determines whether all the lines of the rectangular label 401 have been scanned. As a result of the determination, if all the lines of the rectangular label 401 have not been scanned, the process proceeds to step S414 described later.

On the other hand, if all the lines of the rectangular label 401 have been scanned, the process proceeds to step S411. In step S411, the CPU 1 explains in step S107 in FIG. 7 whether or not there is a gap in the graphic in the rectangular label 401 based on the arrangement (that is, the pixel value) of each white pixel in the rectangular label 401. Judgment is made as described above. If the result of this determination is that there is a gap in the graphic within the rectangular label 401, the flow proceeds to step S412. In step S412, the CPU 1 obtains the position and length of the graphic gap in the rectangular label 401 and stores them in the RAM 3 in association with each other. Then, the process proceeds to step S413.
If there is no gap in the graphic in the rectangular label 401 in step S411, step S412 is omitted and the process proceeds to step S413. In step S413, the CPU 1 obtains the distance between the inner edges of the graphic in the rectangular label 401 in the horizontal direction and the position where the distance between the inner edges exists based on the pixel value of each pixel in the rectangular label 401. They are stored in the RAM 3 in association with each other. And the process by the flowchart of FIG. 12 is complete | finished.

As described above, if it is determined in step S410 that all the lines of the rectangular label 401 have not been scanned, the process proceeds to step S414. In step S414, the CPU 1 shifts the vertical position of the start point in the same manner as described in step S106, and resets the start point position.
Then, the CPU 1 turns on the upward flag stored in the RAM 3 or the register when shifting the starting point position upward, and stores it in the RAM 3 or the register when shifting the starting point position downward. Turn on the down flag. Then, the process returns to the above-described step S404, and the processes after step S404 are performed from the reset position of the starting point.

As described above, if it is determined in step S405 that there is no gap in the graphic in the rectangular label 401, the process proceeds to step S415. In step S415, the CPU 1 determines whether or not the downward flag is turned on. If the result of this determination is that the downward flag is not turned on, processing proceeds to step S422 described later.
On the other hand, if the downward flag is turned on, the process proceeds to step S416. In step S416, the CPU 1 moves the scanning point to a pixel one pixel above the pixel at the time point when it is determined that there is no gap.

Next, in step S417, the CPU 1 determines whether or not the scanning point in step S416 can be moved. For example, when the pixel immediately above the pixel at which it is determined that there is no gap is a white pixel, it is determined that the scanning point can be moved. On the other hand, when the pixel immediately above the pixel at the time when it is determined that there is no gap is a black pixel, it is determined that the scanning point cannot be moved. Note that the processing order of steps S416 and S417 may be changed.
As a result of this determination, if the scanning point cannot be moved, the process proceeds to step S421, and the CPU 1 turns off all the flags that are turned on (up / down / left / right direction flags). Then, the process proceeds to step S407 described above, and scanning in the right direction is performed.

On the other hand, if the scanning point can be moved, the process proceeds to step S418, and the CPU 1 scans the pixel leftward with the moved scanning point as a base point, and reads the pixel value of each pixel.
Next, in step S419, it is determined whether there is a gap in the graphic in the rectangular label 401 based on the arrangement of each white pixel (that is, the pixel value) obtained by scanning the pixel. For example, when there is no black pixel in the left direction with the moved scanning point as a base point, it is determined that there is a gap between figures in the rectangular label 401. If the result of this determination is that there is a gap in the graphic within the rectangular label 401, step S420 is omitted and processing proceeds to step S421 described above. On the other hand, if there is no gap in the figure in the rectangular label 401, the process proceeds to step S420.

In step S420, the CPU 1 determines whether or not the vertical position of the scanning point is a position where the staff was present in the rectangular label 401. As a result of this determination, if the vertical position of the scanning point is not the position where the staff was present in the rectangular label 401, the process returns to step S416 described above, and the vertical position of the scanning point is the rectangular label. Steps S416 to S420 are repeatedly performed until the staff is located in 401.
In step S421, the CPU 1 turns off all the flags that are turned on. Then, the process proceeds to step S407 described above, and scanning in the right direction is performed.

As described above, when it is determined in step S415 that the downward flag is not turned on, the process proceeds to step S422. In step S422, the CPU 1 determines whether or not the upward flag is turned on. If the upward flag is turned on as a result of this determination, the process proceeds to step S423 described later.
On the other hand, if the upward flag is not turned on, the process proceeds to step S441. In step S441, the CPU 1 determines whether or not to shift the scanning point downward by one pixel. For example, when there is a white pixel only below the upper and lower pixels at the time when it is determined that there is no gap, it is determined that the scanning point is shifted downward by one pixel.

As a result of this determination, when the scanning point is shifted downward by one pixel, the process proceeds to step S442, and the CPU 1 moves the scanning point to the pixel immediately below the pixel at the time when it is determined that there is no gap. Then, the process proceeds to step S418 described above.
On the other hand, when the scanning point is not shifted by one pixel, the process proceeds to step S443, and the CPU 1 determines whether or not the scanning point is shifted by one pixel. For example, when there is a white pixel only above the upper and lower pixels at the time when it is determined that there is no gap, it is determined that the scanning point is shifted upward by one pixel.
If the result of this determination is that the scanning point is shifted upward by one pixel, processing proceeds to step S444, and the CPU 1 moves the scanning point to a pixel one pixel above that at the time when it is determined that there is no gap. And it progresses to step S425 mentioned later. On the other hand, when the scanning point is not shifted upward by one pixel, the process proceeds to step S407 described above, and scanning from the start point to the right is performed.

  As described above, if it is determined that the upward flag is turned on as a result of the determination in step S422, the process proceeds to step S423. In step S423, the CPU 1 moves the scanning point to the pixel immediately below the pixel at the time when it is determined that there is no gap.

Next, in step S424, the CPU 1 determines whether or not the scanning point in step S423 can be moved. For example, when the pixel immediately below the pixel when it is determined that there is no gap is a white pixel, it is determined that the scanning point can be moved. On the other hand, when the pixel immediately below the pixel when it is determined that there is no gap is a black pixel, it is determined that the scanning point cannot be moved. Note that the order of the processes in steps S423 and S424 may be changed.
If the result of this determination is that the scanning point cannot be moved, the process proceeds to step S421 described above, and the CPU 1 turns off all the flags that have been turned on (up / down / left / right direction flags). Then, the process proceeds to step S407 described above, and scanning in the right direction is performed.

On the other hand, if the scanning point can be moved, the process proceeds to step S425, and the CPU 1 scans the pixel leftward with the moved scanning point as a base point, and reads the pixel value of each pixel.
Next, in step S426, it is determined whether there is a gap in the graphic in the rectangular label 401 based on the arrangement of white pixels (that is, the pixel value) obtained by scanning the pixel. For example, when there is no black pixel in the left direction with the moved scanning point as a base point, it is determined that there is a gap between figures in the rectangular label 401. If the result of this determination is that there is a gap in the graphic within the rectangular label 401, step S427 is omitted and the process proceeds to step S421 described above. On the other hand, if there is no gap in the graphic in the rectangular label 401, the process proceeds to step S427.

  In step S427, the CPU 1 determines whether the vertical position of the scanning point is a position where the staff was present in the rectangular label 401. As a result of the determination, if the vertical position of the scanning point is not the position where the staff was present in the rectangular label 401, the process returns to step S423 described above, and the vertical position of the scanning point is the rectangular label. Steps S423 to S427 are repeatedly performed until the staff in 401 is located. Then, the process proceeds to step S421 described above.

As described above, if it is determined in step S408 that there is no gap in the graphic in the rectangular label 401, the process proceeds to step S428 in FIG. In step S428, the CPU 1 determines whether or not the downward flag is turned on. If the result of this determination is that the downward flag is not turned on, processing proceeds to step S435 described later.
On the other hand, if the downward flag is turned on, the process proceeds to step S429. In step S429, the CPU 1 moves the scanning point to a pixel one pixel above the pixel at the time point when it is determined that there is no gap.

Next, in step S430, the CPU 1 determines whether or not the scanning point in step S429 can be moved. For example, when the pixel immediately above the pixel at which it is determined that there is no gap is a white pixel, it is determined that the scanning point can be moved. On the other hand, when the pixel immediately above the pixel at the time when it is determined that there is no gap is a black pixel, it is determined that the scanning point cannot be moved. Note that the order of the processes in steps S429 and S430 may be changed.
As a result of this determination, if the scanning point cannot be moved, the process proceeds to step S434, and the CPU 1 turns off all the flags that are turned on (up / down / left / right direction flags). Then, the process proceeds to step S410 described above, and it is determined whether or not all the lines in the rectangular label 401 have been scanned.

On the other hand, if the scanning point can be moved, the process proceeds to step S431, and the CPU 1 scans the pixel in the right direction with the moved scanning point as a base point, and reads the pixel value of each pixel.
Next, in step S432, the CPU 1 determines whether there is a gap in the graphic in the rectangular label 401 based on the arrangement of white pixels (that is, the pixel value) obtained by scanning the pixel. For example, when there is no black pixel in the right direction with the moved scanning point as a base point, it is determined that there is a gap between figures in the rectangular label 401. If the result of this determination is that there is a gap in the graphic within the rectangular label 401, step S433 is omitted and the process proceeds to step S434 described above. On the other hand, if there is no gap in the figure in the rectangular label 401, the process proceeds to step S433.

In step S433, the CPU 1 determines whether the vertical position of the scanning point is a position where the staff was present in the rectangular label 401. As a result of this determination, if the vertical position of the scanning point is not the position where the staff was present in the rectangular label 401, the process returns to step S429 described above, and the vertical position of the scanning point is the rectangular label. Steps S429 to S433 are repeatedly performed until the staff in 401 is located.
Then, in step S434, the CPU 1 turns off all the flags that are turned on. Then, the process proceeds to step S410 described above, and it is determined whether or not all the lines in the rectangular label 401 have been scanned.

As described above, when it is determined in step S428 that the downward flag is not turned on, the process proceeds to step S435. In step S435, the CPU 1 determines whether or not the upward flag is turned on. If the upward flag is turned on as a result of this determination, the process proceeds to step S435 described later.
On the other hand, if the upward flag is not turned on, the process proceeds to step S445. In step S445, the CPU 1 determines whether or not to shift the scanning point downward by one pixel. For example, when there is a white pixel only below the upper and lower pixels at the time when it is determined that there is no gap, it is determined that the scanning point is shifted downward by one pixel.

As a result of this determination, when the scanning point is shifted downward by one pixel, the process proceeds to step S446, and the CPU 1 moves the scanning point to the pixel immediately below the pixel at the time when it is determined that there is no gap. Then, the process proceeds to step S429 described above.
On the other hand, when the scanning point is not shifted by one pixel, the process proceeds to step S447, and the CPU 1 determines whether or not the scanning point is shifted by one pixel. For example, when there is a white pixel only above the upper and lower pixels at the time when it is determined that there is no gap, it is determined that the scanning point is shifted upward by one pixel.

In addition, when both the upper and lower pixels of the pixel at the time when it is determined that there is no gap are white pixels, for example, the position of the pixel at the time is lower than the position where the staff was present. In such a case, the scanning point may be shifted by one pixel downward, and if it is upward, the scanning point may be shifted by one pixel.
As a result of this determination, when the scanning point is shifted up by one pixel, the process proceeds to step S448, and the CPU 1 moves the scanning point to a pixel one pixel above the pixel at which it is determined that there is no gap. And it progresses to step S438 mentioned later. On the other hand, when the scanning point is not shifted upward by one pixel, the process proceeds to step S410 described above, and it is determined whether or not all the lines in the rectangular label 410 have been scanned.

  As described above, when it is determined that the upward flag is turned on as a result of the determination in step S435, the process proceeds to step S436. In step S436, the CPU 1 moves the scanning point to a pixel immediately below the pixel at the time point when it is determined that there is no gap.

Next, in step S437, the CPU 1 determines whether or not the scanning point in step S436 can be moved. For example, when the pixel immediately below the pixel when it is determined that there is no gap is a white pixel, it is determined that the scanning point can be moved. On the other hand, when the pixel immediately below the pixel when it is determined that there is no gap is a black pixel, it is determined that the scanning point cannot be moved. Note that the processing order of steps S436 and S437 may be changed.
If the result of this determination is that the scanning point cannot be moved, the process proceeds to step S434 described above, and the CPU 1 turns off all the flags that have been turned on (up / down / left / right direction flags). Then, the process proceeds to step S410 described above, and it is determined whether or not all the lines in the rectangular label 401 have been scanned.

On the other hand, if the scanning point can be moved, the process proceeds to step S438, and the CPU 1 scans the pixel in the right direction with the moved scanning point as a base point, and reads the pixel value of each pixel.
Next, in step S439, the CPU 1 determines whether there is a gap in the graphic in the rectangular label 401 based on the arrangement of white pixels (that is, the pixel value) obtained by scanning the pixel. For example, when there is no black pixel in the left direction with the moved scanning point as a base point, it is determined that there is a gap between figures in the rectangular label 401. If the result of this determination is that there is a gap in the graphic within the rectangular label 401, step S440 is omitted and the process proceeds to step S434 described above. On the other hand, if there is no gap in the figure in the rectangular label 401, the process proceeds to step S440.

  In step S440, the CPU 1 determines whether the vertical position of the scanning point is a position where the staff was present in the rectangular label 401. As a result of this determination, if the vertical position of the scanning point is not the position where the staff was present in the rectangular label 401, the process returns to step S436 described above, and the vertical position of the scanning point is the rectangular label. Steps S436 to S440 are repeated until the staff line 401 is located. Then, the process proceeds to step S434 described above.

  Here, the flow of the flowchart of FIG. 12 will be described using the example shown in FIG. First, in the vicinity of the center of the drawing, the pixel is scanned in the left direction toward the paper surface. Accordingly, the left direction flag is turned on in step S404. When the pixel 1110 is reached, it is determined in step S405 that there is no gap. Here, it is assumed that neither the upward flag nor the downward flag is turned on. If it does so, it will progress to step S441 through step S415 and S422. Here, since the white pixel exists only under the pixel 1110, the scanning point moves from the pixel 1110 to the pixel 1111 in steps S441 and S442. Then, in step S418, scanning is performed in the left direction with the moved scanning point as a base point, and when the pixel 1112 is reached, it is determined in step S419 that there is no gap. Here, it is assumed that the position of the pixel 1112 is not the position where the staff was present. Then, the process returns to step S416 via step S420, and the scanning point moves from the pixel 1112 to the pixel 1113. Thereafter, by performing the processing from step S417 onward, as shown by the arrow in FIG. 11, the presence or absence of a gap is determined by scanning the pixel along the inner edge of the graphic in the rectangular label 401 (that is, reading the pixel value). Is done.

  However, when the pixel 1110 is reached, scanning in the horizontal direction (right direction and left direction) may be terminated and the starting point may be moved in the vertical direction. For example, assuming that the downward flag is turned on, an example of an outline of the operation in this case will be described. First, in step S404, the left direction flag is turned on, and the pixel is scanned leftward from the vicinity of the center in FIG. When the scanning point reaches the pixel 1110, it is determined in step S405 that there is no gap. Here, since the downward flag is turned on, the process proceeds to step S416. In steps S416 and S417, it is determined that the scanning point cannot be moved upward, and the flag is turned off in step S421. In this way, when the pixel 1110 is reached, scanning in the horizontal direction (right direction and left direction) ends. Then, the process returns to step S404 through the processing from step S407 onward, and the pixel immediately below the pixel 1110 is scanned leftward to reach the pixel 1112. The processing after reaching the pixel 1112 is as described above.

As described above, in the present embodiment, since the pixels are scanned along the inner edge of the graphic in the rectangular label 401 to determine the presence or absence of a gap, in addition to the effects described in the first embodiment, Even if there is a gap in the figure in an oblique direction, the presence or absence of the gap can be detected.
In the present embodiment, the case where the presence / absence of a gap is determined in all directions of a figure has been described as an example, but it is not always necessary to do so. For example, in view of the characteristics of “0”, “6”, “8”, and “9”, if there is no gap in the diagonally lower right direction of the graphic, steps S428 to S433, S445, and S446 are performed. The process of step S408 can be omitted, and the process can proceed from step S408 to step S435.

In this embodiment, the case where the pixel value of the pixel is always read when the pixel is scanned has been described as an example. However, this is not necessarily required. For example, it is possible to turn off the flag for a pixel whose pixel value has already been read, and to prevent the pixel value from being read again for a pixel for which the flag is turned on.
In addition, also in this embodiment, various modifications described in the first embodiment can be employed.

Each embodiment of the present invention described above can be realized by a computer executing a program. Further, a means for supplying the program to the computer, for example, a computer-readable recording medium such as a CD-ROM recording such a program, or a transmission medium for transmitting such a program may be applied as an embodiment of the present invention. it can. A program product such as a computer-readable recording medium in which the program is recorded can also be applied as an embodiment of the present invention. The above programs, computer-readable recording media, transmission media, and program products are included in the scope of the present invention.
In addition, each of the above-described embodiments is merely a specific example for carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereto. . That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

It is a block diagram which shows the 1st Embodiment of this invention and shows an example of a structure of a score recognition apparatus. It is a main flowchart which shows the 1st Embodiment of this invention and demonstrates an example of a process of CPU. 5 is a flowchart illustrating an example of recognition result verification processing according to the first embodiment of this invention. It is a figure which shows the 1st Embodiment of this invention and shows the 1st example of the figure in a rectangular label. It is a figure which shows the 1st Embodiment of this invention and shows the 2nd example of the figure in a rectangular label. The figure which shows the 1st Embodiment of this invention and shows the tablature by which the number 1 is displayed in the vicinity of a stem, and the tablature by which the stem separated into two by the blurring etc. is displayed It is. 4 is a flowchart illustrating an example of image scanning processing in step S12 of FIG. 3 according to the first embodiment of this invention. It is a flowchart which shows the 1st Embodiment of this invention and demonstrates an example of the width determination process in FIG.3 S21. It is a flowchart which shows the 1st Embodiment of this invention and demonstrates an example of the protrusion determination process in FIG.3 S24. It is a figure which shows the 1st Embodiment of this invention and shows an example of a mode that a rectangular label is isolate | separated. It is a figure which shows the 2nd Embodiment of this invention and shows an example of the figure which has a gap in the diagonal direction within a rectangular label. It is a flowchart which shows the 2nd Embodiment of this invention and demonstrates an example of the image scanning process in FIG.3 S12. It is a flowchart which shows the 2nd Embodiment of this invention and continues to FIGS.

Explanation of symbols

1 CPU
2 ROM
3 RAM
4 HDD
5 FDD
6 MIDI
7 Scanner Interface Circuit 8 Printer Interface Circuit 9 Keyboard Interface Circuit 10 CRT Interface Circuit 11 Scanner 12 Printer 13 Keyboard 14 CRT
100 Score recognition device

Claims (11)

An acquisition means for acquiring an image including information of a score on a paper surface from an image reading means;
An erasing unit for erasing a horizontal line corresponding to a string in the tablature from the image acquired by the acquiring unit;
A setting unit that sets a label for a graphic included in the image of the tablature among the images acquired by the acquiring unit after the horizontal line is deleted by the deleting unit;
Recognizing means for matching a figure included in the label set by the setting means and a preset dictionary and recognizing which number the figure belongs to;
Reading means for reading pixel values of at least some of the pixels in the label;
By determining which of the preset conditions the pixel value read by the reading unit corresponds to the number recognized by the recognition unit, the figure included in the label belongs to which number A score recognition apparatus comprising: a discriminating means for discriminating whether or not. Determination means for determining whether or not there is a gap in the graphic included in the label based on the pixel value read by the reading means;
A gap position deriving unit for deriving the position of the gap of the graphic included in the label based on the pixel value read by the reading unit when the determination unit determines that there is a gap in the graphic included in the label And
2. The score recognition apparatus according to claim 1, wherein the determining means determines which number a figure included in the label belongs to based on the gap position derived by the gap position deriving means. . When the number recognized by the recognizing means is 0, 6, 8, or 9, the determining means has a gap in the graphic included in the label based on the pixel value read by the reading means. Determine if there is,
The said discrimination | determination means discriminate | determines whether the figure contained in the said label is 6 and 9, based on the position of the gap derived | led-out by the said gap position deriving means. Sheet music recognition device. When it is determined by the determination means that there is a gap in the graphic included in the label, the gap length for deriving the length of the gap in the graphic included in the label based on the pixel value read by the reading means Deriving means;
Gap length determining means for determining whether or not the gap length derived by the gap length deriving means is equal to or greater than a threshold,
When the gap length determination means determines that the gap length is greater than or equal to a threshold, the determination means determines whether the graphic included in the label is based on the gap position derived by the gap position derivation means. 4. The musical score recognition apparatus according to claim 2, wherein the number belongs to which number.   The determination means determines whether or not there is a gap in the graphic included in the label based on the pixel value read by the reading means along the inner edge of the graphic included in the label. The musical score recognition apparatus according to any one of claims 2 to 4. Based on the pixel value read by the reading means, the inner edge portion derivation means for deriving the distance between the inner edges in the horizontal direction of the figure included in the label and the position where the distance between the inner edges exists,
The discriminating means discriminates to which number the figure included in the label belongs based on the distance between the inner edges derived by the inner edge deriving means and the position where the distance between the inner edges exists. The musical score recognition apparatus according to claim 1, wherein the musical score recognition apparatus is a musical score recognition apparatus. The inner edge deriving means, when the number recognized by the recognizing means is 0, 6, 8, or 9, based on the pixel value read by the reading means, Deriving the distance between the inner edges in the lateral direction and the position where the distance between the inner edges exists,
The discriminating means determines whether the figure included in the label is 0 or 8 based on the distance between the inner edges derived by the inner edge deriving means and the position where the distance between the inner edges exists. The score recognition device according to claim 6, wherein the score recognition device is discriminated.   When the determining means determines that there is no gap in the graphic included in the label, the determining means determines the label based on the position where the distance between the inner edges derived by the inner edge deriving means is maximized. The musical score recognition apparatus according to claim 6 or 7, wherein a number to which an included figure belongs is determined. Based on the pixel value read by the reading means, the outer edge portion deriving means for deriving the distance between the outer edges in the horizontal direction of the graphic included in the label and the position where the distance between the outer edges exists,
The discriminating means is a graphic included in the label based on the distance between the outer edges in the horizontal direction of the graphic included in the label and the position where the distance between the outer edges exists, derived by the outer edge deriving means. The musical score recognition apparatus according to claim 1, wherein the number belongs to which number. The outer edge derivation means, when the number recognized by the recognition means is 1 or 4, based on the pixel value read by the reading means, between the outer edges in the horizontal direction of the graphic included in the label And the position where the distance between the outer edges exists,
The discriminating means is a graphic included in the label based on the distance between the outer edges in the horizontal direction of the graphic included in the label and the position where the distance between the outer edges exists, derived by the outer edge deriving means. 10. The musical score recognition apparatus according to claim 9, wherein it is determined whether the value is 1 or 4. An acquisition step of acquiring an image including information on a sheet music score from an image reading unit;
An erasing step of erasing the horizontal line corresponding to the string in the tablature from the image obtained by the obtaining step;
A setting step for setting a label for a graphic included in the image of the tablature among the images acquired by the acquisition step after the horizontal line is deleted by the deletion step;
A recognition step for recognizing which figure the figure belongs to by matching a figure included in the label set in the setting step with a preset dictionary;
Reading the pixel values of at least some of the pixels in the label;
By determining which of the preset conditions the pixel value read in the reading step corresponds to the number recognized in the recognition step, the figure included in the label belongs to which number A computer program for causing a computer to execute a determination step of determining whether or not.

Images