GB2256708A

GB2256708A - Object sorter using neural network

Info

Publication number: GB2256708A
Application number: GB9212449A
Authority: GB
Inventors: Kazuho Kodaira; Hiroshi Nakata; Shozou Takamura
Original assignee: Sumitomo Heavy Industries Ltd
Current assignee: Sumitomo Heavy Industries Ltd
Priority date: 1991-06-11
Filing date: 1992-06-11
Publication date: 1992-12-16
Also published as: GB9212449D0

Abstract

A sorter, e.g. for fruits or industrial artefacts on a conveyor, comprises a colour camera 23 with a frame store 42, and a neural network 442 trained to sort the objects by colour or by flaws. Evaluation unit 441 derives hue, saturation, and intensity values from the frame store image. <IMAGE>

Description

DETERMINATION DEVICE FOR USE IN SORTING OBJECTS, ESPECIALLY FRUITS.

Background of the Invention This invention relates to a determination device for use in sorting objects, for example fruits such as apples, and more particularly, to a device of the type described for use in determining surface color of fruits.

For the most part, the tint of fruits is visually decided by a worker. However, the decision relying on human sense is not reliable with respect to precision thereof because difference among individuals often causes different result even for the same fruit. In order to overcome such an inconvenience, tint determination devices have been developed for automatically determining the surface color of fruits. Japanese Unexamined Patent Prepublication No. 10123/1990 (Japanese Patent Application No. 158164/1988) disclosed such a determination device for fruits such as apples.

The determination device is for use in sorting fruits according to their categories. The determination device comprises a driving mechanism such as a conveyer chain for successively carrying the fruits along a sorting line with the fruits mounted on pallets.

Disposed in the vicinity of the sorting line, an image-pickup camera picks up an image of one of the fruits to produce an image signal indicative of the image. Connected to the image-pickup camera, an -evaluating unit evaluates a particular one of the fruits in accordance with the image signal to produce an evaluated signal indicative of a result of the evaluation. Connected to the evaluating unit, a deciding unit decides into which categories the particular fruit is classified.

The above-mentioned device requires, however, a large amount of processing time due to complex and bothersome calculations. In addition, this device carries out classification of the categories, in accordance with the result of the evaluation, by using a predetermined threshold value. Accordingly, such a device is disadvantageous in working efficiency because the threshold value should be predetermined when even the same fruits such as apples are different in variety.

It is therefore an object of this invention to provide a determination device which effectively decides categories of the fruits.

It is another object of this invention to provide a determination device of the type described, which is capable of rapid tint decision processing with high working- efficiency.

It is a further object of this invention to provide a determination device of the type described, which can be used for detecting a defect on the fruits.

It is yet another object of this invention to provide a determination device for use in sorting fruits with a neural network.

Other objects and advantages of this invention will be clear on the description proceeds.

Summary of Invention In accordance with the present invention, it has now been found that the foregoing objects and others can be readily achieved.

Whilst particularly appropriate for the sorting of fruit the invention also may be applied to the sorting of other objects (particularly industrial products) which are presented successively to an image detector.

Thus according to the invention there is provided in one aspect a determination device for use in sorting objects into categories, comprising image pick-up means to which said objects may be successively presented, said image pick-up means being adapted for picking up an image of one of said objects to produce an image signal indicative of said image, evaluating means connected to said image pick-up means for evaluating said one of the objects in accordance with said image signal to produce an evaluated signal indicative of a result of evaluation, and deciding means connected to said evaluating means for deciding into which categories said one of the objects is classified, characterized in that said deciding means is a neural network.

Brief Description of the Drawing The invention will be described by way of example with reference to the accompanying drawings, wherein Fig. 1 is a schematic structural view of a conventional determination device for use in sorting fruits according to their categories.

Fig. 2 is a block diagram of a conventional processing unit for use in the determination device illustrated in Fig. 1; Fig. 3 is a block diagram of a processing unit for use in a determination device according to a first embodiment of the present invention; Fig. 4 shows an example of a picked up image of a fruit for use in describing an evaluation operation of an evaluating unit in the processing unit illustrated in Fig. 3; Figs. 5(a) to (c) show waveforms obtained for a fruit of which surface markings are stripes; Fig. 6 is a block diagram of a neural network for use in the processing unit illustrated in Fig. 3; Fig. 7 is a block diagram of a processing unit for use in a determination device according to a second embodiment of the present invention;; Fig. 8 is a view for describing a conversion operation of a color to tint converting unit in the processing unit illustrated in Fig. 7; Fig. 9 is a block diagram of a probabilistic neural network for use in the processing unit illustrated in Fig. 7; Fig. 10 is a block diagram of a processing unit for use in a determination device according to a third embodiment of the present invention; Fig. 11 is a view showing an example of divided sub region constructed of 2 x 2 picture elements in the processing unit illustrated in Fig. 10; Fig. 12 is an explanation view showing a condition where a picked up image is divided in the processing unit illustrated in Fig. 10; and Fig. 13 is an explanation view showing an example of a decision result for the picked up image in processing unit illustrated in Fig. 10.

Detailed Description of the Invention Referring to Fig. 1, description will at first proceed to a conventional determination device in order to facilitate an understanding of the present invention.

In Fig. 1, pallets 20 are connected back and forth with each other for mounting fruits 21 thereon. In the example being illustrated, the fruits 21 are apples. The pallets 20 are driven by a well known driving mechanism such as a conveyor chain 22 to carry at a predetermined velocity on a fruit sorting line as depicted at an arrow A. At one side of a moveway of the pallets 20, disposed is at least one image-pickup camera 23 such as ITV (Industrial Television) camera. For clarity, only one image-pickup camera is shown in Fig. 1 though three or more image-pickup cameras are preferable. Illumination lights 24 are provided beside the image-pickup camera 23 for illuminating the fruit 21 of which image is to be picked up.At the other side of the moveway of the pallets 20, a detector 25 is disposed to detect that a particular one of the pallets 20 is within an image-pickup range where the image-pickup camera 23 is capable of picking up an image of the particular fruit on the particular pallet. On detecting the pallet within the image-pickup range, the detector 25 produces a detection signal.

The image-pickup camera 23 and the detector 25 are connected to a processing unit 26'. In the example being illustrated, the illumination lights 24 are also connected to the processing unit 26' in order to control illumination timing for the illumination lights 24.

However, it is readily understood that the illumination lights 24 are not- necessarily controlled with the processing unit 26' and may illuminate the particular fruit all the time during the operation of the determination device. The picked up image is supplied to the processing unit 26' as an analogue image signal in response to the detection signal delivered from the detector 25. The processing unit 26' is connected to an input/output unit 27 which comprises a keyboard 271 and a display unit 272.

Referring to Fig. 2, illustrated is an example of a practical structure of the conventional processing unit 26'. The analogue image signal is divided into red, green, and blue analogue picture signals which represent red, green, and blue pictures, respectively. The processing unit 26' comprises red, green, and blue analogue to digital (A/D) converters 41R, 41G, and 41B in parallel which receive the red, the green, and the blue analogue picture signals, respectively, which are supplied from the image-pickup camera 23. The red, the green, and the blue A/D converters 41R, 41G, and 41B convert the red, the green, and the blue analogue picture signals delivered from the image-pickup camera 23 into red, green, and blue digital picture signals.The red, the green, and the blue A/D converters 41R, 41G, and 41B are connected to red, green, and blue image memories 42R, 42G, and 42B, respectively. The red, the green, and the blue image memories 42R, 42G, and 42B are for use in memorizing or latching the red, the green, and the blue digital picture signals as red, green, and blue memorized image signals.

The red, the green, and the blue image memories 42R, 42G, and 42B are connected to an image processing device 44'. The detector 25 is also connected to the image processing device 44' through a gate circuit 43 which produces an image-pickup timing signal in response to the detection signal supplied from the detector 25.

The image-pickup timing signal is for use in controlling the image-pickup timing for the image-pickup camera 23 and image information receiving timing for the red, the green, and the blue image memories 42R, 42G, and 42B.

The image processing device 44' is constructed of, for example, a microcomputer or the like. This image processing device 44' supplies information which is required for deciding categories such as surface marking of the fruits 21 (Fig. 1) in accordance with the red, the green, and the blue memorized image signals memorized in the red, the green, and the blue image memories 42R, 42G, and 42B. More specifically, the image processing device 44' comprises an evaluating unit 441 and a deciding unit 442'. The evaluating unit 441 is connected to the red, the green, and the blue image memories 42R, 42G, and 42B and evaluates the particular fruit in accordance with the red, the green, and the blue memorized image signals to produce an evaluated signal indicative of a result of the evaluation.The deciding unit 442' is connected to the evaluation unit 441 and decides into which categories the particular fruit is classified by using a predetermined threshold value. The deciding unit 442' produces a decided signal indicative of a decided category. The decided signal is delivered to the input/output unit 27 to make the display unit 272 display the decided category. As a result, the conventional determination device is defective in working efficiency, as mentioned in the preamble of the instant specification.

A little more in detail, the determination device is implemented in accordance with first through third aspects of this invention. In the first aspect of this invention, the categories comprises a "surface marking present" category where the particular fruit has surface markings and a surface marking absent" category where the particular fruit has no surface marking. In the second aspect of this invention, the categories are grades into which the particular fruit is classified in accordance with a tint thereof. In the third aspect of this invention, the "defective" category where a fruit having a defect belongs and an "indefective" category where a fruit having no defect belongs.

Fig. 3 shows a processing unit for use in a determination device according to a first embodiment of the present invention. The processing unit 26 in Fig. 3 comprises similar parts designated by like reference numerals as in Fig. 2. Description of such parts will be omitted for the purpose of brevity of the description.

The processing unit 26 comprises an image processing device 44 into which the conventional image processing device 44' illustrated in Fig. 2 is modified as will later become clear. The image processing device 44 comprises the evaluating unit 441 and a neural network 442 instead of the deciding unit 442' illustrated in Fig.

2.

The neural network 442 decides, in accordance with the evaluated signal supplied from the evaluated unit 441 that whether the particular fruit is classified into the "surface marking present" category where the particular fruit has surface markings or the "surface marking absent" category where the particular fruit has no surface marking. In the example being illustrated, the surface markings are stripes while no surface marking is solid color. Instead of the stripes, the surface markings may be other pattern such as spots.

The neural network 442 is connected to the keyboard 271 of the input/output unit 27 for use in supplying, on later described training operation, as a teacher data representing one of the "surface marking present" category and the "surface marking absent" category to the neural network 442. Description will be made as regards the training operation and the decision operation carries out by the determination device for fruits having the above-mentioned structure, using an example where the fruit is an apple. At first, the training operation will be described. The decision operation will be described later in the following.

Training Operation: As well known in the art, prior to carrying out an actual decision operation, the neural network 442 must be trained on the basis of a predetermined training algorithm such as a back-propagation training algorithm with the teacher data representing one of the "surface marking present" category and the "surface marking absent" category by using evaluated signals obtained by the evaluating unit 441 whenever the image-pickup camera 23 picks up the image of each of the apples to be classified into one of the "surface marking present" category and the "surface marking absent" category. The training operation comprises first through fifth steps as follows: First Step: First, as sample data, a few or dozens of apples are prepared, which have either surface markings of stripes or no surface marking of solid color.The apples are placed on the pallets 20 on the fruit sorting line and the categories of the apples are supplied to by using the keyboard 271. This input data is transferred to the neural network 442 as the teacher data.

Second Step: When the pallets 20 carrying the apples move on the fruit sorting line 22 and a particular pallet arrives within the image-pickup range for the image-pickup camera 23, the detector 25 detects this to produce the detection signal to the processing unit 26.

Third Step: Simultaneously, the image-pickup camera 23 picks up an image of a particular apple on the particular pallet to supply its image signal consisting of red, green, and blue components to their respective image memories 42R, 42G, and 42B through the A/D converters 41R, 41G, and 41B. Each image memory latches and memorizes each image information.

Fourth Step: The evaluating unit 441 of the image processing device 44 calculates, as illustrated in Fig. 4, the evaluated signal with the memorized image signal for a transversal line (horizontal scanning line) of the picked up image. The evaluated signal has an evaluated value which is obtained in the form of f = (r, g, b) where each of r, g, and b is the evaluated value corresponding to brightness of one picture element consist of red, green, and blue image components.

Fifth Step: Supplied with the evaluated signal, the neural network 442 carries out the back-propagation algorithm with the teacher data representative of one selected from solid color and stripes, given by the keyboard 271. In this way, the neural network 442 is trained so as to produce a correct decided result.

The initial operation often causes incorrect decision, which will be reduced as the first through the fifth steps described above are repeated. The above operation are repeatedly continued until the incorrect decision frequency is less than the predetermined value.

The detail of the back-propagation training algorithm is discussed in an article "PARALLEL DISTRIBUTED PROCESSING", vol. 1, MIT press, 1986.

Now, description will proceed regarding to calculation operation for the evaluating unit 441 with reference to Figs. 5(a), (b), and (c).

Fig. 5(a) shows an example of a brightness signal level obtained for the fruit which has the surface markings of stripes. A pulse waveform as illustrated in Fig. 5(b) can be obtained with the differentiated brightness signal level. Negative components of the pulse waveform are inverted to be decided whether or not each pulse is larger than a predetermined threshold value. The evaluated value is then calculated, for each of the red, the green, and the blue image components for each line, with respect to the number of pulse having a value larger than the threshold value. It is noted that the evaluated value is calculated at least one line.

Turning to Fig. 6, description will proceed to the neural network 442. The neural network 442 comprises an input layer IP, an intermediate layer IT, and an output layer pp. The intermediate layer IT may be called a hidden layer. The input layer IP is supplied with the evaluated signal which consists of red, green, and blue evaluated values ER, EG, and EB. Outputs of the input layer IP are supplied to the intermediate layer IT.

Outputs of the intermediate layer IT are supplied to the output layer OP. Outputs of the output layer OP are collectively produced as the decided signal which consists of two decided components "surface marking present" and "surface marking absent". The neural network 442 is trained with the back-propagation training algorithm by using the teacher data It is noted when a fruit of another kind or the same fruit of different variety is used, only the above-mentioned training operation is required for carrying out the decision operation for the substituted fruit Decision Operation: The decision operation comprises first through fifth stages as follows: First Stage: First, the fruit sorting line is actuated with the apples on the pallets 20 on the fruit sorting line Second Stage:: When the pallets 20 carrying the apples thereon move on the fruit sorting line 22 and a particular pallet arrives within the image-pickup range of the camera 23, the detector 25 detects this to supply the detection signal to the processing unit 26.

Third Stage: Simultaneously, the image-pickup camera 23 picks up an image of a particular apple on the particular pallet to supply its image information of red, green, and blue components to their respective image memories 42R, 42G, and 42B through the A/D converters 41R, 41(3, and 41B. Each image memory latches and memorizes each image information. In this event, the image information has the data amount, for example, of 192 X bytes in total under the assumption that the red, green, and blue components for every one picture element are represented by eight bits with the image constructed of 256 by 256 picture elements.

Fourth Stage: The evaluating unit 441 of the image processing device 44 calculates the evaluated signal of the evaluated value f = (r, g, b) for the transversal line of the image, as illustrated in Fig. 4, in the same manner as described above.

Fifth Stage: The neural network 442 decides whether the particular apple is classified into "surface marking present" category or the "surface marking absent" category. The result of decision is visually indicated on the display unit 272 of the input/output unit 27.

Instead of such a visual indication, the result of decision may be audibly indicated.

Referring to Fig. 7, a processing unit 26a for use in a determination device according to a second embodiment of the present invention comprises similar parts designated by like reference numerals as in Fig. 3 except that the image processing device 44 is modified into an image processing device 44a as will later become clear. Description of such parts will be omitted for the purpose of brevity of the description.

The image processing device 44a constructed of, for example, a microcomputer comprises an evaluating unit 441a and a probabilistic neural network 442a. The evaluating unit 441a comprises a color to tint converting unit 441a-1, an eliminating unit 441a-2, and a calculating unit 441a-3. As described above, the image signal is a color image signal which consists of red, green, and blue components.

Temporarily referring to Fig. 8, the color to tint converting unit 441a-1 converts the color image signal delivered from the image memories 42R, 42G, and 42B into a tint image signal which consists of H (hue), S (saturation), and I (intensity) components. In this event, as well known in the art, the hue H represents fundamental attribute of color such as red and yellow. The saturation S represents vividness of color such as vivid and dim and the intensity I represents lightness of color such as light and dark.The hue H, saturation S and intensity I are expressed by the following equations (1), (2), and (3), respectively: H = 1/2K(7t/2 - tan (F/S) + C), (1) S = 1 - min(RGB)/l, (2) and I = (R + G + B)/3 (3) where R, G, and B represent levels of the red, the green, and the blue components of the color image signal and F in the equation (1) can be obtained in accordance with the following equation (4), (5): F = (2R - G - B)/(G - B). (4) (o if F > 0 c=( (s) # if F < 0 Turning back to Fig. 7, the eliminating unit 441a-2 eliminates unnecessary picture elements from the tint image signal. Colors of unnecessary picture elements are previously known such as the color of background, shining caused by light source, and a vine of the fruit in the image so that decision, regarding whether or not they are the unnecessary picture element, is made for each picture element of the image by using the values of hue H, saturation S, and intensity I obtained by the above-mentioned equations (1) through (3). The eliminating unit 441a-2 produces a necessary image signal with the unnecessary picture elements eliminated from the tint image signal.

Connected to the eliminating unit 441a-2, the calculation unit 441a-3 calculates, as the evaluated signal, a statistic signal indicative of statistics in the determination area by using the necessary image signal having the hue H, saturation S, and intensity I where the unnecessary picture elements are eliminated.

The statistics are, for example, medians of the hue H, the saturation S, and the intensity I components of the necessary image signal. The statistics may be averages of the hue H, the saturation S, and the intensity I components of the necessary image signal. The calculated three kinds of statistics are supplied to the probabilistic neural network 442a.

The probabilistic neural network 442a is connected to the calculating unit 441a-3. The probabilistic neural network 442a decides the tint of fruits to distinct the grade thereof in accordance with the statistics supplied from the evaluating unit 441a.

The Probabilistic Neural Network is, for example, disclosed in Chapter 12 in Neurosoftware Manual issued by HNC Co. in United States, under the title of "Probabilistic Neural Network".

Referring to Fig. 9, simple description will be made as regards the probabilistic neural network 442a.

The probabilistic neural network 442a is for use in pattern classification and comprises an input slab IS (Input Slab), a pattern slab PS (Pattern Slab), a summation slab SS (Summation Slab), and a win slab WS (Win Slab). Nodes for each slab are called as a processing element (PE) and there are N-th processing elements in the summation slab SS where N represents the class of a pattern to be classified.

The input slab IS is supplied with an unknown pattern having a certain input vector x. Each processing element of the pattern slab PS has respective weighted vectors kj (j represents an integer of 1 to p and p represents a dimension-of the vector). On operation, the Euclidean distance dk between the input vector and the weighted vector is calculated by using the following equation:

In the pattern slab PS, gk is calculated in accordance with the following equation (6) by using the Euclidean distance dk calculated by using the equation (5) to produce it to the summation slab SS.

Provided that ~ represents a smoothing coefficient.

The gk is equal to a value of the Gaussian distribution of dk withwk being centered.

Next, in the summation slab SS, calculation is made for the following equation by using the

Provided that mc represents the number of the processing element in the pattern slab PS connected to each processing element in the summation slab SS.

In the win slab WS, a class is selected, as the determination result, where a value V obtained by the following equation is maximized: V = hC2c(fc(dk)) (8) where h and P are parameters for adjustment of each c c class.

Description will be made as regards the training operation and the decision operation carried out by the determination device for fruits having the above-mentioned structure, using an example where the fruit is an apple. At first, the training operation will be described. The decision operation will be described later in the following.

Training Operation: The Training operation comprises first through fifth steps as follows: First Step: First, as sample data, a few or dozens of apples are prepared, each of which has one of grades as the categories. The apples are placed on the pallets 20 on the fruit sorting line and the grades of the apples are supplied to by using the keyboard 71. This input data is transferred to the probabilistic neural network 442a as the teacher data.

Second Step: When the pallets 20 carrying the apples move on the fruit sorting line 22 and a particular pallet arrives within the image-pickup range for the image-pickup camera 23, the detector 25 detects this to produce the detection signal to the processing unit 26a.

Fourth Step: The image processing device 44a carries out the following processing in accordance with these image information. First, the color to tint converting unit 441a-1 converts the color image signal of red, green, and blue components of each picture element which are read out from the image memories 42R, 42G, and 42B into the tint image signal having the hue H, saturation S, and intensity I components.Subsequently, unnecessary picture elements are eliminated from the tint image signal by the eliminating unit 441an2. Colors of unnecessary picture elements are previously known such as the color of background, glare caused by light source, and a vine of the fruit in the image so that determination, regarding whether or not they are the unnecessary picture element, is made for each picture element of the image by using the values of hue H, saturation S, and intensity I obtained by the above-mentioned equations (1) through (3). Those determined as the unnecessary picture element is eliminated from the image. An area subjected for tint decision is the remaining area resulting from the elimination operation of the unnecessary picture element.

In addition, the statistics in the determined area are calculated, by the calculating unit 441a-3, by using the tint image signal having the hue H, saturation S, and intensity I where the unnecessary picture elements are eliminated. The calculated three kinds of statistics are supplied to the probabilistic neural network 442a.

Fifth Step: The probabilistic neural network 442a sets these three kinds of statistics in the data area for the grade supplied from the keyboard 271.

The above-mentioned first through fifth steps of the training operation are carried out for all of the apples prepared. It is noted when a fruit of another kind or the same fruit of different variety is used, only the above-mentioned training operation is required for carrying out the decision operation for the substituted fruit.

Decision Operation: The decision operation comprises first through fifth stages. The first through the third stages are similar to those described above in conjunction with the first embodiment of this invention and explanation of such operations is therefore omitted.

Fourth Stage: The color to tint converting unit 441a-1 converts the color image signal of red, green, and blue components of each scanning line which are read out from the image memories 42R, 42G, and 42B into the tint image signal having hue H, saturation S, and intensity I components.

Subsequently, the eliminating unit 441a-2 eliminates unnecessary picture elements from the tint image signal.

In addition, the calculating unit 441a-3 calculates the statistics in the determined area by using the necessary image signal having hue H, saturation S, and intensity I components where the unnecessary picture elements are eliminated. The calculated three kinds of statistics are supplied to the probabilistic neural network 442a.

Fifth Stage: The probabilistic neural network 442a decides the tint of the apple of which image is picked up to distinct the grade thereof by using the statistics set in the data area. The result of decision is visibly or audibly indicated properly by using a well known method.

Referring to Fig. 10, a processing unit 26b for use in a determination device according to a third embodiment of the present invention comprises similar parts designated by like reference numerals as in Fig. 3 except that the image processing device 44 is modified into an image processing device 44b as will later become clear. Description of such parts will be omitted for the purpose of brevity of the description.

The image processing device 44b comprises the evaluating unit 441, the neural network 442, and a final determination unit 443. The neural network 442 is for use in deciding whether or not the defect is present on the particular fruit in accordance with the evaluated signal supplied from the evaluating unit 441. The final determination unit 443 carries out final determination in accordance with a decision result of the neural network 442 in the manner as will later be described in detail.

Description will be made below as regards the operation for determining the defect of the particular fruit of which variety is new by using the defect determination device for fruits. When determining the defect of the new fruit, the following training operation is carried out to make the neural network 442b train the characteristic of each defected portion.

Training Operation: The training operation comprises first through seventh steps as follows: First Step: Put the fruits having the defected portion on the pallets 20 on the fruit sorting line.

Second Step: The pallets 20 carrying the fruits thereon move and a particular pallet arrives within the image-pickup range of the camera 23. In this event, the detector 25 is disposed so as to detect the particular pallet.

Third Step: When the detector 25 detects the particular pallet, the processing unit 26b separately takes in and latches to memorize them in the image memories 42R, 42G, and 42B red, green, and blue components of the image signal delivered from the camera 23.

Fourth Step: The image of the defected portion is isolated by a man from the picked up image as a small divided region as illustrated in Fig. 11.

Fifth Step: The isolated image is divided into sub regions (small regions) constructed of (K x L) picture element (where 1 < K ~ N/2, 1 # L# M/2: N and M represent the vertical and transversal number of images of the image memories 42R, 42G, and 42B, respectively) as shown in Fig. 12. The evaluating unit 441 calculates the evaluated value for each sub region, which is used as the training data for the defect.

Sixth Step: The first through the fifth steps are repeated until five to ten training data are collected for each defect.

Seventh Step: Make the neural network 442 train the characteristic of each defect by using the training data.

After completion of the above-mentioned training operation, the following determination operation is carried out.

Determination Operation: The determination operation comprises first through sixth stages as follows: First Stage: Put the fruits 21 on the pallets 20 on the fruit sorting line 22.

Second Stage: The pallets 20 carrying the fruits thereon move and a particular pallet arrives within the image-pickup range of the camera 23. In this event, the detector 25 is disposed so as to detect the particular pallet.

Third Stage: When the detector 25 detects the particular pallet, the processing unit 26b separately takes in and latches in the image memories 42R, 42G, and 42B red, green, and blue components of the image signal delivered from the camera 23.

Fourth Stage: The picked up image is divided into sub regions constructed of (k x L) picture elements (where 1 < K# N/2, 1 < Lc M/2: N and M represent the vertical and transversal number of images of each of the image memories 42R, 42G, and 42B, respectively), as illustrated in Fig. 12. The evaluating unit 441 calculates the evaluated value for each sub region. Subsequently, the neural network 442b determines the attribute for each sub region (such as normal color, color of the defect, and color of the rust). Examples of the evaluated value are averages of each of RGB or statistics in other color spaces.

Fifth Stage: Defect maps 50, 51, and 52 are prepared for each defect in accordance with the decision result in the neural network 442 as illustrated in Fig. 13. The defect map 50 is a map for use in detection of a bruise. The defect map 51 is a map for use in detection of a sunburn.

The defect map 52 is a map for use in detection of a rust.

Sixth Stage: By using the defect maps 50, 51, and 52, the size and number of defects are calculated in each of the defect maps 50, 51, and 52 to make the final determination unit 443 carry out the final determination.

In the above-mentioned embodiment, described are regarding to the color determination for fruits.

However, it is needless to say that this is applied to the defect detection of industrial products having a defect which is distinguishable on the basis of the color difference.

Claims

1. A determination device for use in sorting objects into categories, comprising image pick-up means to which said objects may be successively presented, said image pick-up means being adapted for picking up an image of one of said objects to produce an image signal indicative of said image, evaluating means connected to said image pick-up means for evaluating said one of the objects in accordance with said image signal to produce an evaluated signal indicative of a result of evaluation, and deciding means connected to said evaluating means for deciding into which categories said one of the objects is classified, characterized in that said deciding means is a neural network.

2. A determination device as claimed in Claim 1, wherein said image signal is an analogue image signal, said device further comprising analogue to digital converting means connected to said image pick-up means for converting said analogue image signal into a digital image signal, and memorizing means connected to said analogue to digital converting means and said evaluating means for memorizing said digital image signal as a memorized image signal, said memorizing means supplying said memorized images signal to said evaluating means.

3. A determination device as claimed in Claim 1 or Claim 2, comprising carrier means for successively carrying said objects on pallets along a sorting line to present them to the image pick-up means.

4. A determination device as claimed in any preceding claim, wherein prior to carrying out an actual decision operation, said neural network is trained on the basis of a back-propagation training algorithm with a teacher data representing one of said categories by using evaluated signals obtained by said evaluating means whenever said image pick-up means picks up the image of each of the objects to be classified into said one of the categories.

5. A determination device as claimed in any preceding claim, wherein said categories are grades into which said one of the objects is classified in accordance with a tint thereof.

6. A determination device as claimed in any preceding claim, wherein said neural network is a Probabilistic Neural Network.

7. A determination device as claimed in any preceding claim, wherein said image signal is a color image signal which consists of red, green, and blue components, said evaluating means comprising color to tint converting means for converting said color image signal into a tint image signal which consists of hue, saturation, and intensity components, eliminating means connected to said color to tint converting means for eliminating unnecessary picture elements from said tint image signal to produce a necessary image signal, and calculating means connected to said eliminating means for calculating, as said evaluated signal, a statistic signal defining said necessary image signal.

8. A determination device as claimed in Claim 7, wherein said statistic signal represents a median of each of said hue, said saturation, and said intensity components of said necessary image signal.

9. A determination device as claimed in Claim 7, wherein said statistic signal represents an average of each hue, said saturation, and said intensity components of said necessary image signal.

10. A determination device as claimed in any preceding claim, wherein said categories comprise a "defective" category in which an object having a defect belongs and an "non-defective" category in which an object having no defect belongs.

11. A determination device as claimed in Claim 10, wherein said device further comprises final determination means connected to said deciding means for carrying out final determination in accordance with size and number of said defect when said deciding means decides that said one of the object is classifed into said "defective" category.

12. A determination device as claimed in any preceding claim, wherein arranged for sorting fruits.

13. A determination device as claimed in Claim 12, wherein the fruits are apples.

14. A determination device as claimed in Claim 12 or Claim 13, wherein said categories comprises a "surface marking present" category where said one of the fruits has no surface markings and a "surface marking absent" category where said one of the fruits has no surface marking.

15. A determination device as claimed in Claim 14, wherein said no surface marking is solid color.

16. A determination device as claimed in Claim 15, wherein said surface markings are stripes.

17. A determination device as claimed in Claim 10 and Claim 12, wherein said defect is one selected from a bruise, a sunburn, and a rust.

18. A determination device for use in sorting industrial products according to their categories, comprising carrier means for successively carrying said industrial products along a sorting line with said industrial products mounted on pallets, image pick-up means disposed in the vicinity of said sorting line for picking up an image of one of said industrial products to produce an image signal indicative of said image, evaluating means connected to said image pick-up means for evaluating said one of the industrial products in accordance with said image signal to produce an evaluated signal indicative of a result of evaluation, and deciding means connected to said evaluating means for deciding into which categories said one of the industrial products is classified, characterized in that said deciding means is a for neural network.

19. A determination device substantially as herein described with reference to figures 3 to 6 or 7 to 9 or 10 to 13 of the accompanying drawings.

20. Sorting equipment incorporating a determination device as claimed in any preceding claim.