Video transcript: Summerfruit Webinar August 2021- Hand held sensor for Stonefruit

So here, I'm going to present briefly the results of our, the first season that we've done on the Food Agility, CRC project with Ag Vic and Summerfruit Australia. And this was the assessment of the accuracy of the hand-held spectrometer that we developed. The basic idea of this spectrometer is that you've got a single system that can taken in the orchard. So it's a battery operated system. It works over smart phone or a tablet and it can take a scan of a fruit of the tree without picking it. And with a single scan, you get a whole range of parameters that might be of interest. And so what it changes is essentially a calibration model that has happened in the background.

That's how the scanner look like. So this is cover that is flexible. So it can wrap around branches and so on. So we're going to reach the fruit to, to be scanned and,  and then  scan anything in real time, get the parameters of importance, of interest  on your smartphone.

So this spectrometer briefly works in different modalities. And so they are acquired in the same scan. So just by pushing a button, then you'll get all these scans taken. The data is in real time uploaded to servers and through the prediction models down the server, then the the result is returned on a smartphone. What we did the project  for the agility this year, the past season was about developing models for firmness and sugars in Brix, and tried to also determine the occurrence of storage disorders in fruit. So there's two different experiments are gonna talk about now.

Okay, so the first one was done  within Tatura, and so that's the way of five cultivars that were chosen for the experiment. And in all cases the fruit were scanned with Rubens and then, and then the destructive testing was done on the fruit, and so the conventional measurement of sugar, firmness, and then eventually other parameters like the DA meter, the ethylene was also collected and dry matter and and I believe all the other things like size and so on. So then after this was done, will be developed a machine learning model that takes the, essentially the spectra, that Rubens acquires, and trying to predict the the value that the destructive testing was. And so then we can, with those information we can evaluate what is the accuracy. So for instance, here is an example done on,  there's, this is Snow Flame 25 that's white peach. So here on, in these graphs on the X axis, you have the measured quantity done by the destructive testing, and on the Y axis you have what is predicted by the spectrometer. So you would like to have those datas sitting nicely on the 45 degree lines, so that's here. Then there is some scattering around that and that enabled us to evaluate what is the accuracy. And so here we are within five to 10% values the error of the primary data. And then in this case, we did model for all of those things are here to see, for instance, the Brix, firmness and  the ethylene. This was done on over 300 skins for cultivar. In this case here, the graph was shown the data we're taking, the scans were done in the lab, but we also tested the scans done in the orchard. So for instance, here, there was this September Sun.  These scans here, about 90 scans were taken all in the field with Rubens and then the fruit was picked, then the destructive testing was done after that. And we do not see any decrease in accuracy because of that. And so we wanted to test these things and, to have a good test of the, of work in this  scanner in the orchard, and that was important.

And the other experiment that was done was to use this scanner, the Ruben scan to try and predict the occurrence of storage disorders. So this was a bit more complicated. So there was a number of fruit and this scan was done  at harvest, then the fruit put in  cold storage for a variable number of weeks, and they were also then taken out cold storage and left at room temperature for a variable number of days to simulate retail. And what we try to do is use the scan in all those  steps the fruit will scan as well. And at the end, they were visually assessed for any types of internal disorders like browniness, mealiness, rubberiness.

What we found is that if we scan a fruit, we have more than 90% accuracy, we can predict if that fruit at that moment will have a storage disorder. So if we have, for instance, there is browning in there. Now, if we instead of scanning at that  moment, we used the scan that we're taking earlier, when the fruit was first taken out of cold storage. So we have the more than 80% accuracy in the occurrence of internal disorders in a few days. So by the time that the fruits, let's say for three days after the removal of cold storage. And then we did the same with the scan at  harvest. So what happens in that case? We can use the scan at the moment, which food is harvested, to predict the likelihood  of this food displaying some internal disorder in a few weeks time. So after it'd been through cold storage and eventually retail time. And so this was obviously it's done only on a single curly wire. This was the yellow the whole had a yellow peaches.

So it, we probably require s, needs a little bit more validation, but this data is very, it's very promising for what essentially trying to use the scan it to have a prediction in the future of the likelyhood of the quality of this fruit.

Okay. Now the last things I want to say is that  all these models were done on individual cultivars. But in some cases it might be easier to actually deal just with genetic models. For instance, here is a yellow nectarine or white peach or something like that rather than having individual cultivar models. So in this case, there were some data from different types of yellow, different cultivars of yellow nectarine, here, listed here and there were taken from different orchards and in different places in this case was Cobram, Shepparton. And and so we can also generate models that are genetic in the sense as you can see there's Brix and firmness for a yellow nectarine model. That's, that's about it. I would like if I have a few more minutes, actually just show a demo file of how the system can work.

I will share my screen where I have, all right. I have here is. Is that visible? That is the screen of my tablet in there. And I have with me, the prototype spectrometer properly visible in this small camera image. I have connected the scanner to my tablet, where there is the app in there that is visible. So that's connected to the device, and then I'll go on a scan tab. In this case, I haven't got a stone fruit, I've got an apple here, so i've selected apple Granny Smith over here. And I can start the scan. The app asked me to put some labeling there. So for instance, I have, we can say it's a block one or something like that,  labeled. And then then we scan the  apple. If you can see my camera that's how the apple's in the device. And then  get the scan in there, now it's been scanned. And it takes a few minutes, now the data has been transferred from the device via Bluetooth through the, through to the app, and through the server. Now the model is analyzing this data and returning a value for, that is probably a bit small, but it's in there, firmness and Brix value. In this case,  these are the models that are available for this particular fruit. But then you can, if we have more models, say for instance, you need to have ethylene or you need a DA meter, or need a colour or what not, that can be just listed in there, in one single scan. So that's the idea. So one can repeat the measurement a few times and  get a few stats on the same, on a block for instance, and this can be a quick way to estimate the quality of the fruit.  And these data is also available on a dashboard or a  spreadsheet of scans that have been taken and the values of the firmness and Brix for the various fruits. So that can be easy to export for instance, in Excel or something like that.