DA Software

Video Transcript

My name is Daniel from instruments and data tools. And I'm here to present, the introduction to the DA software, which is a user interface that is able to import and process DA meter data. This is a project that we've done in collaboration with the horticulture innovation fund. And, and the da software is available on the HIN website.
the features of the  DAsoftware essentially have two types. One is the ability to import, to load and to process the DA meter data so that you can do basic statistics on the data. You can label with your own labeled information, for instance, your crop, your blocks, or varieties, and so on. You're able to export the data that was so labeled into Excel for instance. And the second important features is the ability to make charts. So you can build a time charts of time plots, which are good to understand trends for instance, a maturity of your fruits, based on the DA meter data. And you can create bar charts for comparisons of the maturity  staus, for instance, in different blocks or between different varieties.
The DAsoftware, it's provided as a compressed folder, that's a zip folder for the windows operating system, it works with windows seven, windows 10. The first time that you are installing it, you extract the folder into your computer drive. You run a VC redist X 64 file, which is needed to install the DLL for windows. And then you can run the DAsoftware.exe. to launch and, work with the program. Then from that time on, you just have to run the DA software.Exe file, to run, to run the software.
So once you download the software, you can, unzip it into your computer drive and then it comes in the form of a folder, which has all of these files. The ones that are important, as we said, is the VC redist X 64,exe that you have to run the first time you run the installation. And then after that, you are just around the DAsoftware.exe file to launch the program.
So that's what we're going to do. So double click on DAsoftware.exe file and a, and you'll see a screen that looks like this.
So before we start, let's look at the different components that we have here. The main page is made of different tabs. So the first tab will contain all the data, all the numerical data that you will be able to import from the DA meters files.
And once we will, or we will load it up, you will see that it looks more or less similar to an Excel spreadsheet. And, and all the other tabs are used for charts and plots. So we have different types of plots that we can do, and we will work step-by-step, over this to understand what is the difference in what they can be used for.
On the left hand side, then we have a box which have filters. So filters are used to select specific characteristics of your data. So you can select, for instance, the season of your data, or you can select the date range in which you want to look data for. You can select the market, which is the DA meter, indicator that can be set once you're taking the measurements and you can select your own labels and we will see what this means, called cultivars blocks and row/plot.
And then you have a menu, which is essentially, the usual file input  -output. So you can load/save any import files and export files. And we will see an example of all of this. And then there is a, an edit, button which contains menus for editing the labels that you want to select for your for your data. And again, we will explain this step by step.
The first step we're going to do is import the data from the DA meter, that this is the first operation you want to do when you are dealing with new data. And then you're going on file and import. And then you have this popup window and the system is looking for textual text data. It is the typical data that are saved by the DA meter.
So here I have a folder with some sample data. This can be any folder where you stored the data from the DA meter. So I open this folder and I have all these files. These are the DA meter, this is a file saved by the DA meter itself. So these are in the form of DA, and then there is a date, for instance, in these cases, zero one zero to 16, and they are all text files.
So you can import a single file, just select. And for instance, click open. Or if you hold down the shift key, you can select more than one file for instance, I hold down the shift key and I click on this one. It will select them, a number of them. And then you can open them at same time in the DA meter. Or if you actually have them, you want to select them all, you simply press control an A, the letter A, and then it will select all of them, which we are, this is the thing we are going to do now. So control A, select, all files now are selected, and then we can open them. Click open, and you'll see that the software is loading them all up, and the software is only selected the relevant numerical values out of these files. So these files, if you open them up, they contain a lot of numerical values that are not relevant. And so these are automatically discarded by the DA software. And the only things you will have is a numerical indicator, just from one to the number of scans that you have, the date, the time of that scan. The marker of the DA meter, and then at the end, the DA value over here. Now the market of the DA meter, that it can be selected, as you can chose to select or to set a marker before you take the measurement. If you do the marker, the column of the marker will have a number. For instance, here, there are some of the data, for instance a market in this case here 26, 63. if you don't choose a markeR, for your data, then the markeR will be automatically imported as negative one. That's just the conventions to say there wasn't a market selected, or set for that particular scan.
All of the data, all the columns of data that are populated, once you import the data, are fixed, you cannot modify those data. But what you can do is actually modified the empty columns here, cultivar, block and row/plot. So these are the labels that you can set. The DM meter will not know what you have been scanning, what type of fruit of what location you're being scanning, so you can, if you know those things insert these yourself and so you can label the data in a way that then it will be, it will make more sense for your operation.
Once you've imported all data, you can actually filter them to just select specific, for instance, dates or specific seasons or marker that you want to, visualize. So the easiest way to do this is for instance, say, let's look down here. So we have this, say this data corresponding to the marker two, four, zero, two.  So we can put here two, four, zero, two, and then click on apply filter. And now the only data that is visualized in here are the ones that are corresponding to the marker where we are, that we have selected. That's enabled you to just quickly scan through this data without having them all in one, place. If you want to go back to the previous visualization, then you have to click on clear filter and this will clear the filter and you go back to the previous visualization containing them, all data. The other things you can do as an Excel spreadsheet, you can actually order these files by clicking on the individual column here. For instance, this, the columns are naturally ordered by the number of the scan from one to, how many scans you have, but you can order it in a, so for instance, you can click on date.  This will be corresponding to the early scans, this correspond to a 2013, for instance. And if you click again, you will have instead, the date ordered in descending order. So you'll have latest scan in this case 2018. And you can do the same with marker. For instance, if you want, and or DA values, that's a, you can do any of the, of the reordering you wish in this visualization.
So far we have imported the data and we've seen how we can visualize or change the visualization of the data that we have. But to make it, to personalize this data and to introduce the information regarding your own operation, you would like to, to define labels instead, for instance, the cultivars or variety, the block and so on. And this is done by in the edit menu. For instance, let's go and edit cultivars. And, there is a windows that a popup. And in this case, here you can define your own specific cultivar. So in this case, I've selected two types of nectarines. So Rose Bright and August Bright. You can add as many cultivars or varieties that you want, so of any other crops or whatever it is, so for instance you can. And to do that, you can just click on add and say, we say Granny Smith and click okay. And that will be added to the list of cultivars that you have in there. And you can delete them and obviously edit them. If you want to change anything you can edit, or you can simply just deleted it . And this has to be done only once. So once you've done it the first time, then you'll have your list of cultivars or your list of blocks all the time saved in your software every time you open it. Don't have to do this every time. So once you are happy with that, you click okay. And then you're going to do the same thing say for blocks and say, let's add some blocks. Block one or any name that makes sense for your farm. Say block two. And add another one, block three. Okay. And click. Okay. Now these are a again saved in the configuration of your files, so you don't have to do it every time. And again , you can also introduce this in the same way, a row plot, if it's relevant for you, you don't have to do them all.  So only the ones that are relevant for your farm. So let's say I say one here, I know. Okay. At this point you can, you can start adding the markers on the, sorry, the labels on the data. So to do that let's for instance, suppose you select just a single season. And here, if I click on the season filter, you'll see that these are the seasons that are available in the data. They will just load it. So the data correspond to scans that were taken from 2013 to 2018. And so the software then automatically derives the scans in seasons. So, the first season then will be the 2013, 2014. So I suppose we just like this first season. And then we apply filter, remove the marker here.  And so we just select the data that belongs to the 2013, 2014 season. And they you go. So these are, as you can see that the date goes from say 2013, November, 2013 to. We need to reorder them, November, 2013 to March, 2014. Okay. Now, suppose that we want to say that these 2013 correspond to a cultivar. So if I go on the cultivar cell and double click on it, you'll see that there is a dropdown menu that is now appearing. And the dropdown menu's got the cultivars that we had set in our list. So in this case, we have Rose Bright in August Bright. So let's say that this is a Rose Bright scan.  So what I've done now is associate this label to the scan where previously, there was none. And so I would say all these 2013 scans belongs to these currently. So I want to extend this label to all of those scans. So to do so, the easiest way, so you could in principle do it cell by cell double-clicking, that will take a very long time. So the best way to do it, if you have lots of data, it's just essentially say control C .Press control C on the cultivars, on the cells that is, contains already the data. Control C and then, select the another one and do control V, and that will be copied in that cell.
Now, if you want it copied in all the on a range of cell, then the best way is select the range of cell holding the shift key down. So you hold the shift key, and in this case, I want to select all the 2013 scans. So I'll go down and look at the date over here and select all the 2013 scans, up to here. Okay. Holding the shift key down, I select them all and then control V to copy it inside all of the cells. Okay. So what we have done is associated a cultivar name to all of these scans. Now we can do the same four blocks. And you can do, if a double click on the cell? You will select from the dropdown list, the block, which that scan belongs to say block one. Now, if you do a mistake, say you, for instance, you copy the content of a cultivar  cells, control C into a block cell control V, what you'll see is that the cell will become, red. So there is an error there because this indicator, the Rose Bright name, it belongs to the cultivar, not the block. So you'll see that there is a problem there. So the cell will be coloured, red to indicate that. So  instead let's copy the block information in this. So we do control C on the block cell and, and then we'll select a number of cells and we do control V and then say, we'll do block two for some more of this cells. And then again, block three for others here. In this case I'm making this up, but obviously that will make sense once you have, for your operation, and control V. Okay. We won't do the row/plot as it works exactly in the same way. Okay.
So now at this point we've got the data that has been labeled. So we know that which cultivar belongs to, which scan belongs to which cultivars and which block. so the best way to add this to go ahead., After this is saving this, this configuration.
We go on file and we do save, okay. Save, the data will be saved in a form that is called data. So essentially is a binary dat file. So let's say that this is a nectarine and we call it nectarine dot, Dat. And save it. What this does is that the next time that we are working on this file, we don't have to relabel all the scans from scratch. We just open, the dat file. So to show how this works, let's close the software. Now we close it. So now I open the DA software again, the DAsoftware start with a blank data tab> so if we only import, if we import again, the data from the DA meter, the labels will be lost. so what we can do instead is load, the nectarine dot Dat file that we saced before. Here it is. In this case, we will pre-select the 2013 seasons as we did and apply filter, then we will see that we have the block information that we saved, previously. So you don't have to do this all the time. You don't have to do only the first time you load this, the stuff and from there-on, you can just save the data and, and keep working on it.
The last things you might want to do, once you have the labeled data this, is export them. So export - the export button will allow you to export a file that you can visualize with Excel. So let's click on export. This is, this will export at CSV file, which is a comma separated value file, which you can import and visualize in Excel. So for instance, let's make this, let's call it nectarine again. okay. This is being exported, so let's look at it. Here it is nectarine dot CSV. So if you double click on it, depending on your settings, you'll have this data can be visualized immediately from Excel.  Just let me close this up and show you that there are two options actually to export.
One is an export and one is an export with an F in brackets. F Means will export all that, the filter that we have applied. So in this case, we have applied the field as a, 2013, 2014 season. So if we do export F, then we can say, we can call it this nectarine, say 2013 and save, and now you can see, you can have this nectarine 2013 dot CSV. So if you open it up, then sure enough, you'll see only the 2013 files and here they are, the cultivars in block information that we have selected.
Now that we have labeled all our data, we can visualize them in charts. So to do that, we just go into the next tab. The next tab is called day season.. The day season tab will plot the DA meter value for each day and the number that will be plot will be the average for each day. And so you will see, say for instance, we are want to, in this case we have Rose Bright and August Bright. So we want to select the culltivar Rose Bright and that we want to plot that. So we select it here and we say plot. Now what we will see here is a dot. There will be a dot corresponding to each day. The dot is the average of the DA meter readings that you had on that day. So if you had more than one reading on the same day, for instance, on the whole block, that will be averaged here.
So besides this, this outlier in this case, probably this was a test measurement in this case, you'll see that the DA meter values decrease for us as you go ahead. That means obviously the fruit is maturing. And so you'll see that the decrease of the DA meter value. To visualiae, to work with this plot, you can use your mouse. For instance, if you click, you left click on the mouse, you can move the plot around. So for instance, this case, I don't want to visualize the first point, I can do this. With the mouse wheel you can zoom in, zoom out. and, and then you can change the appearance of this plot. For instance, you want the change, say the label here. So you double click on the label and you'll have here an editor in which say we want to just have the text Rose Bright on it, which would make it easier to see. And so there it is. And now if you click on the edge of this legend, you are able to place it any way you like in the plot. The plot, it also comes with, with the standard labeling. So they see the title of but you can again, double click on it and you'll be able to change, to change the text in the, in them. in the title and so on. You can also change the appearance of the plot. So the appearance of the symbols and so on to do that, you just have to select symbols and then right click. When you right click you'll have a dropdown list. And in this case we want to change the graph properties. And so for instance, the color or the line style on, so on. In this case, there is no line so we can decide, we want the line that is joining the different points. We can change the thickness of this line. We can change the color of the plot. We can change say the size of the dots and and how they look and so on. So anything that has to do with the appearance of the plot can be changed through the graph properties interface. Once you're happy with that, you can, click okay. Or if you want to reset, you just said, default, click on list, set default.  And it will go back to the, to how it was before. Say, we are happy with this, we can leave it there. And so now we have  the plot as selected. So now let's suppose we want to add another plot on this, on top of this, then we can simply select the, our second cultivar, August, And prot again. And now the August Bright data will be overlaid on the same plot. Now or the August  Bright, sorry. Yes. The August bright data will, we're taking from January, 2014. And so you'll see that this is where this points are appearing and then we can do the same thing. Say we can click on that and say, this is going to be August. And, then we want to change the appearance of this. Say we want a blue. And so now we have, and we can change the title, Rose Bright August Bright.  Okay. And it's, that's a, we can show this chart in this way. Again, every dot in this chart is the average DA meter reading for that day.
Once we are happy with this chart, you can either, you can save it. And so in this case, saving means we will export an image file, a PNG file. And so we can, and you can use this to either, in other presentations or other documents that you might want have.
And so if you go and look, we go and look at those files. We have this, chart dot PNG file that is the chart as we actually visualize it and set.
Okay, this is the comparison for, for the cultivars. Now if you want to plot a different indicator, say we want to plot the blocks instead, and we don't want to mix them up with, with, cultivar plot. What we do is we clear the plot. So we, click on clear. Now we, the plot is good to  start again, so we will delete the cultivar information there and say we select block one instead. And so we can plot block one data. This is the block one. And this is again, block one as a function of the date. So these are the date from October, 2013 to November, 2013. We can add the block two data on it and this will be the data that we labeled as block two appearing in the plot.
The overlay plot is done in, to solve, essentially one problem that you might encounter with the day season plot. So let's look at it. Let's look at the day season plot first. Soppose we want to plot the data corresponding to the 2013, 2014 season. And we'll plot it here. Now, we haven't selected any of the labels. So the data will all look the same or the same column of the market. Now, suppose we want to add the data from the 2015, 2016 season on it. And we would like to do some compaison. If we plot, then what we'll see is, this is a time chart. So obviously the data from 2015, we'll be updating correspond into 2016, corresponding to the year 2016. And so it is hard to make a comparison between this  data and the dada of the previous year, just looking from this chart. So the day season chart is only good to compare data belonging to the same season, the same cultivar during the same season. If you want to compare data from one seasons to the other, the best way is in fact using the overlay plot.
So let's say we upload the 2013, 2014 season, seaon first, and that's how it looks. And so let's call it, say 2013, 2014. Okay. And then we select the 2015, 2016 and plot again, and now the new data will appear in the same scale. So you can, compare this data in the same scale.
So what happens here is that the scale goes from July to the next June or the end of the next June. And, and the season instead will be at like a different indicator of. the marker. So in this case, we'll have that the 2014 season is the blue markers and the 2015 season is the red one. So if you actually ploting data corresponding of the same block or the same variety, on two different seasons, you will be able to compare them in this graph. And so understanding if, on many days, for instance, the maturity of the DA meter that is off corresponding to the previous season and have an idea when you expect, for instance, the harvest compared to date, to that data. The day season and overlay plot are time charts. So the day season will be used to compare the data belonging to the same season while the overlay plots is good to compare data from different season. And, but the data is always in as a function of the day or the date in which the scan was taken.
Now, if you want to compare instead, different cultivars or different blocks, then you can use the cultivar or block or row plot tab. They work in the same way. Let's do, for instance, the block one, just to, we clear all the, all the filters and then we want to select, say 2013 season and a block one, which we selected and we do plot. In this case, we will have instead a bar plot. Now the bar plot, once you've plotted this, the first time, the bar plot there only one bar and the one bar is the average DA value of the block one.
Now we can add the block two. That's all now, you'll see the second bar happening and then block three and you'll get the third. Okay. So this is good to compare for instance, different of your labels as they do for the DA meter.  Now once we've done this, we essentially plotted the whole entire set of block one data, but obviously you can also select a date range. And so you can only select, a specific, range of data corresponding to block one. And so you can compare those values in different tabs. So this,  so all the cultivar blocks and row/plot charts are good for comparison between different, of your labels. So different cultivars, different blocks. So this is not going to be a time chart.. As for the rest, everything works exactly the same way. So you can, you can change the label of, the labels of your plot as we did before you can change the colors and you can change the, the titles and so on, exactly in the same way. Similar, it is going to be a cultivar plot, if you want. One thing that it's important to, to remember is that you need to make sure that the data you are plotting are actually corresponding to the, sorry. One thing that we want to make sure of is that the filter that we apply to our data correspond to actually some data that is, that is selected in some cases, there might not be data that correspond to all the filters that we have set.
So for instance, we have set the block three filters. Only for the data or the 2013, 2014 season. And so here I could go and plot those. But now suppose, let me clear this up, let's suppose we want to plot say the 2016. 2017 season for block three. Now, then there was no data, labeled as block three in the 2016 season.  So if we plot that, we get an error message that says there is no data to plot. So there is no data that corresponds to all the filters that we've selected. In this case, we selected block three in 2016, 2017 season. So there is no data that has correspond to this search, to this field. And so it is case that you'll get an error message that say there is no data to plot. So the idea is that you will, you can aggregate data according to different things. So you can aggregate this plot, block, date. And so you can compare, say block one of this week with block two the same week. So you can compare a DA value and say that block's ahead of the other block in terms of where it is in its maturity, for example, the DA.  Suppose you have a one variety in two different places you want to, compare those two. Or yeah, something like that. It could be a couple of days to a couple of weeks apart in terms of maturing.