A Simple Example of the Validation Process
From Unofficial BOINC Wiki
Contents |
[edit] Introduction
It is sometimes hard to understand how Validation works and what parts of the process might look at. We see the outcomes of the Validator but do not know why we were part of the Quorum of Results or why we were excluded.
I have made some simplified drawings that I am going to use to illustrate how this happens. These are examples, they do not match the real world Validators for any Project.
More to the point, the Project's Validator programs are working in a multi-dimensional space with many internal items in the Result Data Files that are involved in the action. I am going to use an example where there is only one value that is represented as a magnitude and position.
In the real world, the Validators test many aspects of the Result Data Files and make a determination after all of those tests.
But, for the sake of example I am going to keep it simple. You can just think for the purposes of understanding that each aspect of the Result Data that is tested uses its "own" copy of this example. So, if the SETI@Home Project was making the test of 47 different internal values, this example set would be done 47 different times with different data as input.
The real wonder is not that we have Validation failures, it is that we have Validation success!
[edit] Example Introduction
In this first example we are only going to show some of the basic components that we have and will be dealing with.
Since the actual values are not important the graph is unlabeled, but you can see the plot of one Result with it position on the number-line "someplace". It has a height of some value, what that value is, is also unimportant.
However, you can see that there is an error limit box that at this time is centered around the one Result Data point that we have and will be making a comparison with.
[edit] Example #1
Here you can see where we have sent out a Work Unit, and at this time we have enough Results back to form an initial Quorum of Results.
In this case you can see that the three Results that have been returned call fit within the error limits. Now, one of the points that I know people get hung up on is the fact that we do not, in fact, know what the answer is to the data. So, all we can do is to process the data and then compare answers.
In this case we can see that even though we moved the error limit box, all of the data points will fit within the error limits. This is the point that is hard to see.
The error limits are established in relationship to the Result Data Points themselves. There is no "magic" to tell us what is the "right" answer. In this case we have potentially three "right" answers.
And those potentially correct answers are "close-enough" to be considered the "same".
[edit] Example #2
This is the same as Example #1 above, with the addition of the fourth Result. However, as you can see, the fourth Result is just too far out to be included within the Quorum of Results. (Note: it is 'out' on position, being too far 'right', although the magnitude is good.)
In this case, this Result would be excluded from the Quorum of Results and would not be Granted Credit.
[edit] Example #3
If we take Example #1 once more, and only change the order in which the Results are examined (or received), we can see that we can establish a different Quorum of Results.
Yet these are the exact same Results!
This is why you could be in one Quorum of Results and not in another. (Note that this time the fourth result received is too far 'left'.)
[edit] Example #4
Again, if we take Example #1, and only change the order in which the Results are evaluated we will see something like this example. But, in this case, the last Result comes in after the Quorum of Results is established and the Canonical Result has been chosen.
Here you can see that if the first Result (in line, and the first received) is chosen as the Canonical Result, the fourth Result will be out of the error limits and will not be Granted Credit.
[edit] Example #5
If we take Example #3, and only change the order in which the Results are evaluated and which Result is the Canonical Result we will see something like this example. But, once again, in this case, the last Result comes in 'after the Quorum of Results has been established and the Canonical Result is chosen.
Here you can see that if the third Result (in line, the second returned), is chosen as the Canonical Result, the fourth Result will be within bounds of the error limits and will be Granted Credit.
