New Measurement Based rating Catamaran handicap system (NMBR)


The goals of this system is to provide the sailors of small catamarans (up to 22 foot , 6.70 mtr length) with a more dependable and generally accepted rating system. It's intended use is in open class racing on closed course regatta's.

The main characteristic of this system is that it is completely measurement based (Like Texel, SCHRS) and therefor does not require 'rating development time' for new designs as is the case with yardsticks. Nor does the rating numbers show any creep over time when good crews either enter or leave the class. An added advantage of the system is that everything can be explained on physical grounds. There are no more magical black boxes.


Back ground and correct use of the system

Lets be blunt, a rating system does not have to be highly accurate. Of course with sufficient effort and resources one can implement a very complex Velocity Prediction Model and get spot on rating numbers. However there are also two good reasons why this has not been done in the past even though the technology to do so has been around for some time.

The first reason is the system major design goal has always been that it must be practical in application. Complex system are quickly disgarded by both sailors and the race committees. Sailors don’t want to wait long for the results nor want to work with large confusing tables of ratings for their own craft. Race committees really don't want the hassle and delays that these complex systems bring with them.

The second reason is that a highly accurate system is useless in real life application. After a certain minimal accuracy the final scores hardly change. Typically the spread between crew ability in a one-design class easily leads to spreads of 20 minutes per hour over a complete fleet. This where inaccuracies in a decently accurate rating system will only account for offsets as big as 1 or 2 minutes per hour racing. This leads to the situation where the net result of a much more accurate system is only that one or two crews move one or two places up or down, not much else. Very often the top 3 listing doesn't change at all. Therefor; the challenge is to design a system that has sufficient accuracy and then concentrate on making it as practical as possible. Making it so to both sailors and race committees.

Why a new rating system at all ? Because the current systems are either impractical or have a fundamentally and unacceptable large inaccuracy. Several systems used today did well in the past but have lost accuracy due to increased variation between the catamarans of today. Comparing a Hobie 14 to a Prindle 18 is something else than comparing the same Hobie 14 to a rare, lightweight, cat rigged modern doublehander like the Marstrom M20. It is not that the old systems have deteriorated but rather that they are now required to cover a much more diverse fleet. Something for which they were never developped. To get this corrected is the responsibility of the new rating system

A point of note is that rating systems can be made relatively more accurate by making them less sensitive to factors beyond a Race Committees control. Such factors are a gradually changing wind direction or changing wind strength. Note that these impact negatively on a one-design class just the same. However in some cases it can be proven that an open class fleet is impacted more by these factors. A good trick is to group together boats of similar performance or setup. With sufficient participation a regatta can then be made up of say a spinnaker fleet. a low performance fleet and a singlehander fleet. A change in wind strength will impact differently on a spinnaker boat than on a singlehander but much the same on two spinnaker boats or two singlehanders. To maximize rating system performance in this way will always be the responsibility of the race committee and not of the rating system. However the rating system can still maximize the accuracy when rating a diverse fleet under relatively constant conditions and it should very much try to do so.

The hardest test is to accurately rate a fleet of 20 boats or less, as at most clubraces, here one is forced to group all makes together. However here we also arrive at some responsibility of the sailors themselves. As indicated earlier a one-design crew may finish anywhere between 1 second to 20 minutes behind the leader. In a small fleet of different type catamarans the difference between the first finisher, who finishes in 60 min, and last finisher is typically just over 30 minutes. This means that on average there is a gap of 1.5 minutes between each placing. With an rating system inaccuracy of say 1 minute per hour (2%) the chance that 97 % of the placings won't change at all with a more accurate system is very big indeed. Even of the crew that do exchange places, nearly all will only climb or fall one position in the final listing. Meaning that if you weren't in the top 3 to begin with that you most definately lost the race on skill rather than on ratings. That is under the assumption that the use system is accurate to 2 % or less. This is arguably not the case with the systems in use today.

From this we take away that there is at least a 20 minute spread in crew ability per hour and only some 13 minute spread between a Inter 20 and a Hobie 16 as a result of boat design. This should focus the attention of the crews on learning to sail better. However it must also be said this is only possible when a rating system satisfies the experiences on the water rather accurately. Not prefectly, but accurately enough so that any offsets are dwarfed by the differences related to skill. The rating system should at least do what feel logical to the sailors. That will go a long way to have it accepted by them.

This leads us to the design goals of the NMBR system.


The design goals of the new handicap system

-1- Must produce new ratings immediately as soon as a new production or one-off model is available.
-2- Must produces the regatta results quickly and transparently.
-3- Must be practical in use to both the sailors and race committee.
-4- Must adjusts the ratings according to input parameters that feel right to experienced sailors.
-5- Must strikes the best balance between accuracy, complexity and required processing during a regatta
-6- Must be relatively easy to maintain or fine-tune with time and increased understanding
-7- Must show no obvious or significant bias in any conditions to any design.

Significant, as used in point 7, is defined as 'must not contribute to the end result in a share larger than
1/10th when compared to crew skill'. Peferably less than 1/20th


Improvements

Older systems often used statistical data or a limited number of measurements to arrive at a rating. With the decline of catamaran racing and racing with smaller fleets, the statistical systems have trouble preventing rating swings due to influx of good crews as well as getting enough data points to make a dependable estimate for the performance. Measurements systems on the other hand still only measure a few parameters and implictely assume that all other aspects are the same between boats or do not matter. Of course the last assumption can not be supported anymore. There is too much to be done in order to remedy a statistics driven system. In addition, its working core is slave to a reality generating both good and disturbed data and that is to all-encompassing to be fully comprehended by an unskilled human brain. That leaves us with working out a new system using the concept of the "measurement based systems'. Despite its own drawbacks, this system provides the most accesible basis on which to base a new system. It is easily explained to unschooled sailors and its framework allows adaption to future developments. However, in order to provide a foundation to such an adaptation an ongoing statistical analysis of new developments will have to be performed . This is also needed to check wether the system is still up to the task. A measurement system may not be allowed to become stagnant and drift away slowly over time.

It was expressed by many sailors that the current measurement systems fail on points like correctly rating the addition of a spinnaker to a cat-rigged catamaran relatively to adding a spi to a sloop rigged boat. This is now a well understood phenomenon. Another good example it the problem of certain boats becoming relatively faster with increasing wind while others relatively slow down. Think of respectively a Hobie 16 and an Inter-20 in relation to a F18 catamaran. Current available systems do not adress these issues well enough and it is now possible to predict a likely handicap winner on basis of the wind conditions. The new system will correct these points.

An extra issue is the fact that some features of the system appear to be randomly applies or randomly applied. This is not good for the trust that the sailors are expected to put into a handicap system. Clearly a system that is not credited by the sailors themselves will go nowhere. This issue is also adressed in the new system.

Of course each system has its good points and its bad points so in developping the new system we have combined the good points and improved on the bad ones. And of course we have introduced a few new elements as well.

To cut down on required effort to get the system operational, the decision was taken to use as much of the parameters of the old systems as possible in a unmodified way. The end result is that we introduced 1 new input value and recycled all the known parametes and values into a new framework. When required this limitation can be abandonned and even more accuracy or simplicity can be introduced.


Important decisions

First, we have dropped the old standard class. Also known as the reference design. This design (class) was assigned a fixed rating from which all other ratings were derived. It acted as a pivot around which the other rating derived their (relative) meanings. This used to be the classic Tornado, that doesn't exist anymore. In other cases it was the Hobie 16 design, arguably unrepresentative of modern catamaran fleets. The new proposed standard is the Formula 18 class because of its large global presence and it expected domination in the future. Already cat sailors are basing their own performance to sailors in this class as there are always a few of them around. The F18 class is also very representative of modern catamarans racing as a whole. After all the foreseeable future looks like it will be dominated by formula classes.

Later in the project it turned out that the rating of the (new) Tornado class (double trapeze, spinnaker, bigger sails) is constant under the new system with the F18 standard. Ergo the new Tornado class may be regarded as sort of a second standard. The same applies to the Formula 16 class which actually intents to race doublehanders and singlehanders on the same course. This latter class may proof invaluable to fine-tune the ratings of the singlehanders to that of the doublehanded standard class. It is expected that these three classes together will provide a sound base which will produce dependable statistical data to proof or improve the rating system. Such key proving points (classes) are very important in keeping the system up to date and accurate.

Second modification involved the use of modest averaged adaptation. When a dependency was found to be present but insufficient data was available to quantify it really accurately then the concept of an 'centralized educated estimate' was used. This may sound inaccurate but it is actually a good way to improve on the accuracy while keeping the complexity of the system low. Compare it to the game where two people have to give a number between 0 and 10 and the one closest to the mystery number wins. One can choose to name 2 or 8 and be either really close or not close at all. An alternative is to choose 5 and be reasonably close whatever the mystery number may be. Of course, not choosing at all is assured to loose. Hence making an educated guess near to an expected average is always better than doing nothing. In the follow-up iterations the optimal point can easily and quickly be found. This binairy approach is the most simple and quickest way when using the very simple 'More' or 'less' qualifiers.

Thirdly, the parameter expressions were kept simple. An example, the number 1 may have been used in places where 0.98 would have been closer to the truth. However a formula with 10 parameters all containing 3 digits quickly becomes a very confusing expression. If using a single digit number only created an off set of a few seconds it was decided that this was an acceptable price to pay for reducing complexity and increase the ease of use. Remember that by creating the same offset in ALL ratings will cancel out much of the impact of such an offset. Nothing changes in the listing if all handicapped time results are off set by the same amount.

Fourthly, the model of the catamaran was kept simple. A single heeling force estimate is made for identical sail areas and luff lengths even though in reality there may be aerodynamical differences in the rigs. Examples of differences are : having a square top sail or not ; having a very low boom, having skegs or long boards. It is possible to include these differences and be more accurate but it will add a lot of complexity and will make using the system impractical really quickly. Think of what needs to be done when a crew decides to entlarge their square top ? The race and rating committees don't want this hassle and nor do the sailors. A difference was made were sufficient swing in rating was expected. An example of this is the way in which a mainsail and a jib contribute differently to a the heeling moment. A cat rigged catamaran with a given sail area will therefor have a different heeling moment estimate from a sloop rigged catamaran with the same sailarea. This while two different F18 implementations will get the same estimate. It is expected that the round-off error are far to small to constitude any serious threat to accuracy.

Fifthly, the ratings are split in two groups. One group gives the rating for below double trapezing wind condtions and the other group gives the ratings for above double trapezing wind conditions. In both groups a second rating is available for the spinnaker equipped version. This is a simplied version of the wind dependent ratings systems. It is adviced that when 2/3 of the fleet is double trapezing when going upwind that than the high wind rating group willbe used. NOTE, this can be dependent on the composition of the fleet. A fleet of A-cats with a few other singlehanders may qualify for this transition at a lower windstrength than a fleet of Hobie 16's with a few other doublehanders. It is left up to the race committee to decide which group of ratings is used.

It should be noted that the Race committee can make a decision for the use of either number on observing the fleet along. No wind measuring tools are required. Also the transition from non-trapezing tot trapezing is arguably the most important transition from one wind scale to another. Different designs diverge here a lot more than at other wind scale transitions. This both minimalizes complexity as make usage more practical while maintaining the most important performance related transition.

Last, it is strongly adviced that the ratings are used on closed loop courses. This maybe triangles. Windward- leeward loops or even rectangles as long as the point of the start is also the point of finish (by approximation). The ratings are derived for such courses. The new system allows ratings to be calculated for exclusive upwind, exclusive downwind and exclusive reach legs as seen in one–way distance races, however accuracy is expected to suffer a little in these applications. It was decided not to fully fine-tune the system for this application as that would add complexity disproportionally to the frequency this feature would be used. A specilized system can and will be developped for this specialized use. Strong grouping of similar boats (not from performance perspective but from the perspective of general setup) will greatly improve accuracy in these applications although it is understood that the ratings will be less accurate than when used on closed courses.

Wouter