I started a new thread because that other one is mainy a gripe fest...with many valid points being made at times.

I have also volunteed to (thanks for the warning JW) do something rather than talk about it.

I have proposed to have a meaningful discussion about a workable measurement rating system.

I believe that Wouter has done the legwork far better than anyone can fully realize. You know that he has , even if you don't want him to get the credit for it!

I am hoping that several of us can get together soon and review what has been done and go from there. Obviously Wouter will be our eMentor, if he is willing.

Until we actually produce useful numbers, all the bantering is silly.

I want a system that is RC friendly. RC folks should not have to defend or interpolate numbers and they should not have to keep reams of timed results.

In fact, I believe that most small regattas could be run quite well with level rating classes, with boats being placed in a class by similar handicaps, but racing scratch once so placed.

How are you going to attack this? Re-calculate based on elapsed times and publish "parallell" scoresheets? Work with arrangers or??

Rolf,

I guess I don't fully understand the qustion. Are you refering to potential usage of the system or gathering data to help justify numbers?

As someone racing in open class, I like the idea of level classes based on a rating range. But then I am not as competitive as some folks and would rather give up a position than finish well at the expense of pissed off competitors. If I want to get back into serious racing i will consider one-design.

Mugrace,


The old NMBR website is still up and running.

Plenty of info to base a new start-up upon there.

We may need to tweak it here and there but the basic factors and consideration are still valid I think.

http://www.xs4all.nl/~whijink/NMBR/

And modifying the system is not a major undertaking itself so if anybody has some misgiving about it then please lets us know.

Of course the basic working core was inspired by the Texel system and we all understand that that system is pretty good as it is. But NMBR did improve on it on several points.




Damn, Looking that website over I again impressed by the amount of work I had put into that system.

I think Mark Schneider worked out some race data with the NMBR numbers and found it to be promising.


Wouter

I am just curious on how you plan to make this a successful rating system in the USA? I dont want to be a killjoy, but both Texel and SCHRS are up and running, but you still use Portsmouth. How do you plan to change this? I think that is the hardest task at hand if this is to succeed. Perhaps some brainstorming on the subject is a good idea?

At this point it is purely a fact finding mission. If Texel or SCHRS are working then I would be in favor of going in that direction. I would love to hear objective dialogue on their successes and faults from folks that are using them. Wouter seems to have praise for Texel.


However, if the Wouter system has merit, then that might be the direction we will look.

Keep in mind that this is not a proposal to do away with DPN and I hope the people who have invested heavily in developing and maintaining the Portsmouth system don't look at this as a mutinous or undermining endeavor.

If it were to come to that, I would drop this in a heartbeat.

Even with some representative numbers in hand, we are only talking about regional events in the Mid-South (USA) where we have close relationships with one another and the race committees. Yes, I would say we would try parallel scoring to start with.

First, the USA needs some help with it's handicap system. It has had one single person (Darline) as the brain and brawn behind the system and she has done an absolutely astounding job keeping up with it...and I mean that. However, it's too much for one person.

Before I get into nitty gritty, I want to establish a couple of basics:

Handicap systems are imperfect by nature. I don't prefer to race under any handicap system...but when I do, I want it to be as accurate as possible.

I believe that both performance based and measurement based systems have flaws. I believe that both have unique strengths.

On the subject of Wouter and NMBR; Wouter had put a lot of energy in sketching out that system and I was starting to help briefly because I felt there was a need for improvement. I don't think Wouter will disagree that he can be passionately opinionated at times. It's probably safe to say that I suffer from the same character flaw from time to time...it didn't work out so well in that case....however, Wouter does have some interesting ideas and calculations about how to put a measurement system together.

Performance based systems (like Portsmouth) have the potential to be more accurate that measurement based systems but they pay the price of a weakness when applying a rating to new platforms, changes in platforms, or with "dead boat society" classes that typically race with boats in poor condition and some lesser practiced sailors. Reporting of regatta results is happening significantly less which is not helping with the current accuracy of the system.

Measurement based systems, on the other hand, handle new platforms and modifications to platforms quite well. They also don't rely on reporting of results (though I'm guessing they probably do back-analyze themselves from results) However, they suffer in the arena of being designed around and generally being less precise on potentials. I had previously mentioned that I felt the Texel formula was complex and Wouter has recently shown some examples where it is relatively straightforward...so I digress on that point.

I think the ideal system is a marriage of the two. Use a measurement system to establish new ratings, modification factors, and a minimum deviation to prevent positive rating creep. Use performance data to maintain and refine the numbers of boats. You leverage the strengths of each to cancel the weaknesses of the other.

As for the reporting thing, Colin, the creator of Sailwave, has already include FTP functionality for the UK yardstick in the software and is agreeable to the notion of doing the same for the US. Basically, you score your regatta in Sailwave and select "submit results to US Sailing" from a drop down menu. As long as you have an active internet connection, boom...they're submitted.

You might want to talk to me.....

No single system is perfect, changes in conditions affect all results.

So I propose a compromise to the above system, run both systems; measurement and performance. Each boat/class gets rated for each system it wants to be rated for. Times get plugged into the computer and three columns are produced; scratch, a measurement handicap result and a performance handicap result.

There are already systems in place for each handicap method, adopt them in your local area.

Spread the love...

If you do come up with a new system - consider the girth of the captain to give me a chance.

OK, I have just spent some time looking over the NMBR proposal and see nothing that doesn't make sense.

Wouter, you mention "fine tuning" often...and say that 2005 will be the watershed year to collect and process such data. I presume that is where the cat went turtle. Where are we with regard to that in the spring of 2009?

Jake, what would you propose to adjust or modify?

Jack

Wouter's major improvement over the existing Texel formula at them time was to define two wind ranges. The idea was that if 50 % of the boats were trapping out going upwind then you would use the second rating and vice versa.

Portsmouth currently uses Dpn 2 and 3 as one band for the rating. The newer design boats start to fly a hull in Beufort 3 and with spinaker's now in use... they are flying a hull down wind as well. This is non linear performance and we can't capture this in the small amount of data currently collected.

At the time Wouter was working on his system, he also had many technical calls that fall out of the engineering and was periodically consulting with Nico Boon and others in the Texel world with his ideas. Somehow, Texel changed their mind and decided to revisit the forumla and published their revised tables. I believe they incorporated a large number of the fixes that Wouter was using. My memory is that Wouter and I discussed the politics of bringing out a new rating system and since Texel had moved quite substantially towards the NMBR rating, it was not really worth it.

ISAF SCHRS also existed but the rule had not been updated in years and was much less popular then texel. A year later, Following the lead of TEXEL, the ISAF ratings were taken over by Scooby who led a signficant reworking of the ISAF ratings. Unlike NMBR and the Texel rating. ISAF SCHRS does not use a trap... no trap or windspeed two part formula.

The EU has no history of using a wind strength adjusted rating system. The difficulties in getting a Race Committee to move from a single number rating to multiple ratings is enormous. Only one software race scoring package will allow you to do this (Sailwave). All of the "simple" race scoring programs would require you to reload an entire new rating table. Race Commmittes just are not quick to switch race scoring programs... (Hey... it worked last year... It will be just fine... this year as well). The US is much happier using the best scoring program out there.. sailwave and we are used to using windspeed ratings for buoy races.

Distance races where the conditions could vary widely need a different solution.

As Wouter worked through the ratings, I tested the system on a few years of data collected by CRAC (forwarded to USPN since I was on the committe). It was a bit dicey because I had to remember whether an event was a high beaufort 3 race with 1/2 the boats trapping or not. I concluded the system was workable and did a good job of scoring the races. By good job... I value the experience and track record of the helm and expect the results to reasonably follow the sailor and less the boat. The modified ratings tables that Wouter generated were put into Sailwave and did a great job of scoring the races.

So, Where are we at:

1) for any measurement rating system, we will need to measure some US specific boats... actual physical measurements are required. Currently neither the New texel or the new isaf SCHRS can be used in the US without a Wild butt Guess at some ratings.

2) the US Shark is a tough boat for a measurement rating system to deal with. It's such and old school boat that it doesn't rate well.

3) A large amount of work goes into creating a portsmouth system or a measurment system and toes will be stepped on with any change from one to another. Texel led the way with a measurement based rating system and it quickly gained acceptance outside of the Texel race. Later, ISAF sanctioned a french effort to create the competing rating system. It's hard to see the value in a third rating system ... just because us Yanks wanted to be different. The world wide multihull council put this issue on the agenda. I would support a world wide system implemented in the US.

4) One feature that I think a system that the US would support uses at least two wind ranges for ratings. A single rating like the standard DPN should also be available for clubs that can't deal with a ratings table.

5) For what its' worth, I have a an easy to use finish time / sail number program that I use which makes recording and caluclating the results very simple. (It's used at Key West and on the Chesapeake for the big events.)

6)We should keep reporting to US Portsmouth since some clubs have a reason to score cat's with monohull dinghy's.

7) Even One design fleets benefit from seeing finish time deltas' rather then just finish positions. It lets a sailor have a real information as they try to improve their sailing.


Mark Schneider

All come and race in OZ...... It allways blows like snot around here and we never get off the wire unless it is time to change sides

Incorrect.

1, SCHRS is used far more widely than Texel, and always has been, people were just complaining it needed some work. We did that work. Texel is used in Holland and one club in the UK I know about SCHRS, is used in the rest of the EU and other parts of the world.

2, We do have a righting moment compenent of the formula that my team added when we did our review. Thus boats with lots of sail, and little righting momemn (RM) do get a more favorable rating than in the past.

come on, TA, thats a slight exaggeration. I did have one race a few weeks ago where i had to sit in on the hull.

On the subject of Jake and myself and being passionately opinionated, I remember no conflict between us at the time of NMBR and I feel the project was abbandonned for other reasons. Additionally, I much prefer a good fight over a "group hug and love another" all the time. I have no problems what so ever to working with Jake or others again. I understand that my personality can be a little bit abrassive (probably more then a little) but I must admit that the same character flaws also allow me to work through (compex) projects like NMBR very quickly. Two sides of the same coin.


On the topic of Yardstick systems (Performance based systems). In theory these can work well but only if the "system identification algorithms and supplied data" are well developped. This is actually part of my field of expertise as a Control Systems engineer. Having a well developped sys. ident. system is alot more involved then what current Yardstick systems are able to achieve. The quality fo the results, Ratings, are accordingly. We must take care not to assign a confidence in the yardstick systems that may be justified theoretically under ideal assumptions but are not supported by real life (sub ideal) facts. Mark Schneider touches on these point quite well in his posts.

With respect to measurement based systems. It is not forbidden or even impossible to use a Performance based (Yardstick alike) calibration mechanism to fine tune the (simple) describing function (i.e. regression). I know for a fact that Texel does indeed "Back check" the system using new race data and analyses whether a modification or adjustement is in order. Texel is not a static system and its revisions over the years have proved that. In fact, making a marriage between the measured based core and a performance based adjustment is ver much possible and adviced. NMBR was indeed developped using such a calibration method, although this is not documented on the website.

The more fundamental difference between a measurement based system and a yardstick system is that in the first you use ALL the data of all classes to fine-tune the describing function, where in yardstick systems you devided the total data set in significantly smaller sets to fine-tune each class rating individually. This is one reason why I favour measurement based systems so much. You make a lot better use of ALL the information that is included in the data set whereas yardstick system ignore significantly parts of it.

An example :

The perfromance of the occasional Prindle 16 that participates in a race in largely dominated by the very same limits as the Hobie 16's that participate. Systems like Texel use info milked from the H16 data to influence the P16 rating as well, as the describing function aims to approximate all designs accurately with only a handful of parameters. However, the Yardstick system suffers from dead-boat-society problems as only 1 P16 provides insufficient data of statistically poor quality to have the rating converge on. Afterall, who is sailing the P16, is he a good sailor, did he get caught in a windhole on the course etc.

This is one reason why new designs are always more accurately rated in a measurement based system then a yardstick system. No new design completely breaks with the performance trends as established by the other boats and the describing function. Typically, the adhere almost completely to these trends albeit at a slightly different location (coordinates). The describing function covers all coordinates whereas a yardstick system only covers a limited number of point that all coincide with existing designs.

No again, I underscore the fact that the describing function is not a fixed quantity. It can be adjusted on new race data if it turns out that by adjusting it coefficients it can become more accurate. In such a way a measurement based system can grow with the times and adjust itself to new developments. In some cases this means adding a new measurement to keep accurately rate old designs that don't go with the times. NMBR has done that and did indeed introduce a few additional measurements related to out dated designs.

So race result reporting is still much appreciated although much less important then under a Yardstick system. The system identification approach with a model structure to it (describing functions) requires much less data and can operate well with data of significant less quality. It is actually much less sensitive to measurement noise. I will leave the science stuff to this point.



Mark underscores a few good points. It is technically not difficult to make a rating system that does what we want it to do. Making is practical enough and attractive enough to have RC's use it is much less succes guaranteed. One lesson learned from Texe, when it too used two wind dependent rating numbers as NMBR had done, was that a mutany amng RC's killed that modification. Now we are back at single number handicap ratings again. Texel committee understands that this is less accurate but the real life won't swallow the revised system otherwise.

Therefor a good number of drawbacks assigned to measurement based system is not inherent to these system but rather forced upon the system by some groups with lots of power to make or break a system. I feel USA offers a good opportunity to break this particular dead lock as you guys are actually favouring wind dependent ratings. The USA could provide a succeful counter example that would eventually pressure RC's in EU to follows suit. By that Time I'm sure Texel and probably SCHRS will follow as well. Maybe that is the main beneft of NMBR, not a 3rd system competing with the others but as a crowbar breaking open the entrenched positions we find in EU and maybe the world at large.



Wouter

This appears to be a key element and as you say, it is already being used by RC’s in the US. I think two steps are sufficient until better supporting data can be collected.


I think that trying to go worldwide with a change is impossible and not even part of this discussion. Ditto for US as a whole or even mandated to a region. This would just be an experimental option for willing Event Organizers and their RC’s to try, either parallel with DPN or standing alone. No pretense of anything beyond that.


What boats are not on Wouter’s list other than Frankenboats and one offs?
Seems simple enough to get the measurements or slot them in with W.A.G. provisionally.


This is where my PHRF experience comes in: I would have a rating committee that could make arbitrary adjustments when there is a demonstrative anomaly.


We don’t want to ruffle feathers.


Thanks for this info Mark and keep up the good work. I don’t know if this has any legs, but if folks are talking then I suspect it does.

You have repeatedly proven others wrong even when they don't want to accept it. I hope you will jump back into the fray and become a vital resource in this endeavor. I don't think we have anyone else who can or is willing to do the work that you have and we are very lucky you are in the boat.

However, if your personality takes a turn for the better, we will cast you out with the bilge water.

I think can live with that.

Wouter