Winter Nats Final Analysis

UTR-1.0-logoAs I mentioned in my previous post, the folks at Universal Tennis Ratings did some extensive pre- and post-event analysis of this year’s Boys and Girls 16s and 18s Winter Nationals draws and outcomes (click here for their online work).

In addition to the numbers, UTR co-creator Dave Howell provided the following in-depth analysis:

Upsets abound . . . or maybe not so much!

Scrutinizing the main draw of the Winter Nationals B18s, you can find 17 matches which went against the seedings. Seems like a lot, but on closer inspection only 5 of those matches were contested by players whose ratings (UTR) were farther apart than 1.0. And even among those, some were mere hundredths of a point outside the 1.0 UTR standard. It’s fun to look for these things, but is there anything to be ascertained from all this that can lead to improving player experience? What are your thoughts?

One more thought . . . about 1/3 of main draw matches were played by opponents whose ratings were farther than 1.0 apart. Another way of putting that is 2/3 of match opponents were within 1.0 of one another. That sounds pretty good.

The upset trend continues in G18s, but not really.

Again, if you look strictly at lower seeds and unseeded players knocking off higher seeds, you come away thinking there are a high number of upsets, 16. But the UTR summary only found one upset. Which means over 98% of the time the higher-rated player won when the match-up showed players to be more than 1.0 UTR apart.

Another look at total matches inside vs. outside 1.0 shows almost half the matches (48%) were outside 1.0. This indicates a pretty wide range of levels for this event. On top of that 61% of those matches outside 1.0 were Decisive*.

Only 43% of matches inside 1.0 were Competitive*. We’d like to see better than 50%, but this percentage is almost 3 times better than competitive matches outside 1.0, 15.25%. I’m pretty sure these players can do better.


*Competitive, Routine, and Decisive Matches: A match is considered Competitive when the losing player wins more than 50% of the minimum number of games needed to win the match. Similarly, a match is considered Routine if the losing player is only able to win between 34% and 50% of the minimum number of games needed to win the match. Lastly, a match is considered Decisive if the losing player is only able to win less than 1/3 of the minimum number of games needed to win the match.


12 thoughts on “Winter Nats Final Analysis

  1. This proves one thing. The owner of the data is the only one who can’t create an accurate ranking from the data. UTR is spot on perfect. Tennis Recruiting is spot on perfect. And the USTA rankings are a farce. You cannot build proper seedings off of them and you certainly can’t accurately put the right kids in the draw off of them. The amount of worthy kids who didn’t get in was extremely high. And the amount of kids who looked like it was their first year of tournament tennis was high. This is on top of the fact that it snowed in Arizona this year? Why do we have an outdoor tournament in a region of the country with snow or frost at that time of year? There are all kinds of indoor clubs with open courts at that time of year. The flights in Phoenix were fine, but getting to Tucson is almost a guarantee layover for most people? And since it is a level 1, shouldn’t it look and feel differently than a regular sectional tournament?

  2. I just need to comment both on the post and the first comment.

    First, I like UTR/TennisRecruiting and USTA ppr is poor by comparison. BUT…

    The post is seriously out of line by saying there were 16 upsets when looking at USTA seeding and only one by UTR ranking. Thats some serious spin. Why does UTR get to say “1.0 difference to count as upset”? But then saying “If a seed loses to a lower seed, it’s an upset”? C’mon. Thats a lot of wiggle room for UTR and none for USTA.

    Playing this game, USTA could spin it like this: UTR upset is ANY lower rated player beating a higher rated player (not the 1.0 padding); and USTA upset is “Only if an unseeded player beats a seed”… Now the numbers will favor USTA!

    So don’t snow us with “98% of the time the higher-rated player won when the match-up showed players to be more than 1.0 UTR apart”. And by the way..I didn’t go line by line, but UTR analysis isn’t even counting right by their own criteria! In G18, Pick was 1.5+ lower than Peus…why didn’t that count as an upset?

    Again, I’m a fan of UTR/head-to-head ranking…it’s good, but don’t throw this sort of spin job.

    Now for the first (and only post)…like so many that show up on these junior boards. Want some cheese with your whine? We’ve done all the National tourneys and loved them all. Clay, Hards, Winters, Easter. All are put on by good people who try very hard. All have their own challenges (I’ll take Phoenix in Dec over Memphis in JULY!!)

    And as for seeds and selection…seems to me seeds did fine overall…not too many unseeded players in later rounds. I find it hard to believe that we had an ‘extremely high’ amount of worthy kids that would have been bracket busters didn’t make the cut. More likely, and without even lookingk, I’d be willing to wager nearly any/all alternates would have been 1st or 2nd rd losers.

    And here’s a few tips ‘wits end’:
    1. Tuscon is a just a nice drive in a rental car away from Phoenix.
    2. Junior tennis is a very long road and if you look for the good instead of the bad, it will be a lot more fun.

    Many thanks to all that made WinterNats great. And did anyone see those 16/18s trophys that were hand-carved by native americans…they were majestic!! What a great touch!!

  3. Hello “Enjoying the Ride”,

    We appreciate that you “like UTR”, and wish to provide additional clarification to the above article so that you have the opportunity to like us even more!

    You mentioned “The post is seriously out of line by saying there were 16 upsets when looking at USTA seeding and only one by UTR ranking.”

    The main premise of the analysis states that even though there may have been a high number of seeded players losing to unseeded players, that these outcomes were NOT upsets.

    The Press (and many in the public) often suggest a tennis match upset occurs when a seeded player losses to an unseeded player. (Reference earlier today where ESPN reported “Top-ranked Novak Djokovic upset by Ivo Karlovic” at )

    The above analysis does not suggest that the USTA has a definition for a match upset, but does state that many perceived upsets were not upsets at all. The rationale for our claims are based on how Universal Tennis defines an upset, which is when a player whose UTR is more than 1.0 higher than the player’s opponent ends up losing the match.

    To date the UTR system has processed 1,874,073 match results, and counting. Based on our analysis, we have made suggestions as to what we believe are competitive, routine and decisive match outcomes — as well as upsets. We use this information to help tournament directors improve the experience for their participants; teaching pros and coaches to help their students measure improvement; tennis associations to help implement player retention initiatives; college coaches to assist in selecting student-athlete recruits that are a good fit for their teams; and a measurement for tennis players to help find the appropriate level of competition.

    We continue to provide support to our many customers that are associated with the USTA, in a joint effort to improve and grow tennis in the U.S. We also share your sentiments that the organizers of the USTA National Winter Championships did a phenomenal job, and deserve much praise.

    You are correct that we do like to “spin” and “play games” — but only on the tennis court.

    Best regards,
    Bruce Waschuk
    CMO, Universal Tennis, LLC.

  4. So in the spirit of ‘liking you more’, I reread the above a few times and I see where you are coming from. After my first read, I came away with impression of “UTR is 98% correct” vs seedings showing many upsets. (and it’s not hard to come away with that..)

    But if read more slowly, I see your point and that you are just saying “upset, smupset…” and tossing out the idea of an upset when you have <1.0 gap. And it's what I knew already about UTR. It's a great way to get good matchups which is what we all really want.

    I REALLY like how you can go to your site, pull up any college team and instantly see where you would fit in. That is a really neat tool for any junior when considering their college options so hats off for that data!

    One question I have is how are you able to integrate male/female when there is presumably little intersecting data? (And I will say that looking at local players, my gut tells me the girls and boys are probably pretty well ranked relative to each other)

    Keep up the good work.

  5. 30 degrees in the morning this year….. Doesn’t make sense.
    Would vote for a warmer location or a location with indoor courts.

  6. Comparing UTR to USTA rankings is kind of like shooting fish in a barrel. I don’t think there is any question about that. I’d like to see a similar comparison to Tennis
    My observation is that TR.NET is still a more accurate predictor of head to head match up’s when you are looking at players in the same class year. Different ages, etc, UTR takes over.

    I can’t back this up with any sort of study or statistical analysis, but when I look at players from our section, within the same grad year, where the orders are different, TR.NET is definitely more accurate.

    I suspect it’s a data issue. Looking across some players, TR seems to be picking up more matches, about 25-30% more per player. For instance, UTR doesn’t seem to be picking up the back draw of many sectionals.

    Thoughts Bruce ?

  7. Hi “Enjoying the Ride”, you asked how UTR is able to integrate males and females into the same ratings.

    Our ratings for boys and girls are tightly correlated based on the practice match results we monitor. Plus, tennis associations (like USTA NorCal) run co-ed tournaments. Feedback has been positive, and additional mixed gender data continues to increase as more organizations run level-based tournaments.

    The UTR Team

  8. Hello “Ajt”

    Both UTR and TR.NET offer a unique set of features and benefits for those juniors, and their parents, when planning for college tennis. As a consumer, I’ve used both services (and several other “tools”) to help my daughters find their college tennis programs.

    I am not aware of comparisons between UTR and TR.NET that have been made public, that evaluate the accuracy of head-to-head predictions. Having an applied math background, my bias would be to comment on analysis based on mathematical merit.

    We are also not in the position to comment on the quantity of matches another organization uses to determine their rankings and/or ratings.

    Universal Tennis has processed main draw and back draw results for USTA sanctioned junior tournaments (BG 12 – 18) since September 2013. If you feel results from a specific tournament have been missed, please forward the draw link to info(AT) for our UTR Team to verify.

    We currently processes results from the ATP, WTA, ITF, ITA, USTA, Tennis Canada, select U.S.-based high school tennis, and others. This includes ITF U18 Junior Circuit, USTA open and junior circuits, select Tennis Canada junior events, and select U.S. high school tennis association results. ITA includes NCAA events and ITA/USTA summer circuit. We anticipate additional high school associations, as well as national tennis federations, will join the above list during 2015, and submit their results to be processed by the UTR system.

    There are now more than 180,000 players from 193 countries rated by the UTR system. However, instead of us continuing on about the amount of data we process, we would suggest that you (and others reading this post), sign up for free at and make your own conclusions on how well UTRs work.

    Best regards,

    Bruce Waschuk
    CMO, Universal Tennis, LLC.

  9. I’ve signed up with UTR and it is pretty neat to see ratings based on skill. For example if your player played foreign player who is playing ITFs, and he lost, it shows up on TRN as a loss to not rated player but on UTR it shows what level that player is and I think that is a good thing. Kids obsess over these losses to not ranked showing up on TRN. As for winter nationals I thought UTR was very accurate, but there are going to be some non competitive matches when you have kids ranked in 600 and kids ranked in top 50 USTA rankings playing in the same tournament. I am not against this since sometimes going to a tournament for experience is a more useful than playing same players at the same level in your section.

  10. This is the best discussion I have seen on this site (and there have been many). The bottom line is this. The UTR ranking system is now THE ranking system for NCAA coaches. As I have been consistent in saying there has to be a universal ranking system that includes USTA and ITF matches. I was saying that prior to knowing about UTR.

    The USTA rankings are significant in that you have to abide their ranking system in order to get quality matches at the regional/national level and ultimately at the ITF level.

    As I have also been consistent in stating, in order for the USTA ranking system to become relevant aside from getting into the appropriate tournaments the USTA has to adopt a January 1 age cut off to become compliant with ITF and to give their rankings more credibility instead of being a floating target.

  11. The winter nationals have a deep history in Arizona spanning back to the Fiesta Bowl and Copper Bowl. Tuscon did an excellent job in hosting the tournament! I have spent many a Christmas in Scottsdale and it has also been an excellent host!

Share Your Thoughts

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: