Wine Tasting Guide

The winery ratings evolution

By Tyson Stelzer

12 Jul, 2021

The Halliday Wine Companion winery rating system has been refreshed, evolving with the 2022 edition. Here, chief editor Tyson Stelzer explains the methodology behind the approach.

Star ratings provide a highly coveted and oft-quoted snapshot of the calibre of a winery, based on the ratings of its wines in recent years. In the past, James has applied a modicum of discretion in determining promotions, demotions and the rolling over of past years’ ratings. Under our new structure, in which tastings are shared across a wider team, a more transparent, objective and consistent system of applying ratings is appropriate.

James’ age-old criteria for calculating star ratings from wine scores have been upheld for consistency and continuity, with the same cut-offs maintained, as set out below. For the first time this year, I have introduced a three-year rolling average, in which ratings are calculated for each year and the average of the star ratings for the three most recent years is then rounded to the nearest 0.5. This removes subjectivity, providing a sufficiently up-to-date snapshot, while smoothing out the fluctuations of erratic harvests.

Blockquote Section Icon
The refreshed system removes subjectivity, providing a sufficiently up-to-date snapshot, while smoothing out the fluctuations of erratic harvests.

The three-year rolling average replaces James’ judgement calls, based on the most recent three years’ ratings, and his mercy rule, in which a demotion had previously been no more than half a star. Further, this also supersedes his decision that nobody’s rating would be reduced for the 2021 edition, due to drought, poor flowering, bushfires and Covid in 2020. Admittedly, none of these effects influenced the wines tasted for the 2021 edition, though they have, of course, significantly impacted our tastings for this edition.

To be fair, equitable and transparent, I have gone back to all 18,000-plus wine reviews in the 2020 and 2021 Companions and recalculated their star ratings based on the cut-offs (I knew I’d use that degree in computational mathematics one day!). These are the star ratings I’ve averaged together with the 2022 ratings, as calculated from the 9129 wines tasted for this edition, to determine the star ratings that you see throughout this year’s Companion.

Blockquote Section Icon
The result of these changes has been a slight shuffling in the ranks of our top-rated wineries to single out those that have set the very highest standards in the most recent seasons.

As in the past, five red stars are awarded to wineries that have upheld 5-star ratings this year and in the previous three years (hence four years in total). Here I have used the star ratings that James published in 2019, 2020 and 2021 along with my 2022 rating. Winery names printed in red again denote the best of the best with a long track record of excellence. And to formalise this criterion, I have introduced the rule that they must have upheld a 5-star rating this year and in the previous nine years (10 years in total).

The result of these changes has naturally been a slight shuffling in the ranks of our top-rated wineries to single out those that have set the very highest standards in the most recent seasons. The inherent challenges in the 2020 harvest (dubbed by James as ‘freakishly terrible’) across many regions of south-eastern Australia, and, to a lesser extent, the ongoing impact of drought in 2018 and 2019, has resulted in an expected lowering of the averages. This is particularly stark in the context of the mercy rule and absence of demotions applied in the 2021 edition. 

The refreshed criteria for star ratings 

★★★★★ Outstanding winery regularly producing wines of exemplary quality and typicity. Will have at least two wines rated at 95 points or above, and has held a 5-star rating for the previous three years (i.e. 4 years in total at 5 stars). 

Where the winery name itself is printed in red, it is a winery with a long track record of excellence, having held a 5-star rating continuously for 10 years – truly the best of the best. (Formerly a winery generally acknowledged to have had a long track record of excellence in the context of its region.)

★★★★★ Outstanding winery capable of producing wines of very high quality and did so this year. Will have at least two wines rated at 95 points or above.
 
★★★★✩ Excellent winery able to produce wines of high to very high quality, knocking on the door of a 5-star rating. Will have one wine rated at 95 points or above, and two (or more) at 90 or above.
 
★★★★ Very good producer of wines with class and character. Will have either one wine rated at 95 points or above, or two (or more) at 90 or above. 

★★★✩ A solid, usually reliable maker of good, sometimes very good wines. Will have one wine at 90 points or above. 

★★★ A typically good winery, but often has a few lesser wines. Will have at least one wine at 86–89 points. 

NR The NR rating is given when there have been no wines scoring more than 86 points, or where the tastings have, for one reason or another, proved not to fairly reflect the reputation of a winery with a track record of success. The NR rating mainly appears on winecompanion.com.au.