PR Week's Agency Excellence Survey

Let me start by saying that any piece with such a grand title sets a high bar for itself. You can guess by that first sentence that I don't feel the piece reached it. Far from it, this piece seems like a great way of handing out medals to the establishment and does little to see where the real excellence in our industry lies. The feature is a good idea in theory but the results should tell you that there's a flaw in the process. The tables show that the larger agencies (which have the most clients) win. It doesn't take a genius to figure out that this is the most likely statistical outcome; especially if you determine that you are only going to include those with a large enough sample size. Indeed the article even points out that the sample size is consistent with the agency size. Put another way, this research tells us that the big agencies have lots of clients and the small ones don't. It doesn't really tell us much more, other than they have reasonably happy clients. Indeed all it really says is which of the large agencies has the happiest clients. So if I dislike this article's approach what would I propose instead? I think the first thing I'd do is to work with an organization like the Council of PR and agree on what constitutes excellence. I'd then publish this and get debate going around that definition BEFORE attempts are made to measure people against it. Once there is a broadly agreed standard, I'd then conduct the survey BUT I'd do it in a way where samples were limited so that each agency was given say 20 clients that worked with them (this is where a real researcher will tell you what sample size is needed to draw meaningful conclusions). If you give the larger agencies as many clients as they have they will frankly always come out on top - the law of averages just says so.

I guess I applaud PR Week for taking on this topic but in the next twelve months (assuming they run the same feature next year) we need to come up with a better approach than was taken this year. That's a challenge for the industry not just PR Week.