Frequently Asked Questions

WhatNotToSing.com is the Internet's most complete public reference site on the popular U.S. television series American Idol. The heart of the website is the WNTS.com Performance Database containing information on every season, episode, contestant, performance, song, and artist in the show's run. We also feature news, analyses, essays, and other Idol-related features, mostly pertaining to the data we've collected. [Back to Top]

We've been collecting data weekly since Season Four in 2005. Ratings for Seasons Two and Three were calculated retroactively from surviving web sources and archives. WhatNotToSing.com went online in 2007, and the full database was opened to the public in early 2008. [Back to Top]

Quite honestly, we started Project WNTS because we grew tired of watching one promising contestant after another kill their once-in-a-lifetime opportunity by making inferior, foolish, or downright suicidal song choices. Many Idols are kids or not far removed, and some have no real familiarity with the entertainment industry. Choosing the right material and arranging and presenting it to one's best advantage are skills that music professionals take years to hone.

We could have just set up a website to publish our opinions, but what good would that do? (Most of you have heard the old saying about opinions, which isn't suitable for repeating on a family site.) Instead, we elected to trade in hard facts by applying our mathematical and musical backgrounds to the effort. By studying what has and hasn't worked in the past, we hope to enable future contestants to make better, more educated choices.

For the two young ladies who in particular motivated us to get off our butts and start this project, see the dedication on our About Us page. [Back to Top]

WhatNotToSing.com was founded by three longtime American Idol fans with long and varied backgrounds in mathematics, music, and web design. Two are old enough to be the fathers of most of the contestants, but the third is a college student and our resident singing expert; she swears that one of these years she'll audition for the show. All of us make our homes in the Philadelphia area. [Back to Top]

It's complete in the sense that every season, episode, contestant, performance, song, and original artist is in there. We even keep it up-to-date in real time during every episode whenever possible, so that visitors can quickly see who's performed particular songs before. However, it'll never be totally complete, because we're always coming up with new pieces of information to throw in there. [Back to Top]

No. There's quite a bit of data that we can't make available just yet. Sometimes it's because the information is incomplete or not up to our standards, other times because we haven't gotten around to building the necessary search pages. If there's a particular report that you need, and you think we might be able to generate the information for you, let us know. [Back to Top]

A primary artist is one who is most widely associated with a song by the general public. If we asked you who sang "Stairway To Heaven", we're going to wager that your first answer isn't going to be Dolly Parton (though she did cover it, along with many others.)

A secondary artist is one who recorded a notable but perhaps not as widely known version of the song. Generally speaking, it has to meet one of these criteria:

1. The artist had a Billboard Top 10 (or so) hit with it, but they're clearly not the first artist most people think of when they hear the song. (Examples: Cyndi Lauper for What's Going On; Aretha Franklin for I Say A Little Prayer.)
2. The artist is noteworthy and their unusual arrangement was the basis of an Idol performance. (Examples: Live for I Walk The Line; Chris Cornell for Billie Jean.)
3. The artist is noteworthy and they were the original songwriter, performed a well-known cover, or fall into that broad "because it seems right" category. (Examples: Carole King for You've Got A Friend; Van Halen for You Really Got Me.)

Songs may have multiple primary and/or secondary artists. A great example is Because The Night: We've credited both Patti Smith Group and 10,000 Maniacs as its primary artists since both had popular hits with it, and we list Bruce Springsteen as a secondary artist because he co-wrote it and frequently plays it live. [Back to Top]

Because Idol's cover of Hollywood Week is too spotty and inconsistent. Many performances never air because of time considerations, lack of copyright clearance, or the producers' general capriciousness, and the ones that do air usually last no more than 15 seconds. We list (but do not rate or cross-reference) audition songs for each contestant where we can, and we'll consider doing the same for Hollywood songs someday. If anyone cares to compile and donate a comprehensive list, that would definitely get the ball rolling. [Back to Top]

Please do! Our goal is to keep the database as accurate as possible. Just send us an email. [Back to Top]

Every performance in the WhatNotToSing.com database is assigned an approval rating, which represents how much (or not so much) the Internet fans of the show liked the performance. Rating values, which are usually displayed in boldface on the site, range from 0 to 100, with 50 being average.

Seasons, episodes, contestants, songs, and artists also have approval ratings, which are calculated from the average of the relevant performances. [Back to Top]

Ratings are calculated from reviews of each episode posted on publicly-accessible websites, newsfeeds, blogs, forums, message boards, chat rooms, and other online sources. We sample dozens of sites each week and tally opinions statistically. [Back to Top]

It's easier to list the sources we don't use. We don't count the judges' opinions nor any posted on the official American Idol website (so as to remain fully independent of sources controlled by the show.) We don't consider fan sites that advocate for or against a particular contestant, nor any site that has an obvious bias or agenda. We visit only free websites or ones with open signup policies; any site that requires us to pay money to access, we skip. Finally, like any good pollsters, we don't count the opinions of the WhatNotToSing.com staff, nor our families' or friends'.

Other than the above, pretty much anything goes. [Back to Top]

A few are focused solely on AI, others on Reality TV in general. But most sites we've used have nothing to do with the show at all – they're online newspapers, community message boards, hobby sites, tech sites, blogs, and wherever else we find folks discussing the latest episode of American Idol.

We've pulled opinions from at least 200 different sites so far, and we try to visit three or four new ones each week. [Back to Top]

No. We choose a different sampling every week. However, we do return most often to the websites with the largest American Idol communities. [Back to Top]

Never fewer than 200, and usually 300 or more. [Back to Top]

Yes. We only consider opinions timestamped between the performance and the start of the results show. We've found that after the results are known, particuarly if there was a 'shocking' elimination that week, reviewers' opinions of a performance often change drastically. People switch from judging the contestants to judging America's voting habits.

After the Finale, and before we freeze the ratings for good, we perform one last fine-tuning based upon the Idolsphere's "Best Of The Season" and "Worst Of The Season" wrapups. Only a handful of performances are affected, and rarely by more than two points upwards or downwards (so as to remain within the margin of error of the '24-hour' ratings.) The season is then considered closed, and no further adjustments are made. [Back to Top]

In part, but many other factors go into an approval rating as well: song choice, presentation, the judges' remarks, the contestant's behavior and composure, personality, past performances, and how well other contestants performed that night. Singing quality is the single most significant component, but it's far from the only one. We measure how well America liked the whole package. [Back to Top]

"No, no, a thousand times no!" American Idol voting patterns are far too complex to boil down to a single number. What matters instead is how well a contestant motivates viewers to vote for him or her. Strange as it may seem (unless you've watched the show for years), how much they liked or disliked a performance on a particular night isn't always the primary factor in their decision. Therefore, never mistake approval ratings for a voting prediction system. [Back to Top]

That's not a question. [Back to Top]

Because that's how Internet fans rated them the night of the show. If performances grow or shrink in stature over time, whoopee. We can't measure that, we don't try to measure that, and until the day comes when American Idol allows us fans to vote on past seasons retroactively, we couldn't care less about measuring that.

If it makes you feel any better, there are a few performances in our database that we happen to think are overrated or underrated, too. That's to be expected in any consensus-based poll. We believe we have a solid and accurate methodology, and we apply it uniformly week in and week out. In the specific cases of Crocodile Rock and Build Me Up Buttercup, which are the two performances we're asked about the most, you can click on their respective links to read our analysis of why we think they scored as they did. But no, we're not going to change their ratings years after the fact to bring them in line with some perceived conventional wisdom (which is a moving target anyway.) [Back to Top]

When it comes to entertainment ratings, most folks are more familiar and comfortable with a scale of 1 to 5 stars. So, just to be nice, we often express ratings in terms of stars, by dividing our 0 to 100 scale into five 20-point intervals:

	Stars	Numeric Rating
	1 star	0 to 19
	2 star	20 to 39
	3 star	40 to 59
	4 star	60 to 79
	5 star	80 to 100

It's a statistical measure (abbreviated σ) of how much variation there is in a data set. Put simply, the higher the value, the more the reviewers disagreed. A love-it-or-hate-it performance, such as Blake Lewis's beat-boxed You Give Love A Bad Name, will have a high standard deviation, while one that was universally liked or disliked, e.g. Bo Bice's a cappella In A Dream, will have a low σ.

On WhatNotToSing.com, the average standard deviation of a performance is about 18. [Back to Top]

Because when we were originally preparing the past seasons' ratings, we didn't locate enough surviving sources on the Internet to meet our standards. This was particularly true for the semfinals and wildcard episodes, which aired before American Idol's TV ratings took off. To supplement our data, we established the Season One Review Crew, a panel of impartial AI critics and fans to help us review the performances from that long-ago season. Thanks to their invaluable input, we were able to finalize the AI1 ratings and publish them in May of 2008. [Back to Top]

It refers to the online community of American Idol fans: bloggers, forum posters, web columnists, roundtable panelists, etc. The word is probably derived from "blogosphere". We didn't come up with it, and we don't know who did, but it's a useful term. If you'd like to claim credit, drop us a line. [Back to Top]

Nonexistent. The Internet doesn't need another public Idol board; it has plenty of good ones already. Besides, our goal is to remain independent of both the show and its community of online reviewers. The last thing we need is to offer forums of our own, thus discouraging fans from posting reviews on other websites. [Back to Top]

Again, there are enough good (and not-so-good) American Idol-related websites out there that we feel no particular need to become yet another media clearinghouse, particularly as all images and audio from the show are copyrighted. Like everyone else over the age of 5, we know where to find Idol media clips on the Web when we need them. We've included a few photos from the show here and there that we think qualify under 'fair use' guidelines. [Back to Top]

Throughout the site, links into our database pages are tagged with icons so that you know what section of the database you're entering:

This helps avoid confusion, particularly since Performances and Songs usually share the same name, and because a few former Contestants now have their own Artist pages too. [Back to Top]

You're using an old browser that doesn't fully support the modern PNG image format, and we'll lay dollars to donuts that it's Internet Explorer 6. It takes a tremendous amount of additional work for web developers to support the long-obsolete IE6 fully. The WhatNotToSing.com design staff has compromised on this matter - we've wasted invested a lot of time ensuring that the site is fully functional in IE6, but little to no time trying to make it appear attractive in it. On behalf of millions of web designers worldwide, may we ask you to upgrade to IE7 or switch to Firefox, Opera, or Safari? Pretty please? [Back to Top]

Because we're a research and analysis site, not a fansite, so we try to keep things formal. It's easier for everyone to stay objective if we don't sound too chummy with the Idols, none of whom we've ever met personally anyway. (But yes, when discussing contestants among ourselves, we say "Carrie and Bo" like everyone else, not "Underwood and Bice". We're not that prissy.) [Back to Top]

If you'd like to volunteer your talents on WhatNotToSing.com, be it as an author, editor, music historian, analyst, graphic designer, or any other role, just drop us a line. We'd love to have you join the team. [Back to Top]