The Practical Problems with a Catagory Rating System:
Let's say we use a system a typical 1 through 5 with five being the best possible, and with catagories to include:
- Image Quality
- Ease of Use
- Features
- Warranty
- Price/Performance or Value
And of course there are other catagories that could be considered.
We looked at other review sites, in both the projector industry and elsewhere, and quickly came to realize the limitations of using such a system. I think it's important to explain why, and I believe that those of you who have visited other review sites will have to agree, to some degree.
First: Keep in mind that reviewers need products to review, and, like it or not, they come from the manufacturers in most cases. And therefore:
Second: If, you use a 1-5 scale, and you give some manufacturer a 2.5 or 3 overall rating (don't even think about a 1 or 2 rating), it's probably safe to assume that, that manufacturer isn't going to be sending out any additional products for review. And we reviewers do need something to review.
Third: Using rating scales like these, are way too general to separate out similar products. Let's say that the Optoma XYZ has slightly better black levels than the InFocus ABC, but the InFocus has better "out of the box" color, which gets the higher mark? (And remember - to keep manufacturers happy, every catagory better have at least a 3+...
Fourth, and getting to the heart of it, do we use a fixed scale, not accounting for price? If so, assume that $15,000 3 chip DLP's are the best and rate 4.5 or 5.0. If a $10,000 single chip projector comes very close to those above, does it rate a 3.5 or 4.0, or does it rate a 5.0 for being very close to the best, but being more affordable.
Certainly $2000 projectors are not going to be close to $15,000 projectors in performance
Of course we could fix the scale within price ranges, That is between $1500 and $2500, the best image gets a 5.0, next closest projectors get a 4.5, etc.
The problem there is that prices are falling all the time, and products are being replaced. A projector like the Panasonic PT-AE900u, which last fall, sold for right around $2100, now - 6 months later - sells (if you consider full value to their rebates - $400 cash, and $300 blockbuster rentals), you would say that the street price is more like $1400-$1500 today.
The job of re-ranking all the projectors every 60 - 90 days is just too daunting. If we don't, then the scores no longer reflect the overall value.
And perhaps most important, having the best image quality (to keep to the same example), doesn't do the reader any good, if that projector simply won't position correctly in their room.
And that brings us, again, to what a Hot Product Award means.
A projector receiving a Hot Product Award, indicates that the projector in question is not only of very good quality overall (not that there's much junk out there anyway). And most importantly, that we would consider it the best projector for at least a significant number of people with a budget to match.
The best way to clarify that is to provide a couple of examples.
In the under $2000 price range:
The Optoma HD72 earned the reward. It's bright, has a great picture (but needs an easy to perform, basic end user calibration for best enjoyment), its easy to use, has an average warranty, and is very competitively priced. So it deserves the award. The HD72 has a zoom lens with a limited range, and no lens shift, however, so it won't work for everyone.
The Panasonic PT-AE900u, although not as bright, is a hair less sharp, has a shorter warranty, but has excellent out of the box color, and currently is about the least expensive in that class. Equally important, it has a 2:1 zoom lens and lens shift with a wide range, so it will work in just about anyone's setup, whereas some will not be able to properly place the Optoma in their room.
So, the Pansonic, too, earns a Hot Product Award, appealing to people needing its setup flexibility, and those who just don't want to be bothered with even a simple calibration (most buyers).
And I'll throw in one more into the mix. The Sanyo PLV-Z4. It too has a rebate (free lamp) at the time of this writing, so it nets out price wise between the other two. But it has even more placement flexibility than the Panasonic (in the lens shift area), and is perhaps the sharpest (at least a touch sharper than the Panasonic) image. And it has a 3 year warranty, a real plus.
The Sanyo,, though, is not as bright, and unfortunately, it's out of box color leaves much to be desired. . So the Sanyo appeals to hobbiests who aren't using the largest screens, and those who like the piece of mind of a superior warranty.
Each of these has its own loyal following, each, overall, performs extremely well, but each has a group of potential buyers that it serves better, than any of the other projectors.
Another example:
Sony's HS-51A, perhaps the best of the LCD projectors out there: It still sells for well over $2000, but was close to $3000 only a couple of months ago. It did not get the award, despite being overall better than the significantly less expensive Sanyo and Panasonic (exceptionally sharp), because of a competing projector, the Optoma H78DC3 (just discontinued). The Optoma's typical price (after the usual rebates, etc.), was about $300 - $400 more than the Sony, and I felt the Optoma was easily worth the difference. Like the Sony, the Optoma had lens shift for better plaement flexibility, etc. On top of that it was notably brighter (the Sony is one of the less bright units out there).
Things change, however. The H78DC3 is formally discontinued, and most dealer supplies are gone as of this writing. The replacement HD7100 looks to be a improved projector. Not drastically so, but better.
However the HD7100 is selling in the under $3500 range, and the Sony, in the two and a half months since I published the review, seems to be now selling for significantly less, putting the typical spread between the HD7100 and the Sony, at more like $800.
As a result, the HD7100 is not in the same general pricing range, for there are many buyers out there who could stretch their budget to get the Sony, but not be close to the new Optoma price. And the Sony, is, as mentioned, is a cut above the other mentioned LCD based projectors, and offers similar placement flexibility. The Optoma HD72 DLP, is still less money, and far brighter, but won't work for a lot of people.
Bottom line, I will be shortly be publishing an update to the Sony, which now bestows upon it, a Hot Product Award, as it is no longer a direct competitor to the new Optoma.
I should note, as far as the examples above, I didn't mention that a very small slice of potential buyers, can't buy a DLP projector because they are susceptable to the "rainbow effect". And it can get more complicated. The Optoma HD72 being DLP, has a 4X wheel, not as fast as many of its competitors with 5X wheels, so there is even a slice of potential DLP buyers who will need a DLP other than the Optoma because of the wheel speed difference (we're talking about a percentage point or two of the entire buying population).
So, I've now probably explained this to death, but want to finish by say again, that the award goes to projectors that will work best for a least a small but signficant portion of the buyers out there.
Over time, a projector, due to pricing changes, or competitive changes, may earn the award even if it didn't on first pass. No, I don't go taking awards away, its just not done, and remember few of these projectors stay on the market more than a year.
Thanks for listening!
We attempt to provide comprehensive reviews, and often comment about competing projectors within reviews, to help you better decide. In addition in price ranges where we have reviewed several projectors, we often provide comparison articles to further assist you.
Lastly, our review summaries have the Pros, Con's and Typical sections which should really help you figure out which is best - far more so than a catagory rating system limited to 3.5 to 5.0 ratings that cover everything from $999 to $19,999..
Good hunting! -art
