Third Party Lenses Magazine Reviews Listing

Third Party Lenses Magazine Reviews Listing
by Robert Monaghan

Related Local Links:
Third Party Lenses
Listing of Third Party Lenses by Mfger
Listing of Third Party Lenses by Focal Length and Speed
Third Party Lenses Related Postings
Lens Testing Chart and How-to-Use Info

Related Links:
Lens Testers Anonymous (Stephen Gandy)
Lens Tests and Reviews
Camera Review Site (35, MF, LF)
Ruether's Nikon Lens Page
Phil Greenspun's PHOTONET Reviews
Photodo Magazine Lens Tests
Lens Performance Survey (Sigma, Tamron, Tokina)
Sigma Lens Tests (from Pop. Photography)
Easy Guide to Lenses Ratings
Buying Used OM (Olympus)
Tutti Fotographie Test Results
Third Pary Lens Incompatibility Pages (and compatibility pages)
Bjorn Rorslett (Nikon) Lens Review Pages
Olympus Lens and Third Party Lenses Tests [8/18/99]
Pentax Lens Resolution...
Comparing sharpness and photodo ratings
Classic Pentax Lenses Tested (Peter Spiro) [10/4/99]
Buy Nikon Guide and AF Compatibility Chart
Manufacture and Performance of Photographic Lenses by Heinz Richter
Photobluebook Normal 35mm SLR Lens Reviews [3/2001]
Misc. Lens Test Results
Brit. Jrnl Photogr. Third Party Lens Comparison Review
Subjective Lens Evaluations (Niklas Nikitin) with variations info [8/2002]

Local Reviews
TX lenses

Heliar 15mm Lens Tests by 3 Magazines - [c=center, e=edge, FV, PopPhoto, Am.Photo]

What's wrong with the above chart? Everything! In theory, the Heliar 15mm lens should get the same resolution ratings regardless of who tests it. But guess what? Not only did the three magazine tests get different results, they got widely or wildly differing results! Yeow!
So keep this chart in mind when you look at different reviews. Even resolution tests have subjective factors, requiring judgment calls on which chart section is in focus, and which is just barely too fuzzy to qualify. Worse yet, different charts or films may be used, so you have systematic errors. A lens getting very high ratings from one magazine's tests (e.g. the Heliar 15mm in Amateur Photogr.) may be related more to their testing charts and techniques. The same lens might get lower resolution ratings when another test chart or setup is used (as with Popular Photography results reported above).
Still unconvinced about the need to use lens reviews with care, and do your own testing? Consider this quote from a noted former lens tester for a major USA magazine (Modern/Popular Photography):
One more important point. When a magazine requests a lens for testing, the maker carefully selects a superior example and sets it aside for magazine tsets. When I was testing for Modern I could immediately spot one of those especially selected lenses. I would then insist on buying it. The maker wouldn't admit it was a special lens since he had told me it was a random sample. I got a lot of outstanding lenses that way. (grin) ...Arthur Kramer
What do you think are the chances you will get a random lens out of the box that performs as well as the best lens the manufacturer could find and tweak for magazine lens testing? Slim,huh? ;-)
In short, use magazine and other reviews and resolution tests as a means to identify possible lenses for testing. But use your own tests and criteria to select which lenses you ultimately end up buying.
[Ed. note: thanks to Frank Filippone for providing these lens review data...]

Caution: Before you get too excited about any particular lens based on its magazine reviews, read about variations in three lenses from the same batch as reported in a Modern Photography review some years ago...

From Joseph's List of Pop. Photography Lens Reviews

LENS TESTED DATE TESTED Adorama M-series 21mm f/2.8* 4/96 Adorama M-series 28mm f/3.5* 4/96 Angenieux 28-70mm f/2.6 4/91 Phoenix 28-105mm f/2.8-3.8 8/96 Quantaray 75-300mm f/4-5.6 1/94 Samyang 18-28mm f/4-4.5* 2/92 Sigma 14mm f/3.5 1/93 Sigma 28mm f/1.8 3/93 Sigma 300mm f/4 11/95 Sigma 400mm f/5.6 APO 5/92 Sigma 18-35mm f/3.5-5.6 D 2/95 Sigma 21-35mm f/3.5-4.2 2/92 Sigma 24-50mm f/4-5.6 8/92 Sigma 24-70mm f/3.5-5.6 D 5/95 Sigma 28-70mm f/3.5-4.5 11/90 Sigma 28-70mm f/2.8 6/93 Sigma 28-70mm f/2.8-4 6/96 Sigma 28-105mm f/4-5.6 9/94 Sigma 28-200mm f/4-5.6 4/91 Sigma 28-200mm f/3.8-5.6 2/95 Sigma 35-80mm f/4-5.6 DL 4/94 Sigma 70-210mm f/4-5.6 11/90 Sigma 70-210mm f/2.8 APO 5/93 Sigma 70-300mm f/4-5.6 6/96 Sigma 75-300mm f/4.5-5.6 APO 4/92 Sigma 75-300mm f/4.5-5.6 4/92 Sigma 75-300mm f/4-5.6 APO 8/93 Tamron 300mm f/2.8 LD 4/93 Tamron 20-40mm f/2.7-3.5 2/95 Tamron 24-70mm f/3.3-5.6 3/94 Tamron 28-70mm f/3.5-4.5 11/90 Tamron 28-70mm f/3.5-4.5 10/92 Tamron 28-80mm f/3.5-5.6 5/96 Tamron 28-200mm f/3.8-5.6 4/93 Tamron 35-90mm f/4-5.6 4/92 Tamron 35-105mm f/2.8 9/92 Tamron 35-135mm f/3.5-4.5 4/91 Tamron 70-210mm f/4-5.6 11/90 Tamron 70-210mm f/2.8 LD 5/93 Tamron 70-300mm f/4-5.6 2/94 Tamron 80-210mm f/4.5-5.6 5/96 Tamron 90-300mm f/4.5-5.6 4/92 Tamron 200-400mm f/5.6 8/95 Tokina AT-X 17mm f/3.5 1/94 Tokina AT-X 300mm f/2.8 4/92 Tokina 300mm f/4 11/95 Tokina 400mm f/5.6 5/95 Tokina 20-35mm f/3.5-4.5 5/92 Tokina AT-X 24-40mm f/2.8 2/92 Tokina 28-70mm f/3.5-4.5 11/90 Tokina AT-X 28-70mm f/2.8 9/92 Tokina 28-70mm f/2.6-2.8 2/95 Tokina 70-210mm f/4-5.6 11/90 Tokina AT-X 80-200mm f/2.8 4/91 Tokina AT-X 80-200mm f/2.8 4/96 Tokina 100-300mm f/4 8/93 Vivitar Series 1 19-35mm f/3.5-4.5 7/96 Vivitar Series 1 24-70mm f/3.8-4.8* 2/92 Vivitar Series 1 28-300mm f/4-6.3 7/96 Vivitar Series 1 100-500mm f/5.6-8* 4/91 * Manual-Focus lens

Psst: Know of any reviews of third party lenses you don't see here?
Email Us! the info to post! Thanks!

rec.photo.equipment.35mm
From: for7@aol.com (FOR7)
[1] Re: Why the huge price difference?
Date: Wed Nov 04 23:21:13 CST 1998

>Take the reviews with a "grain of salt." Have you ever read a truly negative >review?

Yeah the British Magazines usually give reviews that clearly state one lens is better than the other. Popular Photography may be careful to word their reviews but the numbers they come up with their sysytem have proven quite accurate for many.
for7@aol.com

rec.photo.equipment.35mm
From: "Marc P." marcp@ireseaufox.com
[1] Lens test sites?
Date: Tue Mar 16 11:01:12 CST 1999
I had only one reply to my original post with the following sites:
David Reuther - Grover Larkin

> http://utopia.knoware.nl/users/leover/nikon/nikkor.htm > > Michael Liu > http://www.mir.com.my/rb/photography/hardwares/speciallenses/index.htm > > Photodo > http://www.photodo.com > > Photozone > http://www.cmpsolv.com/photozone > > Walter Pietsch > http://wpxx02.toxi.uni-wuerzburg.de/~winstel/walter/nikon.html

I knew of the Photodo and Photozone sites. Two of the other three feature Nikkor lenses which I can't afford and the third deals with "special" lenses.
If you know of any good and extensive lens evaluating sites (current "regular" lenses from all companies), I would greatly appreciate any info on this. I've searching but have found nothing interesting so far.
Thanks.
Marc P.

From: bob@bobshell.com
Newsgroups: rec.photo.equipment.35mm
Subject: Re: Heres why I use Contax and not Nikon!
Date: Fri, 29 Jan 1999
remove the dot and y wrote:

>bob@bobshell.com says... > > >Magazine tests are another matter. You can design your tests to favor > >whatever you want to favor. If the majority of your ad dollars come from > >Japanese companies, then you can design your tests to favor Japanese lenses, > >and your results will show that the latest lens from Niknolta is head and > >shoulders bette than the best Leica lens. > > I thought you said advertising dollars had no effect on editorial > content Bob. Remember the ongoing thread about fraudulent mail > order firms ? Make up your mind now :)

Since Shutterbug does not do lab tests it should be obvious that I am not talking about Shutterbug in my comments. Do I have to spell it out for you??

> >Shutterbug does not do lab tests. We tried it some years back in our sister > >magazine PhotoPro but stopped because people didn't want to hear objective > >test results. > > Probably because they didn't get them.

The lab tests for PhotoPro were done by Optikos Corp. in Cambridge, a totally independent optical company who do the lens and other optical design work for Polaroid. They provided totally objective tests on lenses submitted by the US distributors. Some well known lenses tested out as really super, as expected, while some others were clearly awful. We got indifferent reader support for our work and outright hostility from the industry, so we just gave up. But we did try.
Bob

rec.photo.equipment.35mm
From: canon50e@aol.com (Canon50E)
[1] Third party equipment incompatibilities?
Date: Fri Mar 19 22:47:09 CST 1999
Well, a Singapore professional photographer has asked me a few days ago if I had experienced any incompatibilities of my EOS 50E (Elan IIE) with attached my past-Tokina lens and my Tamron lens. And I said, "no, I haven't." Though he's been telling me that with his F100, F5, EOS 3, and EOS 5 he has been having incompatibilities with third party lenses that sometimes it overexposes or sometimes it just basically doesn't work on that camera.
Therefore he has inspired me to write a survey/page that lets users to report situations where you feel that a certain lens and camera body just doesn't work out together. Usually the incompatibilities people encounter are from newer camera bodies with older third party lenses/flash.
For further info. please take a look at this page here:
http://www.kyphoto.com/thirdparty/
There will not be any kind of results given with these inputs, but I honestly hope that people will share their experiences with these problems and let other people be aware. I am also open up to opinions, like if you are interested or not interested...I know I can't satisfy all people but tell me if it's of some sort of interest to anyone.
Thank you for your attention.
Sincerely yours,
KY Mak
----------------------------------------
My Photography Page:
http://www.asiaphoto.com/ky

From: Hans Martens hans.martens@wildpicture.com
Newsgroups: rec.photo.misc
Subject: Telephoto lens tests
Date: Wed, 05 May 1999
Now online on the photography section of my website are test results of telephoto lenses. Tested are lenses of 300mm, 400mm, 500mm, 600mm and even a bit longer.
Have a look.
Greetings,
Hans
==================================
Hans Martens
Wildlife Photographer, Field Guide
Wildlife & Nature Photography
http://www.wildpicture.com
==================================
see tele lens tests

From: "Joseph Vogt" jvogt@ct1.nai.net
Newsgroups: rec.photo.marketplace
Subject: Re: NIKON LENS TESTS?
Date: Wed, 5 May 1999
Check the "Readers Guide to Periodical Literature" at your local library. It's in book form, by year. Look under "Lenses - Photographic" and it will give listings my month and year for all the periodicals with lens tests. (I just happened to do it today).
DAVE audio2000@my-dejanews.com wrote

> IS ANYONE AWARE OF AN INDEX TO NIKON LENS TESTS IN POPLULAR PHOTOGRAPHY, > PETERSONS PHOTOGRAPHIC ETC.? > THANK YOU!!! > -- > DAVE

From: Bjorn Rorslett nikon@foto.no
Newsgroups: rec.photo.equipment.35mm
Subject: New lens evaluation resource opened
Date: Fri, 30 Jul 1999
hi everybody I have recently added a comprehensive Lens Survey and Evaluation section to my web site at http://www.foto.no/nikon/index.html . If you pass the entrance test, select 'Lens Evaluation' from the menu bar. Here you'll find in-depth comments on a large number of lenses, mostly Nikkors. All lenses have been used by myself for actual shooting and my comments reflect this insight. You'll find. for example, precise statements regarding the best performing apertures for each and every lens. A number of fairly exotic lenses are presented, amongst these is the 300/2 Nikkor.
The resource is in its beginning stage so some lenses have only received their ratings and the corresponding comments will be added shortly.
Any criticsm or comments are welcome.
regards
Bjorn Rorslett
Visit http://www.foto.no/nikon for UV Colour Photography and other Adventures in Nature Photography

rec.photo.equipment.35mm
From: mbergma2@popd.ix.netcom.com
[1] Re: Which is the sharpest lens according to photodo.com
Date: Sun Aug 22 00:05:53 CDT 1999
Photodo does have a search engine that allows one to compare ratings. Choose Lenses - 35mm - use the Advanced Search - pick 50mm as the min and max - when the results appear scroll down to the bottom on the left scroll bar and choose show as table.
In the 50mm the highest ratings are for the Contax 50mm f/1.7 and the Leica 50mm f/2.0 at 4.6.
For 28mm it is the Contax Distagon T 28mm f/2.8 at 4.3.
For 135mm it is the Canon 135mm f/2.0 at 4.5.
I believe the Canon 200mm f1.8 is the highest rated lens on photodo. It tested at 4.8.
Marc
Dr. A.Routh MD. wrote:

> Which lenses are the sharpest lens in the prime lens category among > different manufacturers- [a] 50 mm lens [b]28 mm [c] 135 lens according > to Photodo.com? Is there a tabulated list where the lens sharpness can > be compared at a glance? With thanks.

[Ed. note: possible cure for those lusting after Zeiss glass? ;-)]
From Nikon Digest:
Date: Wed, 25 Aug 1999
From: Don Lintner lintner@uwp.edu
Subject: Re: [NIKON] Autofocus Accuracy
Last January, I switched from Contax back to Nikon. I miss some of the Zeiss lenses a little but love the F100 bodies. Their inrterface is great and, while I put down autofocus for years, I now think its great. I can get shots I couldn't without it and while it may not be as accurate as perfect manual focus, I have many fewer out of focus slides due to dark conditions and my over 40 eyesight. I also think even a slow focusing autofocus lens is still faster than I can do manually with any chance of accuracy.
As an aside, Nikkor lenses are generally indistiguishable from Contax lenses in most situations in terms of sharpness and contrast, etc. (I haven't done exhaustive testing, this is just what I see on my slides). The big difference is the Zeiss glass had a sublty nicer and more consistant color from lens to lens but this is very subtle and subjective - - my wife doesn't see it at all.

[Ed. note: while a photodo score is of limited value as an average and only focused on MTF, not giving info about many issues, here is a review..]
From: Roland roland.rashleigh-berry@virgin.net
Newsgroups: rec.photo.equipment.35mm
Subject: The great "Lens Resolution" Competition
Date: Fri, 20 Aug 1999
Welcome to the start of the great "Lens Resolution Ratings" competition where we have 4 of our players waiting to slug it out. Ratings are as given on the photodo site ( http://www.photodo.com ) and the judges award up to a maximum of 5 points depending on finishing position and performance. Competitors may skip an even at any point by playing a joker card. There are two legs to this competition, the "fixed" leg and the "zooms" leg. The winner will be the one awarded the most points by the judges.
Yes, they're lining up, they're under starters orders -- they're off !!!

FIXED ----- 20mm (pentax=3.7, nikon/minolta=3.5, canon=3.4) Canon 3 Minolta 4 Nikon 4 Pentax 5 24mm (canon=3.9, nikon/pentax=3.7, minolta=3.4) Canon 3 + 5 = 8 Minolta 4 + 2 = 6 Nikon 4 + 4 = 8 Pentax 5 + 4 = 9 28mm (minolta=4.1, canon/pentax=3.8, nikon=3.2) Canon 8 + 4 = 12 Minolta 6 + 5 = 11 Nikon 8 + 2 = 10 Pentax 9 + 4 = 13 35mm (pentax plays the joker and the other players fight it out to adjust positions. canon/minolta=4.0, nikon=3.9 - the judges declare a draw so scores unchanged) Canon 12 Minolta 11 Nikon 10 Pentax 13 50mm (pentax=4.6, canon/nikon/minolta=4.4) Canon 12 + 4 = 16 Minolta 11 + 4 = 15 Nikon 10 + 4 = 14 Pentax 13 + 5 = 18 100/105mm Macro (minolta=4.5, canon=4.4, pentax=4.3, nikon=3.9) Canon 16 + 4 = 20 Minolta 15 + 5 = 20 Nikon 14 + 1 = 15 Pentax 18 + 3 = 21 135mm (canon=4.5, nikon=4.3, minolta=3.6, pentax=3.5) Canon 20 + 5 = 25 Minolta 20 + 1 = 21 Nikon 15 + 4 = 19 Pentax 21 + 0 = 21

180/200mm F2.8 (Pentax plays another joker and the rest fight to adjust positions. canon/minolta=4.1, nikon=3.6 with canon also playing their 200mmF1.8 card). Canon up one point, minolta stays there and nikon moves down one point.

Canon 25 + 1 = 26 Minolta 21 + 0 = 21 Nikon 19 - 1 = 18 Pentax 21 + 0 = 21

The first leg is over. We have the "Zooms" leg to go but at this stage we have Canon as the clear leader with Minolta and Pentax in joint second place and Nikon not too far away in fourth.

ZOOMS ----- 20-35mm (nikon=3.6, canon=3.5, pentax=3.4, minolta=2.8) Canon 26 + 4 = 30 Minolta 21 + 0 = 21 Nikon 18 + 5 = 23 Pentax 21 + 3 = 24 28-70mm (canon=3.9, minolta/nikon=3.7, pentax=3.3) Canon 30 + 5 = 35 Minolta 21 + 4 = 25 Nikon 23 + 4 = 27 Pentax 24 + 1 = 25 28-105mm (canon=3.3, pentax/nikon=3.2, minolta=2.7) Canon 35 + 5 = 40 Minolta 25 + 0 = 25 Nikon 27 + 4 = 31 Pentax 25 + 4 = 29 35-70mm (nikon=3.8, minolta=3.5, pentax=3.1, canon=2.8) Canon 40 + 1 = 41 Minolta 25 + 4 = 29 Nikon 31 + 5 = 36 Pentax 29 + 3 = 32 35-80mm (canon=3.2, nikon=2.8, minolta=2.4, pentax=2.2) Canon 41 + 5 = 46 Minolta 29 + 1 = 30 Nikon 36 + 2 = 38 Pentax 32 + 0 = 32 70-210mm (canon=3.1, pentax=2.9, nikon=2.8, minolta=2.7) Canon 46 + 5 = 51 Minolta 30 + 1 = 31 Nikon 38 + 2 = 40 Pentax 32 + 3 = 35 80-200 F2.8 (canon=4.2, nikon=4.1, minolta=3.9, pentax=3.2) Canon 51 + 5 = 56 Minolta 31 + 3 = 34 Nikon 40 + 4 = 44 Pentax 35 + 0 = 35

100-300mm (nikon plays the joker card and the rest fight it out for position adjustments. canon=3.3, minolta=3.2, pentax=2.4. Judges put both Canon and Minolta up one position and Pentax down two positions).

Canon 56 + 1 = 57 Minolta 34 + 1 = 35 Nikon 44 + 0 = 44 Pentax 35 - 2 = 33

And that's the end of the "Lens Resolution Ratings" competition folks and the results are clear. Canon a full 13 points ahead of Nikon with Minolta 9 points behind with Pentax only 2 points behind failing to get a medal this time.

Canon = Gold medal on 57 points Nikon = Silver medal on 44 points Minolta = Bronze medal on 35 points Pentax = (also ran) on 33 points

From: rmonagha@smu.edu (Robert Monaghan)
Newsgroups: rec.photo.equipment.35mm
Subject: lens resol. doesn't matter cuz' Re: The great "Lens Resolution"
Date: 20 Aug 1999
first, a single point composite score on some selected MTF freq. analysis is not terrifically useful; that's why MTF charts are used, mainly to pick out the best performance points etc. - not to average out and pick lenses, since most of use our lenses at their best or sweet spots, right?
second, lenses vary a whole lot within and between batches, and esp. for used lenses, so the quoted numbers may be at least an entire unit better or worse than the one you have in your hand, so any arbitrary score is only valid for the lens they tested, not the one you have or buy (see my lens variation pages at http://www.smu.edu/~rmonagha/third/variations.html
third, lens resolution is essentially irrelevant if you are using color films, either slides or prints, for your work (as most of us do). The reason is that the lens is not the limiting factor, but rather the overall system resolution, and that in turn is determined by the film. In other words, my 300 lpmm aerial resolution nikkors can at best deliver maybe 120 lpmm on black and white thin emulsion specialty films, but when used with average contrast, real world subjects, my Ektachrome 100 film delivers a lousy 50 lpmm (1:1.6 avg contrast ratio). This is also why all those pricey lenses which do great on lens chart tests (at 1:1000 contrast ratios in black and white) can't be reliability told apart on real world shots - as the film is limiting, not the lens.
In other words, if you are using color film (and 96% of amateurs do), then lens resolution is good enough, whether you are using a Zeiss or a Nikon or decent third party lens, it isn't the limiting factor, the film is see http://www.smu.edu/~rmonagha/mf/lenslpm.html for more facts etc.
Finally, if you really want to do large blowups where lens resolution matters, it is much cheaper to switch to medium format. The larger film size will provide much more enlargement capability, and a $100 used TLR like a rolleicord can blow away the resolution and tonality of any consumer or professional 35mm lens system - size matters, it is that simple. see third/quality.html How Much Quality is Enough for related info...
regards bobm

Subject: Re: Which is the sharpest lens according to photodo.com
From: "Jim Williams" jlw@nospam.net
Newsgroups: rec.photo.equipment.35mm
Date: Sat, 21 Aug 1999

>Which lenses are the sharpest lens in the prime lens category among >different manufacturers- [a] 50 mm lens [b]28 mm [c] 135 lens according >to Photodo.com? Is there a tabulated list where the lens sharpness can >be compared at a glance? With thanks.

Warning, warning, warning: Photodo.com provides modulation transfer function (MTF) data. While *interpretation* of MTF data can tell you a lot about how a lens performs, MTF does ***NOT*** directly measure "sharpness." In other words, comparing MTF numbers (or any other single numbers) will ***NOT*** tell you which lens is "sharpest." Sharpness is a perceptual, qualitative criterion which incorporates such quantitative factors as resolution, macro-contrast, micro-contrast, and "bokeh" -- AND depends on conditions such as subject distance (lens performance can be optimized only for one specific distance, so one lens may be "sharper" at infinity while another is "sharper" at 10 feet), subject type, two-dimensional vs. three-dimensional subject, etc. It's been shown that it's impossible to quantify all the aspects of lens performance into any single "figure of merit" that would allow lens sharpness to be "compared at a glance."
I know this is not what you wanted to hear, but a lot of people waste energy and money seeking the "sharpest" lens without realizing that this is a very amorphous term, and any decision you make on how to determine which lens is sharpest will be partly an arbitrary personal judgement. That's why there are so many arguments about lenses on this group!

From: rlsaylor@ix.netcom.com (R. Saylor)
Newsgroups: rec.photo.equipment.35mm
Subject: Re: Which is the sharpest lens according to photodo.com
Date: Sat, 21 Aug 1999
....
Just a word of caution. The sharpest lens is not necessarily the best lens. A lens can be incredibly sharp but suffer from distortion, light falloff, and (yes) bad bokeh, among other things.
Richard S

From: LL lewislang@my-deja.com
Newsgroups: rec.photo.equipment.35mm
Subject: Re: Canon vs Minolta vs Nikon lenses
Date: Fri, 20 Aug 1999
Numbered lens tests are always a mixed bag at best. I prefer to see ³results²/blow up sections from a lens with my own eyes and mind/judgement. The main reasons that ³optical real world testing with your own eyes² are better than the best test labs is as follows. No manufacturing process is perfect - there will always be a certain degree of lack of consistency or optical quality variations from sample to sample. A magazine may test a plum while you get a dog. Even taking that into account the best judge of a lenses quality is your own eyes. Buy, rent, or borrow the lens you wish to use and do your own real world testing and come to your own real world conclusions based on your own real world film/shooting situtions/lighting conditions/etc. Everybody has their own optical "standards" of quality regardless of any numbers or letters a photo testing lab, no matter how accurate/objective might assign to a lens. I can¹t comment on ³photodo² but in my opinion Popular Photography rates its lenses much too high at a given magnification. Lenses they would give an ³A² to would barely muster a ³B² grade in my eyes. Then again I probably have a more critical eye than most of the people they use to do their testing as I like to stand 10² or closer to a 16x20² print (and formerly a 30x40² print in my more critical days past) to see how both the lens and the film have rendered fine detail. I am extra persnickety with my quality standards as I expect excellent quality at at least 16x20² for fine art prints as opposed to the ³5x7² or 8x10² (formerly 4x6² until they dropped that level of magnification as meaningless) at two feet away from the print mentality² of the more casual/less critical viewer. Also, Pop tests its lenses (on the bar chart at least), if I¹m not mistaken, at infinity.
Since I like to shoot people (only with a camera :-)) most of my shots are not macro/micro and not at infinity (I¹m not into shooting landscapes or U.F.O.¹s in the night sky). For me infinity begins in the three to five feet range (I like to get close to my subjects than many other photographers). Many months ago (March 98 issue?) Pop did a² re-testing² of a Tamron and Sigma 28-105mm zoom lenses. What this re-testing showed was that the original SQF test results (numbers/% in colored bars test) indicated that one lens did better than the other at infinity but when Pop re-tested at closer portrait distances (a few meters) the positions of the lenses (in regards to which lens was optically better/sharper than the other lens) were literally reversed. The ³poorer² SQF lens now (in my opinion) way out performing the ³better² SQF rated lens. Don¹t ask me which lens did better/worse as I don¹t remember - if you¹re curious go back and look it up for yourself. For these reasons and more I ³know² longer have ³faith in (lens test) numbers.² For me, with regards to lens tests, anyway, ³seeing is believing.² ³The truth is (not) out there² - its in your own subjective quality factor evaluation(s) (and reality). Take lens tests worth a grain of salt or silver halide. Come to your own optical conclusions. Use your own eyes and your own mind. Don¹t allow a magazine/etc. to think about or vicariously experience a lens¹s qualities for you. To paraphrase The Beatles ³think (and experience) for yourself.² Colored graphs may be pretty to look at and numbers may be interesting to compare but the real truth about a lens¹s quality(s) is beneath the numbers and behind the colored bars in real world experience. Intimately experience your own lens¹s qualities and come to your own real world optical "subjective quality" conclusions.
Happy shooting (and testing),
Lewis (A+ at 24x magnification ;-)) Lang

From: golem@shell.acmenet.net (David Rozen)
Newsgroups: rec.photo.equipment.35mm
Subject: Re: Lens field evaluation
Date: Fri, 01 Oct 1999
Maryanne Lindsay (MLindsay@mail.net) wrote:

: Does anyone have a favorite routine for lens evaluation? : purchase a lens or take one on trial how do shoot first : roll of film? : : Shoot all stops using high contrast subject (scene or gridlines)? : : Who has a good tried and tested routine?

Some real-world procedures, if you want white-coat lab fantasies, read no further.
I tend to ignore sharpness, as too much hassle to test truly accurately, but I do test for unsharpness. Of course I pay for this privelege -- i just don't buy any trashy lenses. Beyond that threshold don't worry about differences between them, unless through regular continued use a certain lens begins to show itself as special. Even then, chances are that the particular lens would not have revealed this extra quality in formal tests, due to too many variables and too little reality.
Forget high contrast targets, they will falsify the results if you're testing sharpness, but a true black target is useful for flare testing. "Solid" black won't do [doesn't exist], you need a black hole against which to observe results at all stops of a flare inducer used just outside the field of view as well as within it, at various distances from center. You can do this pretty well even without film but it's better on film. Other than flare, I check distortion, which needs to be on film since finders lie. Don't forget that the focus range can affect this. Do the near and middle range as well as infinity. Any obvious target will work, and you only need to chack the edge area. The only other concern I have is illumination, which can only be checked on film, best with underexposed chromes viewed by strong light. Choose your target carefully, and never trust the sky, in any weather, as an even target. I don't use any real subject but prefer multiple layers [space between] of diffusion material, similar to a multilayer softbox, right on the lens. The material doesn't need to be at the focus distance [better if not] but you should check a far mid and near range on the focus scale as this can also affect illumination. You only have to do the first few stops.
I did mention checking for unsharpness. I do look the first 2 stops to see if there is any obvious loss away from the central area, and especially if I've had to break my own rule and dabble in some cheap "f8 only" optic I'll look for any visible circular boundary at which sharpness takes a sudden dive, easily visible and visually disturbing. This I've seen only with zoom ultrawides. The effect if this flaw is present is to see a decently sharp circular image area outside of which things are visibly worse, and the circular boundary between is rather abrupt, though certainly not hard-edged. No special subject matter is required for this but a fairly strong, even texture shows it right away.
Regards, - dr

Date: Wed, 17 Nov 1999
From: Kar Yan Mak canon50e@netvigator.com
To: rmonagha@post.cis.smu.edu
Subject: Third Party Compatibilities & Incompatibilities Page
Dear Robert,
Hi, please be advised that I have slightly changed my third party page:
http://www.kyphoto.com/thirdparty/
Instead of just submitting the incompatibilities, I also offer people to submit the compatibilities amongst third party equipment. It might sound a bit repetitive and requires a little bit more work, but hopefully it'll be a good reference as well.
Thank you for your attention.
Sincerely yours,
Kar Yan Mak
http://www.asiaphoto.com/ky

Date: Thu, 30 Dec 1999
From: tssmith@best.com (Tim Smith)
Newsgroups: rec.photo.equipment.35mm
Subject: Re: Why are they afraid of printing the truth?
es323@my-deja.com wrote:

>Ever buy a camera that is praised to death by magazines, but when you >use it the camera breaks down consistently or is a piece of junk? I >have. To say the least, it is not a happy experience. Defective >cameras and lenses are out there, but we never read about these >terrible "lemons" in Popular Photograhy, Outdoor Photography, >Shutterbug, etc. We just eat the cost, while absorbing the economic >pain and suffer with their garbage. > >Why are most Photo magazines afraid to have "frequency of repair >records" just as the computer industry. At least they have the courage >to let people vote on the best and worst. People respect that. > > How about service? I have been horribly disappointed the >unprofessional, lousy repair service with one of big SLR giants. >Beleive me, if I knew a company had licensed mechanics trained in auto >repair working on cameras, then there is no way I would buy that >product. Would you? > > Information helps us all. Are the big magazines just protecting their >financial pockets at our expense? How come we never hear of a >photographic defective product, repair records, terrible service or >anything negative about a manufacturer? Do photographic magazines know >where their bread is buttered (not with us)? Why are these magazines >afraid of printing us the truth? It's unfair, stinks and hurts all of >us. > >Comments appreciated.

Edward, I think you do not understand how magazines work. You believe that you are the consumer, but in reality, you are the product.
You are the product that is delivered to the consumer (the advertisers). The more of you (as product) that can be delivered to the consumers via the medium (the producers: the magazine), the more the magazine profits.
This is not all bad. You, the product, benefit from lower prices, in most cases. It's only your self esteem that suffers.
(I value my precious self-esteem, which is why I try to preserve what little is left of it by not buying the stuff that the magazines are trying to sell.)

From Nikon MF Mailing List:
Date: Sun, 05 Mar 2000
From: Jonathan Castner jonathan@jonathancastner.com
Subject: Re: MTF and lens tests
The Photodo MTF charts are particularly biased because they are based on the lens focused at infinity. Some lenses are fine at infinity but are outstanding at middle distances. Macro lenses are usually better at close and middle distances than infinity, I know that the Nikon AF 105 Micro is like that. There are many lens characteristics that are not listed in an MTF chart that are important to take into consideration when buying a lens. There is: flair resistance, field curvature, coma, distortion, boke, actual resolution and not just the MTF combination of resolution and contrast, also the lens's contrast and it's "look", color bias in the lens not to mention build quality. How well a lens does at infinity on an MTF chart is the last thing that I want to know. Unless you do all of your photography at infinity of non-colored subjects in non flair situations where distortions will never be obvious, then you need to know more about your lens than that. The bulk of my photography is near to middle distances ( 3-25 feet) and often in low light where I have my aperture wide open. The Photodo charts tell me nothing about how my lenses will work under those circumstances. Every lens that I own was used and tested on film in the situations like those that I work in before I bought them. I know that they perform the way that I need regardless of what the numbers are supposed to say.
"Listen to the experts and then make up your own mind"
Jonathan Castner -Photojournalist
Denver, Colorado
Online folio at: http://www.jonathancastner.com

Date: Mon, 27 Mar 2000
Subject: Re: Mamiya 645 AF vs Contax 645 Lens Tests
From: Chris Lee chrislee1@home.com
Newsgroups: rec.photo.equipment.medium-format
(I swear I saw a very similar thread somewhere else very recently.)
Another fact to take into account is that Popular Photography do not test medium format lenses themselves. Lacking the proper equipment to test medium format lenses, PP assesses the lenses on the basis of the data that manufacturers supply to PP. (This is stated explicitly in the reviews.) As such, these so-called tests are all done by different manufacturers under different circumstances in accordance with different standards, and are hardly comparable.
That is not to say that the Mamiya lenses could not better the Zeiss offerings, of course. But these tests alone are never conclusive. It would be so much better if magazines here begin to publish actual photos taken by these lenses under controlled conditions. This is very often done by Japanese magazines, and sometimes the results can be quite strikingly visible.

From: tab@IPA.FhG.de (Thomas Bantel)
Newsgroups: rec.photo.equipment.35mm
Subject: Re: Value of photodo ratings?
Date: 07 Jun 1999
"Billy R" willreed@worldnet.att.net writes:

>Can you elaborate on these multiple conflicting test results and provide >evidence that Photodo's ratings are incorrect?

There's no need to prove Photodo's ratings incorrect. I think they are as correct as they can be. But you have to understand how the measurements are done and how the ratings are calculated from the raw data. They are very clear and honest about all that - and probably they are aware of the shortcomings as well.
First, MTF values don't tell the whole story about a lens. There is also flare resistance, distortion, color rendition, bokeh :-) ... Other tests (or personal experience) might include these and other qualities in their ratings and therefor lead to conflicting results.
Second, there are only MTF values for 10, 20 and 40 lp/mm. Nothing beyond that and nothing in between.
Third, all the MTF measurements are done with the lens focused for the center of the image. For a lens with a perfectly flat field, this is ok. For other lenses, this decreases the off center MTF values. From photodo's MTF curves, you will never know if a bad value in the corners means the lens is not sharp in the corners or if it was simply not focused for the corners. If you're taking pictures of a three dimensional scene, flatness of field isn't all that important. A lens with relatively bad corner values may still be able to render a subject in the corner as sharp or sharper than a lens with better corner values. Therefor, the measured MTF values do not necessarily reflect the real world performance of a lens.
Also, one lens may be excellent when it comes new out of the box, but after some use it may get worse because the lens elements get out of alignment. So ruggedness and construction are also a quality factor.
Thomas Bantel

From: liam@ork.net
Subject: Re: Value of photodo ratings?
Newsgroups: rec.photo.equipment.35mm
Date: Tue, 08 Jun 1999
Billy R willreed@worldnet.att.net wrote:

> It's unfortunate that you chose to discount the value of MTF data because > MTF measurements address specifically those same issues that you mentioned. > MTF curves show the lens' ability to maintain the image spatial frequency > (in number of line pairs per millimeter) from the center to the edges of the > image field -- a lens which rapidly drops off in MTF away from the center > axis will produce images that are not uniformily sharp. Typical MTF curves > further evaluate the lens performance at different f-stops and include both > sagital and tangential information. MTF is the key to modern optical > technology and it is so accurate that lens designers can rely on MTF > calculations to optimize the lens design without needing to run prototype > samples. It is clear that Photodo did not randomly choose to rate lens > performance based on MTF data.

However, it has been suggested in the past that Photodo simply chose one lens of a type, and tested that one only. So while its tests are correct, perhaps they should have at least tested a few, mayb e 2 or 3, in case they got an especially bad (or especially good) sample by accident.
Do you have any further information to shed on this issue?

From: johnchap johnchap@wdn.com
Newsgroups: rec.photo.equipment.35mm,rec.photo.equipment
Date: Tue, 25 Jul 2000
Subject: Free Lens Testing Instructions/kit
I have finally gotten my lens testing site, which I had to move last December, back up and, I believe, operational. Please go to href="http://wdn.com/~johnchap/lenstest/testlens.htm" for some description and links to the instructions which you can download and print out. Also Included in the material are the resolution charts which are needed.
The material will instruct you how to set up the charts, shoot the tests, and interpret the results. I have always found it very enlightening how certain lenses do. The results you will get may very well surprise you. I have found several inexpensive lenses with excellent optical quality. On the other hand, I have found some very expensive lenses that are not nearly as good as their much less expensive brothers.
If you encounter any problems, have any questions, or have any advice how I can improve access and usability of the site, please email me.

From Leica Mailing List:
Date: Thu, 24 Aug 2000
From: "Erwin Puts" imxputs@knoware.nl
Subject: [Leica] Value of test reports?
Recently we could read on this list a remark about the value of the measurement of characteristics of a lens that are related to image quality.
In fact a reappearance of Mr. Johnston's well-known view about lens testing, it is stated that any objective lens test (that is a test that tries to establish numerical values on a set of parameters) can only capture those characteristics of optical performance that are irrelevant or unimportant for the true appreciation of a lens' performance by an artistically or expressively trained photographer's eye. Those aspects of a lens that delight or excite the working photographer in viewing his/her results in print or on screen, cannot be measured or even discussed objectively. As we are entering the domain of belief or even religion here, it is futile to try to argue against this view. You can not discuss in any meaningful way unless you try to follow the same set of rules or basic premises.
The more intriguing question is why do some persons believe that objectivity in lens testing is irrelevant or counterproductive. The obvious fact that all manufacturers use MTF tests and all other kinds of measurements to create and produce the lenses with characteristics that some only wish to discuss in personalised statements is a logical contradiction. But so be it.
Why negate the value of objectivity in lens testing and evaluation? One very obvious reason is a commercial one. Quite recently I was emailed by a customer in an USA store who asked me this: the salesperson had for sale two Summilux lenses 1.4/35, one the aspherical and one the ASPH. The aspherical was twice as expensive as the ASPH, because the salesperson stated that the first (aspherical) version was much better optically than the current (ASPH) version. Now this is nonsense and that I told the buyer, who went for the ASPH version for half the price. If the salesperson had presented the buyer with objective test reports he would never have made this statement and so could not justify the difference in price. Yes, yes, the aspherical is a collectors item and because of scarcity may demand a higher price, but that is not what the salesperson told the customer who was obviously not interested in a collectible.
Second reason why objective lens reports are not popular is the loss of fun factor. If we believe whatever report the discussion is closed. It is established that lens A is better than lens B. Period. So buy lens A if you need best quality and start taking pictures. No fun at all? But if we believe that a test can not give conclusive evidence we are in for a never ending discussion, which is enjoyable in itself. Then we can point out that PopPhoto notes that the 1.4/35 asph has best wide open performance of all lenses tested, that Modern however remarked that stopped down the asperical is better, that Viewfinder in an article did not find significant diferrences, but noted more coma in the far corners, that CdI gave 5 stars, but that a friend who is a professsioal photographer swears by the ASPH, but that a noted NatGeo-rapher had sold his as he was not content with the bo-ke and so on. Of course I am fantasising here, but the message is clear and recognisable.
The discussion on this list re the quality and merits of the Minolta and Leitz designs is a proof. I am not going to jump into this discussion, I already overstretched, regrettably, my backbench postion by commenting on Dan's presentation of 4 comparative pictures. There has been a reference to a site which presents the results of several magazines of the same lenses. While it is helpful to note that test results stray widely, it does not answer the fundamental question: if we want to get reliable info based on measured results, which one to trust.
There are so many stories here that are not true that I do not know where to start: The notion that you should need a statistically representative sample to make meaningful statements, is not realistic: first: a representative sample would comprise at least 20 items. Which magazine can afford this? And what manufacturer can give 20 lenses per magazine. As there are about 200 magazines in the world who need fair treatment, so the factory would have to deliver 4000 lenses. Assume the Leica 1.4/35 aspherical which has been produced 2000 times. The full production is not enough to deliver the sample to all magazines. And would magazines be happy with 20 lenses. Not all all!
It takes me a few months to test one item!! And is it necessary? No, QC nowadays secures minimum standards. Is it true that a magazine gets specially prepared versions of a lens? Most unlikely. The magazines I work for get off the shelf boxes. My Leica test lenses are taken from the shelf by myself. Is it true that a magazine keeps testing a series of lenses till they find one that meets their standards? Nonsense. Try to work for a magazine and you will find out that this is impossible. You have a deadline: get a lens in week one, test it in week two, find in week three it is not OK, ask a new one (often if it is a new lens, only one is available!!!) and you get one three weeks later, you test it etc. Deadline passed. No review needed anymore as all other magazines have reports on the lens! Every magazine has its own procedure of testing and style of reporting. YOU CANNOT COMPARE THEM!!! Unless you know intimately and in great technical detail what they do and how they work.
Magazines do not tell you or in such terminology that you do not understand what exactly they are doing. Take Photodo. MTF tests are fine. The crucial question at what distance they set the focal plane, when testing the lens is never answered. I asked them several times to specify this simple fact. They refuse. Without such a knowledge the results are most misleading. If you do not know about the basics of optical shop testing and the magazines are as evasive as the Russians about the sinking of the Kursk, you are in the desert. Compare this behaviour with the one at Zeiss or Leica where the people explain to the most minute detail what they evaluate, why they do it, what the results are, what interpretations they use, where the grey areas are, what the margins are and I must say I believe the manufacturers data more than the results in the magazines.
Erwin

From: mtclev@aol.com (MtClev)
Newsgroups: rec.photo.technique.nature
Date: 29 Aug 2000
Subject: Re: 400mm
It depends on how much you shake.
Here's how to find out:
Place a newspaper on a wall about 20ft from the front of your lens.
Start at 1/1000 of a sec. and shoot a photo of the paper.
Work your way down to 1/60 of a sec.
Remember to close down your f-stop each time you lower the speed.
Look at your chomes and see what is the slowest speed you can still read the normal-sized print . I use a 4x loupe. Some people use a 10x and think I'm nuts to try without a 10x.
That is the slowest speed YOU can hand-hold and still get sharp photos. Of course, if you have too much coffee, your results may differ.
Chris

From Contax Mailing List:
Date: Tue, 27 Feb 2001
From: Bob Shell bob@bobshell.com
Subject: [CONTAX] OT: doing new stuff
Just a note to let all of you know that I'm now working for BestStuff.com as their Photo Guru. Just signed on last week. I'm still doing all the same things as before, but just added this on.
If you go to www.beststuff.com you will see my report on scanners from PMA featured, and at the bottom is a click link to my longer PMA report.
I'll be doing monthly articles about things related to photography there, so take a look now and then.
Bob

From: dbaker9128@aol.com (DBaker9128) Newsgroups: rec.photo.equipment.medium-format Date: 29 Sep 2001 Subject: Photodo or PhotoDon't ??? In the past on this forum and in others I have been a staunch defender of the web site Photodo ( http://www.photodo.com/ ) and its methodology for testing lenses for Modular Transfer Function (MTF). I must now share with the group that I have had some doubts creep in, given a recent challenge gone unanswered by Photodo and its progenitor Lars Kjellberg. Specifically, Mr. Erwin Putts a well known and respected Leica expert stated in his September 4th Leica E-Newsletter (#5/22) regarding MTF graphs: "Thirdly there are several image planes where the designer may want to focus his lens upon. Remember that a point is not focused as a point but as a patch of light. The rays that enter the lens are focused not to a point but to a caustic, which is a kind of paraboloid that is an hourglass shape turned horizontally. Therefore the image plane can intersect the waist of the hourglass at several locations, giving a different balance of contrast and resolution. You can go for the finest resolution, but loose contrast or the other way around. If the person behind the MTF gear does not know where the designer wishes the focal plane to be, he can seriously go wrong. I asked the man from Hasselblad who does the Photodo measurements what focal plane he chooses. He refused to answer and that means that the Photodo results are very suspect." I e-mailed Photodo on September 5th informing Mr. Kjellberg of Mr. Puts's assertion and requesting some form of response. No response has been forthcoming, to my knowledge. Doug from Tumwater
From: "Tom Bloomer" bloomer@/"NoSpam>"/snip.net> Newsgroups: rec.photo.equipment.medium-format Subject: Re: Photodo or PhotoDon't ??? Date: Sat, 29 Sep 2001 The interesting thing about the MTF graphs on Photodo is that those graphs tend give results that are aligned with the assertions of folks that have commented on the qualities of lenses on other web sites. For example, I have seen many folks comment that the Zeiss 80mm F2.8 for Hasselblad tends to be a little soft in the corners. The MTF graph on Photodo shows the contrast falling off sharply in the corners of that lens. The Nikon 85mm F1.4 is known to have flat low contrast wide open and very high contrast and sharpness stopped down. The subjective comments from Grover Larkins and David Ruether at http://utopia.knoware.nl/users/leover/nikon/nikkor.htm for the most part agree with the Photodo ratings for the same Nikkor lenses. Comments by users on www.photographyreview.com also agree with the test results from Photodo. One must realize that MTF graphs do not tell the whole story about a lens. Many have commented on the fact that some lenses perform exceptionally well within a very narrow range of focusing distances. This kind of information does not show up in a MTF graph unless the test is done at multiple distances. I don't believe that it is practical for one to do such testing for all lenses without being compensated. I am happy that a site like photodo exists. It provides a commendable volume of test information that can help someone who is shopping to make an educated decision about which lenses to buy. >If the person behind the MTF gear does not know where >the designer wishes the focal plane to be, he can seriously go wrong. So is there a specific lens tested on Photodo that you disagree with the results from personal experience? If you doubt the validity of those tests, can you show us other tests that are published on the web that you consider valid? -- Tom Bloomer Hartly, DE
From: Anders Svensson anders.-.eivor.svensson@swipnet.se> Newsgroups: rec.photo.equipment.35mm Subject: Re: Photodo or PhotoDon't ??? Date: Sun, 30 Sep 2001 ArtKramr wrote: > Anders, > I have a few questions about the MTF graphs. Do they represent an average lens, > a specific lens or no lens at all just designers aim points? Does every lens > you buy accurately represent the published MTF graph regardless of whatever > production variations may be present? You tell me. (s) As for the Photodo MTF curves, Lars Kjellberg openly says that they are from tests of one lens only. That's obviously a weakness for the reasons you mention, and it isn't the only problem with Photodo, either. Photodo does not judge build quality or "optical sweetness", for example, nor do they test macro lenses at a realistic working distance. But wouldn't you agree that it might be better to get some kind of performance indication from a independent source, even if its methodology is less than 100 % reliable? The real value of Photodo is (IMHO) that they have tested a lot of lenses (including some pretty old ones) the same way, and compiled a consistent data base, available for free. I bet there are many other, perhaps better ways to test lenses, but one of the problems are to align them to a reasonable standard. Chasseur d'Image has a three star rating including a subjective grade, David Ruether has a rating and a annotation, some German magazines use graphs and a verdict, PopPhoto gives *their* numbers and a verdict. I think they all can be used to shortlist a few lenses to look further at, if their limitations are understood. -- Anders Svensson mail: anders.-.eivor.svensson@swipnet.se
From: T P please.reply@newsgroup> Newsgroups: rec.photo.equipment.medium-format Subject: Re: Photodo or PhotoDon't ??? Date: Mon, 01 Oct 2001 "thc" a2toothfairy@mw.mediaone.net> wrote: > Unfortunately, the thread does not reveal where to find Mr Puts' Leica > E-newsletter. > > Ladies and gentlemen - Please be aware of the disclaimers posted at Photodo > and other sites that purport to rank photo gear according to quality. > Photodo explains how their rankings are derived, and it is our fault if we > misinterpret or misuse their data. What nonsense. Photodo publish overall ratings. You can't blame anyone who knows no better than to accept those ratings at face value. The disclaimers are hard to find on the quirky Photodo web site and are not written with newbies in mind. THERE ARE *NO DISCLAIMERS* ON THE OVERALL RATINGS PAGES, AND *NO ENCOURAGEMENT* TO LOOK ANY FURTHER FOR THEM. No apologies for shouting, but you really need to wake yourself up. > With all due respect to Erwin Puts, I believe that the criticism of > Photodo's modular transfer function (MTF) is somewhat of a red herring, > possibly even a straw-man argument. Photodo does not make any great claims > about their MTF data. See above. I reiterate that MTF tells you only about a combination of resolution and contrast that photographers perceive as apparent sharpness. MTF tells you *absolutely nothing* about distortion and control of aberrations, which leads to some people buying lenses that appear sharp but tend to produce harsh results. > But again, Mr Puts does not publish any test data to support his > conclusions. Just where does Puts publish the MTF graphs that support his > conclusions? See above. MTF is only a part of the story. Useful, yes, but only a part. > Photodo publishes its MTF graphs for all to see how well (or > how poorly) its test data support its conclusions. That's complete nonsense. Photodo's conclusions ARE the MTF test data. The test data don't "support" Photodo's conclusions because THEY ARE those conclusions. The overall rating is mathematically derived from the MTF figures. > Can't we mere mortals assume that an operator at > Victor Hasselblad AB knows how to obtain valid MTF graphs? If Mr Puts > believes that the operator at Hasselblad is incompetent, why are we expected > to trust MTF graphs published by Leica, or Nikon, or Canon, or anyone else? The operator at Hasselblad *may* be competent, but the gross assumptions which went into designing the test methodology are *extremely* suspect. > Photodo also states, "No other variables, such as distortion, flare, or > ghosting are taken into account." Photodo also states that they generally > test lenses at f/4 and at f/8, and that seems to be what angers Mr Puts, who > writes elsewhere (refer to the reference found 2 paragraphs below), "But for > the modern Leica lenses, where optimum performance quite often starts at f/2 > and is at its height around f/2,8 this choice is bad, as the best part of > the data is not put into the equation." Unfortunately, Puts is incorrect > about Photodo not putting the f/2 and f/2.8 data into the equation, and > Photodo states up front that they measure at full aperture as well as at f/4 > and at f/8. Go to http://www.photodo.com/nav/prodindex.html and you can > find 12 Leica-M lenses with MTF graphs for full aperture. That's a fair point. But Photodo rarely (or maybe never) test lenses at apertures smaller than f/8. They state that their reason for this is that 35mm lenses are diffraction limited from f/11 onwards. Of course this is patent nonsense. It's true that diffraction is starting to show at f/8, but the idea that it becomes a limiting factor to MTF results at f/8 or f/11 is quite incredible. As you stop down beyond f/8, most 35mm lenses will maintain or even improve their MTF until f/11, and some at f/16, because the factors that are improving MTF still greatly outweigh the deleterious effect of diffraction. Beyond f/11, the MTF of most 35mm lenses reduces as diffraction begins to overcome the other factors. But the extent of that reduction, and the rate at which it occurs, make a *huge* difference. Some lenses perform exceptionally well at f/22 and others don't. I repeat that the difference can be *huge*. HUGE! This is well documented - you only have to look at the MTF or resolution curves in reputable magazines to see that. The difference can easily be 100%, and is often more. Landscape photographers, who mostly use their lenses stopped down for obtaining the maximum possible depth of field, all know this. This is PRECISELY the information they need, and look for, when buying a lens. Where do they find this on the Photodo site? IT'S NOWHERE TO BE FOUND! Because of Photodo's arrogant and simplistic approach to testing and the presentation of MTF test data, the only information on Photodo's site that is of any use whatsoever is the statement that suggests all lenses perform the same at apertures smaller than f/8. OF COURSE THIS IS *NONSENSE*! > In its lens tests, Popular Photography (US) magazine describes its MTF > results for infinity distance and for closest focusing distance. Also, an > article ("Popular Photography upgrades SQF optical testing system," page 78, > August 2001 issue) describes their optical bench testing as using a > pin-point light source instead of a slit, and extending the light's spectral > distribution to allow a wider range of visible red light. Photodo does not > test at closest focusing distance. Anyone who presents "Bait and Switch Monthly" as a paragon of virtue amongst publishers of lens test data is probably in need of psychiatric help. Your credibility just slipped below zero. > Mr Puts is aware that different testing methods will generate > MTF graphs that are not comparable, and Photodo states how it > derives its MTF graphs. What's the argument about? See above. -- Best regards, TP
From: "Tom Bloomer" bloomer@/"NoSpam>"/snip.net> Newsgroups: rec.photo.equipment.medium-format Subject: Re: Photodo or PhotoDon't ??? Date: Mon, 1 Oct 2001 Both www.hasselbladusa.com and www.zeiss.com have MTF graphs published. Their graphs are similar but not identical to the ones on Photodo. Another non-sceintific coincidence? -- Tom Bloomer Hartly, DE "Q.G. de Bakker" qnu@worldonline.nl> wrote thc wrote: > Can't we mere mortals assume that an operator at > Victor Hasselblad AB knows how to obtain valid MTF graphs? We can. But why is it that MTF performance of Zeiss/Hasselblad lenses is measured (and published) by Zeiss, not Hasselblad?
From: rmonagha@smu.edu (Robert Monaghan) Newsgroups: rec.photo.equipment.medium-format Subject: sample variation, beating 100 lpm, Re: Photodo or PhotoDon't ??? Date: 1 Oct 2001 sample variation is probably one order of magnitude more important than any modest variation in setting focal plane positions during MTF testing ;-) I've been collecting more data from lens resolution and contrast testing, and variations of 12% in resolution are "nominal" (based on USAF 1951 charts). As near as I can extrapolate, it looks like prosumer lenses vary circa +/- 1/2 grade out of 5 (cf. photodo) during reported tests. Moreover focusing errors of as little as 2mm rotation (1mm on many AF lenses) (see mf/critical.html) can reduce lens resolution by up to 50%, or more (at f/5.6 or faster). I suspect this explains many of the sometimes anomalously high or low lens resolution values reported during a series of such tests on many lenses (e.g., Pop Photo..). another interesting study http://people.smu.edu/rmonagha/mf/limits.html#100 shows how many cheapy 50mm f/1.8 optics can deliver over 100 lpmm (!!) on HCF copy film and microektachrome color slide films. Many such lenses are evidently running into film resolution limitations rather than inherent lens resolution limits (see mf/lenslpm.html for film resolution values). One tester consistently gets 1/3rd higher resolution values by using strobe lighting of targets (Dr. Eugen J. Skudrzyk) which helps eliminate the effects of small camera vibration at these high resolution testing limits. My impression is that we are really testing our procedures for testing the lenses, rather than really testing the lenses ;-) And even if we had a very careful lens test report, it would only apply to that particular lens, and probably not to the one we end up buying and using ;-) grins bobm
From: "thc" a2toothfairy@mw.mediaone.net> Newsgroups: rec.photo.equipment.medium-format Subject: Re: Photodo or PhotoDon't ??? Date: Tue, 02 Oct 2001 I'm very sorry that T P has such difficulty in controlling his anger when he replies to a civil posting. So far as I am aware, this is the first newsgroup meeting I've had with T P, and I believe that he chooses to present himself poorly on the Internet: He writes in capital letters (the equivalent to screaming) and also states, "No apologies for shouting, but you really need to wake yourself up." No apology for unprovoked screaming? My first impression of T P is not a favorable one. My original post gave reference to exactly where ("Understanding the MTF graphs, numbers and grades") Photodo stated its disclaimers about the limitations of its data, and a hotlink to that disclaimer's URL appears at the bottom of each page of MTF graphs. However, T P chose to invent his own reality, and he was civil when he wrote, "The disclaimers are hard to find on the quirky Photodo web site," but he then screamed in capital letters, "There are *no disclaimers* on the overall ratings pages, and *no encouragement* to look any further for them." In my humble opinion, I believe that it is not difficult to discover those disclaimers, for one only needs to click on the URL hotlink("Understanding the MTF graphs, numbers and grades") that appears at the bottom of each page of MTF graphs. Duh, just how much "encouragement" does a reader need, when Photodo has the "Understanding" URL hotlink on the same page as the graphs? T P writes that MTF tells *absolutely nothing* about distortion and control of aberrations, and Photodo's disclaimer explicitly states, "No other variables, such as distortion, flare, or ghosting are taken into account." Golly gee gosh, folks. It appears that T P wasn't aware that he and Photodo do agree on some of the same issues. Maybe T P needs to comprehend better what he reads, before he so rashly and rudely writes his rebuttals. T P writes, "MTF is only a part of the story. Useful, yes, but only a part." However, Erwin Puts (in his article, "How to exploit the Leica quality? The ultimate Leica FAQ," which can be found at http://www.imx.nl/photosite/leica/technics/faq.html ) expresses his opinion that the MTF graph, "Presents the most comprehensive information about the optical performance of the lens and correlates very well with the perceived image quality. But the MTF graphs are difficult to interpret and when generated by different methods are not comparable." I tend to agree with Erwin Puts more than I agree with T P. Photodo states (in "Understanding the MTF graphs, numbers and grades"), "We have been testing lenses with Hasselblads Ealing MTF equipment since 1991. All MTF reports are made with the same equipment and by the same operator (Per Nordlund at Victor Hasselblad AB)." T P attacks the validity of those MTF graphs, "The operator at Hasselblad *may* be competent, but the gross assumptions which went into designing the test methodology are *extremely* suspect." However, T P provides no references to support his contention that the testing methodology is *extremely* suspect. All Photodo claims is that a technician at Victor Hasselblad tested the lenses and obtained the results that Photodo publishes. Another poster lit a candle rather than curse the darkness, and provided some illuminating information that probably causes an apoplexy for T P: > "Tom Bloomer /snip.net>" bloomer@/"NoSpam> wrote > Both www.hasselbladusa.com and www.zeiss.com have MTF graphs published. > Their graphs are similar but not identical to the ones on Photodo. Another > non-sceintific coincidence? > -- > Tom Bloomer > Hartly, DE T P conveniently ignored again Photodo's disclaimers, and he criticized Photodo for not providing MTF graphs for f/11 and f/16 and f/22 apertures, and he screamed loudly, "It's nowhere to be found!" Photodo explicitly stated its limitations, but T P accused Photodo of being arrogant and simplistic because Photodo did not meet his particular wants and desires. Testing methodology needs to be improved constantly, and Popular Photography (US)magazine announced (in its August 2001 issue, page 78) that, even with the full financial backing of its parent company (Hachette Filipacchi Magazines, Inc.) and the unstinting help from the staff of experts at the leading supplier of high-end optical testing equipment in the United States (Optikos Corporation of Cambridge, Massachusetts), it took over six months for them to upgrade their optical bench to provide improved MTF data. Popular Photography has published for over 50 years, which implies that it must be successful in meeting some of the needs of its subscribers. Quite frankly, I applaud that magazine for spending the considerable amount of money necessary to obtain that improved MTF data for publication. I am a mere mortal being, and I do not have access to much more MTF data than what Photodo and Popular Photography make available to the reading public. However, T P sarcastically expresses his venomous opinion, "Anyone who presents 'Bait and Switch Monthly' as a paragon of virtue amongst publishers of lens test data is probably in need of psychiatric help. Your credibility just slipped below zero." T P unfortunately is unable to offer any other sources of lens test data. He denigrates what is offered, and he offers no alternative to what he despises. I suspect that T P probably would find fault if Popular Photography were to publish the statement, "The sun will rise tomorrow." T P (who Q. G. de Bakker called "Tony," as noted in "Q.G. de Bakker qnu@worldonline.nl> wrote in message news:9p9rm7$cgr$1@nereid.worldonline.nl...) appears to be a highly opinionated person who apparently cannot be civil when he disagrees with a post. He apparently chooses to curse the darkness rather than to light a candle, he chooses to stir up mud rather than to provide reputable information for the Internet, and he eventually resorts to defaming personal insults ("in need of psychiatric help") of people he does not know. I know of children who have more maturity, and they do not refuse an apology for their unprovoked shouting and uncivil behavior. T P is not a person with whom I would choose to break bread. ___________ "T P" please.reply@newsgroup> wrote... > "thc" a2toothfairy@mw.mediaone.net> wrote: > > > Unfortunately, the thread does not reveal where to find Mr Puts' Leica > > E-newsletter. > > > > Ladies and gentlemen - Please be aware of the disclaimers posted at Photodo > > and other sites that purport to rank photo gear according to quality. > > Photodo explains how their rankings are derived, and it is our fault if we > > misinterpret or misuse their data. > > What nonsense. Photodo publish overall ratings. You can't blame anyone > who knows no better than to accept those ratings at face value. The > disclaimers are hard to find on the quirky Photodo web site and are not > written with newbies in mind. > > THERE ARE *NO DISCLAIMERS* ON THE OVERALL RATINGS PAGES, > AND *NO ENCOURAGEMENT* TO LOOK ANY FURTHER FOR THEM. > > No apologies for shouting, but you really need to wake yourself up. > > > With all due respect to Erwin Puts, I believe that the criticism of > > Photodo's modular transfer function (MTF) is somewhat of a red herring, > > possibly even a straw-man argument. Photodo does not make any great claims > > about their MTF data. > > See above. I reiterate that MTF tells you only about a combination of > resolution and contrast that photographers perceive as apparent > sharpness. MTF tells you *absolutely nothing* about distortion and > control of aberrations, which leads to some people buying lenses that > appear sharp but tend to produce harsh results. > > > But again, Mr Puts does not publish any test data to support his > > conclusions. Just where does Puts publish the MTF graphs that support his > > conclusions? > > See above. > > MTF is only a part of the story. Useful, yes, but only a part. > > > Photodo publishes its MTF graphs for all to see how well (or > > how poorly) its test data support its conclusions. > > That's complete nonsense. > > Photodo's conclusions ARE the MTF test data. The test data don't > "support" Photodo's conclusions because THEY ARE those conclusions. > The overall rating is mathematically derived from the MTF figures. > > > Can't we mere mortals assume that an operator at > > Victor Hasselblad AB knows how to obtain valid MTF graphs? If Mr Puts > > believes that the operator at Hasselblad is incompetent, why are we expected > > to trust MTF graphs published by Leica, or Nikon, or Canon, or anyone else? > > The operator at Hasselblad *may* be competent, but the gross assumptions > which went into designing the test methodology are *extremely* suspect. > > > Photodo also states, "No other variables, such as distortion, flare, or > > ghosting are taken into account." Photodo also states that they generally > > test lenses at f/4 and at f/8, and that seems to be what angers Mr Puts, who > > writes elsewhere (refer to the reference found 2 paragraphs below), "But for > > the modern Leica lenses, where optimum performance quite often starts at f/2 > > and is at its height around f/2,8 this choice is bad, as the best part of > > the data is not put into the equation." Unfortunately, Puts is incorrect > > about Photodo not putting the f/2 and f/2.8 data into the equation, and > > Photodo states up front that they measure at full aperture as well as at f/4 > > and at f/8. Go to http://www.photodo.com/nav/prodindex.html and you can > > find 12 Leica-M lenses with MTF graphs for full aperture. > > That's a fair point. But Photodo rarely (or maybe never) test lenses at > apertures smaller than f/8. They state that their reason for this is > that 35mm lenses are diffraction limited from f/11 onwards. > > Of course this is patent nonsense. It's true that diffraction is > starting to show at f/8, but the idea that it becomes a limiting factor > to MTF results at f/8 or f/11 is quite incredible. > > As you stop down beyond f/8, most 35mm lenses will maintain or even > improve their MTF until f/11, and some at f/16, because the factors that > are improving MTF still greatly outweigh the deleterious effect of > diffraction. > > Beyond f/11, the MTF of most 35mm lenses reduces as diffraction begins > to overcome the other factors. But the extent of that reduction, and > the rate at which it occurs, make a *huge* difference. > > Some lenses perform exceptionally well at f/22 and others don't. > I repeat that the difference can be *huge*. HUGE! This is well > documented - you only have to look at the MTF or resolution curves in > reputable magazines to see that. The difference can easily be 100%, and > is often more. > > Landscape photographers, who mostly use their lenses stopped down for > obtaining the maximum possible depth of field, all know this. This is > PRECISELY the information they need, and look for, when buying a lens. > > Where do they find this on the Photodo site? > > IT'S NOWHERE TO BE FOUND! > > Because of Photodo's arrogant and simplistic approach to testing and the > presentation of MTF test data, the only information on Photodo's site > that is of any use whatsoever is the statement that suggests all lenses > perform the same at apertures smaller than f/8. > > OF COURSE THIS IS *NONSENSE*! > > > In its lens tests, Popular Photography (US) magazine describes its MTF > > results for infinity distance and for closest focusing distance. Also, an > > article ("Popular Photography upgrades SQF optical testing system," page 78, > > August 2001 issue) describes their optical bench testing as using a > > pin-point light source instead of a slit, and extending the light's spectral > > distribution to allow a wider range of visible red light. Photodo does not > > test at closest focusing distance. > > Anyone who presents "Bait and Switch Monthly" as a paragon of virtue > amongst publishers of lens test data is probably in need of psychiatric > help. Your credibility just slipped below zero. > > > Mr Puts is aware that different testing methods will generate > > MTF graphs that are not comparable, and Photodo states how it > > derives its MTF graphs. What's the argument about? > > See above. > > -- > Best regards, > TP
From: rmonagha@smu.edu (Robert Monaghan) Newsgroups: rec.photo.equipment.medium-format Subject: magazine testing, only good news? Re: mfger supplied test lenses Date: 3 Oct 2001 Hmm? For someone who does testing of complex systems for a living, you don't seem to have a lot of confidence in tests ;-) ;-) re: value of multiple test reports Actually, I was interested to note that climatologists are using a large number of reported temperatures to derive a highly accurate (0.1 deg. C) average temperature (e.g., for Europe 100 years ago) despite working with widely different techniques and instruments (thermometers, freezing lakes) not calibrated to a precise standard. I think the same process is happening heuristically with us using online and published lens tests; with enough tests, you begin to approximate the sampling issues which I am asking about, and can isolate lenses in which there are some sample variation problems to be wary about ;-) So while tests may not be precisely comparable, the net tendency (pun intended) does reveal aspects of the underlying lens population being sampled, and that's why we go to the trouble of checking multiple sources to get a "feel" for the lens quality and sample variation issues ;-) still, we would prefer more accurate data, but there isn't any interest evidently on the magazines (as captives of advertisers?) or mfgers in providing such statistically valid data to us. They have it, we don't, and that's the way the industry likes it ;-) my real point is that the issue of sample variation undercuts the validity of single point ratings (a la photodo) and even MTF charts from (possibly cherry picked) samples. I am also arguing that this is why you have to test the lens you select, but also, as Bob S. pointed out, because abuse in shipping or other events (demos? returned goods) may have impacted (pun intended) the lens and its performance. You can't rely on somebody else's tests, but can only use them to narrow the field of potential lenses to test... If you were charging $1,000 US more for your lenses than a competitor, wouldn't you want to make buyers paranoid about their higher chances of getting a lemon lens with your competitors, if only indirectly by praising the tight tolerances of your own products? I would ;-) ;-) I think the real reason this info isn't more widely available is primarily that the present system, aimed at the statistically non-savvy public, GENERATES ONLY GOOD NEWS for the mfgers and magazines. We have some mfgers who publish MTF charts based on the ideal, computer design lens, but not the actual production lenses. Others publish lens test data on a production lens, but only one lens, and not a spectrum of performances within the production run (at +/- 1, 2, 3 s.d.), although I presume they have such data from Q/C sources readily available. At best, a more valid statistical sampling view of actual production would only bring the average quality expectations downward for buyers with access to this info. So ideal or (possibly selected cherry samples) actual MTF charts are likely to be only good news, yes? It also keeps buyers from trying to find the better lenses by cherry picking them in the stores, if they assume that all lenses are absolutely identical to 2 or 3 decimal places as in the magazine ratings ;-) ; -) And as I have suggested, lower end mfgers would be nuts to not "cherry pick" their better/best lenses which they provide for magazine tests, given how critical such reviews are for their sales and profits! ;-) And we know that magazines such as MP/PP consult with advertisers about anomalous results, such as the bad production (mis-spaced) vivitar teleconverters I cited earlier in this thread. Again, I understand that magazines are dependent on happy advertisers, but the aura of independent lens testing is thereby suspect given such examples. Bob S also makes a good point regarding advertorials, which are ads disguised as editorial commentary, but labeled as such in small type ;-). He is too much of a gentleman to point out the editorials and articles in some magazines which are ghost written by lens mfgers' marketing types (obviously excluding HPMktg here) and passed off as independent research and writing in those magazines. Really now, how many really bad lens reviews have you seen? ;-) Okay, name just one bad lens review! ;-) And there is also the slavish copying of press releases in magazine "announcements" section, down to glowing text about products, which is often seen as a benefit for advertising (so many inches of free press for so many pages of ads?). How many lenses have you seen reviewed under import or store labels (e.g., Ritz) which don't advertise in that magazine? Why do you suppose that is, if the magazines are so independent? I mean, aren't there a lot of import and store label lens sales by Ritz etc., versus the handful of pro sales for lenses (e.g., fast 600mm f/4 teles) which are of minimal interest to the average magazine subscriber but get lots of glowing reviews anyway? And when a glowing report on say Leica M6 is published in a magazine, doesn't there always seem to be a full page ad for Leica nearby? ;-) Again, we as subscribers have to realize that we are the PRODUCT, it is our eyeballs (and wallets) which are being sold to the advertisers, at least in the major photo pubs which are filled with such ads ;-0) yes? ;-) We only pay the postage and profits, the costs are from the ads. But it is important to keeping subscribers happy to promote our delusions that these magazines are really serving our interests as consumers, right? ;-) As for the magazines, how many of their subscribers resubscribe mainly to get these lens reviews (given the repetitive nature of annual cycle of articles)? If the magazines admit their single sample reviews are irrelevant, won't that cost them subscribers? and ad revenues based on eyeball counts? ;-) If lens vary, and for 35mm prosumer/consumer lenses that seems to be about +/- half a grade average (third/variations.html), then doesn't that put their single test process into question as far as its utility in selecting lenses? The difference in photodo scores between the average fixed lens and zoom lens is only 0.8 out of 5 units, so a +/- .5 range within a lens batch is larger than this average zoom vs fixed lens difference! It is also lots larger than the range usually seen between lenses of different mfgers for comparable cost similar zoom or fixed optics. This is partly why I have become more concerned with issues like film flatness, which is more often limiting the results from pro medium format optics, and higher resolution limit color films, than the optics themselves. These are elements of the final system resolution results which are easier to tweak than the individual optics, but which guarantee better results with all the lenses I'm using ;-) grins bobm
From: "Webmarketing" webmarketing@kconline.com> Newsgroups: rec.photo.equipment.35mm Subject: Re: Photodo or PhotoDon't ??? Date: Mon, 1 Oct 2001 The only really useful way to test lenses meaningfully is to make photos of known repeatable subjects and then view the negatives or chromes with a high power loupe. It's the way I test my lenses. Only in that way can you see the effects of lens design on corner sharpness, contrast, distortion and aberrations. There are always tradeoffs. In order to get a high MTF (corner sharpness) one has to give up some contrast and vice versa. In order to get rid of coma you need to give up some MTF etc. etc. These are just a couple of examples. No magazine does this kind of testing and Photodo certainly doesn't either. I agree with the individual who said that a personal test is the only really meaningful one except that I would include the need to shoot known and repeatable subjects and view the film directly. Good shooting. Fred Maplewood Photography "T P" please.reply@newsgroup> wrote... > "Jay Washington" jdwashtn@earthlink.net> wrote: > > > Okay, I'm not trying to start a flame war, but I would like someone to name > > me another site that is something other than just a list of user opinions. > > I'm not asking as a challenge, I would like as many sites as possible that > > at least gives me some objective data and is not sponsored by the > > manufacturers. Really, I would. > > Nikon is well served by sites that give more useful information than > Photodo. I don't know about other lens manufacturers. > > > I don't understand why photodo get such a > > bad rap. No one is constantly slamming Popular Photography's reviews--maybe > > we all know those are skewed, but I really am confused as to why Photodo, a > > free source of information, always gets rained on. > > Because its subjective approach to testing lenses, and their > presentation of test data as an overall rating, are profoundly > misleading. "Bait and Switch Monthly" is already to be subjective, > biased and grossly misleading. Most people already know that, and > distrust the magazine, so there's no need to repeat it on here. > > > For me at least, they've > > been accurate--even after the fact (I bought a Canon 20-35mm non-L and was > > surprised by the lack of sharpness--when I found out about photodo, it's > > rating matched my experiences). Every lens I've purchased since then with a > > 4.0 or greater photodo rating has been exceptionally sharp (Canon 85 1.8, > > Canon 50 1.8, Sigma 105 2.8). My Canon 24mm TS is not so sharp, but it's > > okay and is amazing for what it does (perspective correction). This okay > > sharpness matches with the photodo rating. I guess I've just been lucky > > that photodo has been accurate for my use. > > There is much, much more to learn about a lens than its MTF curves. If > you really think MTF is the be-all and end-all of lens testing, then > maybe it's time you learned a little more. > > > Anyway, good luck with whatever sources you use before you purchase. > > The best and most reliable test of all is to test it *yourself*, after > you buy it, and return it if it isn't up to scratch. > > Always buy from a store with a good returns policy, and test with a good > slide film such as Fuji Provia 100F. Use a lightbox and a good loupe. > > -- > Best regards, > TP
From: jrd@cc.usu.edu (Joe Doupnik) Newsgroups: rec.photo.equipment.35mm Subject: Re: Photodo or PhotoDon't ??? Date: 3 Oct 01 NickC n-chen@mediaone.net> writes: > I'm not so sure I completely go along with the analogy of what > constitutes a proper method of testing lenses. In designing anything, This familiar discussion basically boils down to some folks being unhappy that Photodo does not provide _all_ information about lenses, particularly statistics of manufacture and so forth. Frankly, they don't have the money to try that, they don't handle entire production runs, and they do publish rather than wait for production to cease, and so on. Even if they tried mass testing there is one small problem: the lens in your hands was not tested by them and may not reside near the median of stat graphs. If you insist upon manufacturing variance information the outfit to pester is clearly the manufacturer. Not a whiner has suggested that. Nor has anyone required a vendor to provide a compete test report on that particular lens in your hands. Rather expensive, that bit. Rather than knocking Photodo on this basis, just accept they provided some info, more than you would have without their testing. We then apply a confidence test using the vendor's reputation and a sampling of customer comments. This gives an indication as to whether the Photodo results are in agreement with field experience. In practice this technique works rather well. History comes into play, and that covers vendor tolerances as observed in the field, and build quality standards, and more. The arguments I hear are of the hand wringing variety: gee, we can't possibly state with certainty that this particular lens, in your hands, is good/bad/better than some others. It is asking someone else to assume all the risks and tell us precisely which is better. Smart folks are able to integrate diverse information to yield an inference that lens X is probably better than alternatives. They don't expect a formal proof nor total knowledge. They also return merchanise which fails to live up to realistic expectations. Joe D.
From: NickC n-chen@mediaone.net> Newsgroups: rec.photo.equipment.35mm Subject: Re: Photodo or PhotoDon't ??? Date: Thu, 04 Oct 2001 Joe Doupnik wrote: > > NickC n-chen@mediaone.net> writes: > > I'm not so sure I completely go along with the analogy of what > > constitutes a proper method of testing lenses. In designing anything, > > This familiar discussion basically boils down to some folks being > unhappy that Photodo does not provide _all_ information about lenses, > particularly statistics of manufacture and so forth. Frankly, they don't > have the money to try that, they don't handle entire production runs, and > they do publish rather than wait for production to cease, and so on. Even > if they tried mass testing there is one small problem: the lens in your hands > was not tested by them and may not reside near the median of stat graphs. But they do not hesitate to publish information the tends to sway the uninformed public's purchasing decisions. There is no warning to the reader or lens researcher that the information provided is not to be considered as being indicative of the final capabilities of a lens line but merely indicate the limited test results of the lens that was tested. Such tests merely indicate what _may_ be inherent in a particular lens design and only inherent in whatever data points that were plotted. > If you insist upon manufacturing variance information the outfit > to pester is clearly the manufacturer. Not a whiner has suggested that. No. The manufacture is not the one publishing what may well be misleading test information. When a magazine takes on the responsibility of publishing lens/camera test data, they are the ones responsible for providing meaningful information. If meaningful information can not be provided due to limited funds, time consuming tests, or having to meet publication deadlines, then the publisher should somehow state those reasons as a warning to readers to be cautious in accepting the test data as being universal. > Nor has anyone required a vendor to provide a compete test report on that > particular lens in your hands. Rather expensive, that bit. However, the vendor, or design/manufacturing company does have test standards and maintains quality assurance data, and does batch testing in order to be assured that a quality lens is being produced in a production run. Depending upon the nature of in-house quality controls, it may be possible to calculate an SD from a product line and print that information. Though it may still be questionable, the information may be more applicably meaningful than a one data point plot of one lens. > Rather than knocking Photodo on this basis, just accept they > provided some info, more than you would have without their testing. Again, if the information is being absorbed by one who accepts the data a being universal, then the practice of publishing such data is wrong, if the information does not carry with it a disclosure that the tests are extremely limited. > We then apply a confidence test using the vendor's reputation and a > sampling of customer comments. Aha!! Yes, by all means, a confidence level of a vendors reputation, gleaned from trials and experiences of many users is very important. >This gives an indication as to whether > the Photodo results are in agreement with field experience. In practice > this technique works rather well. No, I can't agree. In practice, it does not work rather well. I can validate what I say from my own experiences with lenses and camera bodies. >History comes into play, and that > covers vendor tolerances as observed in the field, and build quality > standards, and more. Can't quite agree with this either. I once went off to Fiji armed with brand new camera bodies and an assortment of lenses made by a popular manufacture and after a week there, not a damn thing would work properly. That's a trip I shall never forget. To this day, I shy away from that particular line of equipment. > The arguments I hear are of the hand wringing variety: gee, we > can't possibly state with certainty that this particular lens, in your > hands, is good/bad/better than some others. It is asking someone else > to assume all the risks and tell us precisely which is better. No, I don't think it's quite that bad. I have no hang-up with published information so long as the information being published is not misleading. That can be resolved by the publisher disclosing that the information being furnished can not be considered as being universal or absolute. > Smart folks are able to integrate diverse information to yield > an inference that lens X is probably better than alternatives. By smart, I assume you mean knowledgeable, and upon that assumption I would agree. But, we should consider the tyros who become interested in photography and tend to take what is published as being gospel. Inappropriate published information can virtually destroy a product line by causing newbie potential buyers to shy away from a product line. >They don't > expect a formal proof nor total knowledge. They also return merchanise > which fails to live up to realistic expectations. Again, knowledge is the driver. But one can not gain useful knowledge from reading misleading publications. Nick > Joe D.
From: kwinkler@sennheiserusa.com (Karl Winkler) Newsgroups: rec.photo.equipment.35mm Subject: Re: Why are Un Leicas So Inexpensive ? Date: 5 Oct 2001 "Max Perl" max_perl@post11.tele.dk> wrote > I know we have some Leica users which also have e.g. Nikon > equipment. I think it could be interresting if we could see a > blow up of two identical images taken with a Leica and e.g. a > Nikon. It should of course be one the e.g. Nikon lenses which > a said to be very good. I could suggest the AIS 105/2.5. > I have always belived if you select your lenses carefully most brands > have made "super" lenses where it is purely the 35mm format which > is the limiting factor. > > If a blow up is shown it should be possible to see the difference on > the web? Let me start by saying that these threads seem to pop up every few days, and the funny thing (to me) is that they're just like threads of this type in other groups. One of the most similar is rec.audio.pro where people argue about microphones, speakers, amplifiers... etc. Another one is the rec.audio.high-end, where they argue about this tube vs. that tube, $10,000 speakers cables vs. $1,000 speaker cables (get it?). So here I go, jumping into this one anyway.... I've used a number of different camera systems over the last 20 years or so, including Pentax, Olympus, Nikon, and most recently, Contax rangefinders. Based on shooting slides, making B&W; prints, doing scans, etc., I've reached the following conclusions: 1) The photographer still makes the most difference. Today I shoot with the famous Zeiss rangefinder glass, and yet my images aren't as good as the ones I see in National Geographic, which were mostly shot with Canon & Nikon cameras. Those photographers just blow me away. 2) There is a difference between lenses. But what I've found is that each manufacturer seems to have a few "magic" designs, then some very good lenses, then some mediocre lenses, then a few dogs. As much as I love my OM4 for the way it operates (and God I love that multi-spot meter!), I've really not found the Zuiko lenses to be all that hot (28 f/2.8, 50 f/1.4). That is, with the exception of the 85mm f/2, which is just a wonderful portrait lens. 3) The Zeiss lenses (and Leitz also) are really amazing. But I've seen shots taken with Nikkor, Canon, Pentax, etc. that are stunning. 4) Although it's generally not possible to see the difference in sharpness between lenses by posting pictures to the 'net, you *can* tell the difference in color rendition, contrast, bokeh, etc., which are of course the more important factors in the "look" of a lens. 5) Rangefinder systems seem to produce sharper images with better color and excellent shadow detail. I've heard several theories as to why this is, but I couldn't say for sure I know the reasons. But by using less elements, putting them closer to the film plane, and removing the mirror from the whole works, it begins to make sense. Of course, rangefinders are a pain for some things, including using ultra long lenses (like, anything over 90mm on the G1) and of course for being able to see what is happening with the focusing... but I digress. 6) There's no way to take "exactly the same picture" with two different cameras and lenses (unless your subject is a lab chart - and I for one don't really care to know if one lens reproduces lab charts better. I'm interested in real world performance). So I humbly suggest that the only cure for "is that lens better than mine" is to just stop worrying about it and take lots of photographs. Isn't that why we're all into this in the first place? -Karl
From: "Tom Bloomer" bloomer@/"NoSpam>"/snip.net> Newsgroups: rec.photo.equipment.medium-format Subject: Re: Where can i find MTF for zenzanon lens Date: Mon, 29 Oct 2001 http://www.tamron.com/bron/etrsi_lens.htm http://www.tamron.com/bron/sq_lens.htm http://www.tamron.com/bron/gs1_lens.htm Each lens has a single MTF curve on its page, the graphic is very small. Make of it what you will. Some folks believe the MTF information is useless. YMMV -- Tom Bloomer Hartly, DE ...
From: artkramr@aol.com (ArtKramr) Newsgroups: rec.photo.equipment.35mm Date: 13 Aug 2001 Subject: Re: "Mod. Photo" tests? >Subject: "Mod. Photo" tests? >From: "Mark McKown" bytemark@kscable.com >Date: 8/13/01 > >Hi- >Does anyone know of an online archive of the old "Modern Photography >Magazine"'s tests[cameras, and esp. lenses]? They always seemed fairly >accurate and objective [NOT 'SQF']! > >Thanks for any help [clickin' it Old School!] >Bytemark I did many of those tests and if you let me know the month and year of the tests you are looking for, I may be able to look it up and send it to you. Arthur Kramer Las Vegas NV Visit my WW II B-26 website at: http://www.coastcomp.com/artkramer
From contax mailing list: From: "Mike Romoff" mike@nostarch.com Date: Thu, 27 Dec 2001 Subject: [Contax] Lens comparisons http://www.cmpsolv.com/cgi-bin/output.cgi Thought you might like to see this interesting lens comparison resource. If list members have good experiences to add (especially on the N mount lenses) I think they need more data. M
From contax mailing list: Date: Tue, 18 Dec 2001 From: Dietmar Goldbeck dietmar.goldbeck@acm.org Subject: Re: [Contax] Contax vs leica, canon, nikon ELIAS LOPEZ MARTINEZ wrote: > Which of these lenses have the best overall perfomance ? > > Contax / Zeiss Planar 50 1.4 mm mount > > Leica summilux 50 1.4?ROM R > if you check the MTF results at http://www.photodo.com/prod/lens you'll see that nothing beats the Planar :-) I also remember reading a test in a german photo magazine, stating, that the Zeiss lens is slightly better and the Leitz much more expensive. Ciao Dietmar -- Alles Gute / best wishes Dietmar Goldbeck E-Mail: dietmar.goldbeck@acm.org
From: "Mike Romoff" mike@nostarch.com From: Contax Mailing List: Date: Thu, 27 Dec 2001 Subject: [Contax] Lens comparisons http://www.cmpsolv.com/cgi-bin/output.cgi Thought you might like to see this interesting lens comparison resource. If list members have good experiences to add (especially on the N mount lenses) I think they need more data. M

From: Serge Boucher sboucher@ulb.ac.be Newsgroups: rec.photo.equipment.35mm Subject: Re: Leica.. Date: Thu, 7 Feb 2002 > > >> >> no, I've never found on the web a seriuos comparative test between > > >> >> nikon (for example) and leica Although it's far from perfect, I've found a french site which tries to fill the blank : http://www.pictchallenge.com/testhl2.html What do you think ? Serge

Newsgroups: rec.photo.equipment.35mm Subject: New Leica M Lens Test... From: nospam@thankyou.com (Simon ALIBERT) Date: Sun, 14 Apr 2002 Hi, I just put online all the lens test (MTF) of the 'M' Line, it's from a french magazine, Hope it will help :-) http://simon.alibert.free.fr/leica/lens/ -- Simon. simonalibert@mac.com

From minolta mailing list: Date: Mon, 29 Apr 2002 From: "ctgardener" ctgardener@yahoo.com Subject: Re: Minolta lens test? --- In Minolta@y..., "schwerpunkt1" schwerpunkt1@i... wrote: > Does anyone know of good websites which do SCIENTIFIC lens tests? > I have only found photodo.com but would be interested in finding > more..... Not exactly, but you can find some old magazine tests (based on mtf at varying focal lengths (for zooms) and apertures at: http://www.par.univie.ac.at/~bob/photo/lenses/ Note that relating to the 500mm lens thread, this site has lens tests that rate the Tamron 300/2.8 higher than the Minolta (at least at f2.8, where I assume you'd use it most of the time) and call it the best 3rd party 300/2.8. Useful background info, though I've found that photodo ratings are a little more consistent with independent opinions I've read here. - Dennis

From minolta mailing list: Date: Mon, 29 Apr 2002 From: ake.axenbom@bigfoot.com Subject: Wanted: A personal lens test method MTF tests is a very good thing. It is a pity that most manufacturers do not present MTF charts and/or ratings of their new - and discontinued - lenses. Photodo.com helps a lot. Personally I also often look at Olle Bjernulf's site which summarises MTF tests in the Swedish magazine FOTO. The site is useful although the rating algorithm is not the same as photodo's. But I want also a reasonably objective method with which I can rate my own lenses, for my personal joy, or as a consumer information prior to selling. I think that it would be possible using the following procedure: 1. You have some kind of standard test picture to shoot at. I think I saw somewhere that there are standard test pictures, but I do not know if they could be improved. Sharpness would be measured by reading the point where a number of converging lines float together on the frame. 2. You scan the negatives. Perhaps it could prove to be justified to use correction factors due to scanning density. 3. You look at the scanned frames in any program allowing you to blow up the scanned frame. as much as needed to read the various variables. When digital SLR-s with full frame becomes available, another option opens up using the same basic procedure. I would be most interested in any facts in this topic. regards, ake

From minolta mailing list: Date: Mon, 29 Apr 2002 From: "aranda1984" stephen@aranda4.com Subject: Re: Minolta lens tests I have seen many photo magazine " lens resolution" tests. I started to compare and purchase lenses based on that alone, which was a mistake. No single lens test can compare two lenses or brands because the variation between individual samples within the same brand can be larger then the variations averaged out between the different brands. Once I saw a comparison between the big 3 Japanese: Nikon, Canon and Minolta lenses. Nikon and Minolta averaged out to be exactly the same while Canon was just slighly ahead of the two. However the statistical differences were so small, that in engineering I would ignore them. / I am talking about quality and not prices./ There is another thing to consider in the comparison, which is the bokeh of the lens. That is something which is totally interpetive and no machine can test nor grade. Many people stated / opinion / that Minolta has a better bokeh then Nikon or Canon. I was told by many that Minolta bokeh is second only to the super expensive German lenses. I stopped looking at these tests and I just rely on my judgement of how the picture looks. If I like what I see, then ... that lens is OK. I have a " cheap MD 200/4 Celtic lens" which takes exceptional quality pictures. Great resolution, overall great quality when it is supposedly a cheap lens. I bought and sold other expensive lenses which produced pictures, not up to my liking. I can't always explain why I do not like a picture, but I always know the ones, in an instant, that are great. My unquestionable opinion grading the lenses I own "The Best", was based on the most scientific approximation and gestimation using only the purest of feelings and emotions. /While trying very hard, not to be a snob./ Other then that, photography being such a subjective art... Good light, ... good inspiration and fresh batteries to all.... Stephen I. Molnar

Date: Mon, 07 May 2001 From: Anders Svensson anders.-.eivor.svensson@swipnet.se Newsgroups: rec.photo.equipment.35mm Subject: Re: Photodo - Some photo magazines & my opinion about them Karri Hartonen wrote: > May I ask where to look for correct MTF test results? > Karri There is no other place than Photodo to look for a large series of consitent tests, done with the same metod (and methodology) and comparing the same set of performance indicators. The *free* site is probably most valuable for buyers of used equipment, as they they have a large selection of tests of older lenses and newer lenses are added after the real customers have used the result. Actually, it's the Photodo site that uses the TIPA tests, not the other way around. It is correct that the method has some flaws, like not testing lenses at small apertures (they stop at f/8 or so) and that some lenses (macros) are tested at infinity instead of their intended focus distance. MTF testing does not say anything about lens flare in varying light, mechanical performance or handling. A common complaint is that the grading is too simplistic (a weighted performance number) but that is a red herring - the basis for the grading is available and the MTF curves are there to look at for those who want to do that (and understand what they say). When people complain about it, they usually do that because a particular lens they own or use seem to be very different than the Photodo verdict (either way). They could have good reasons for that statement, including sample variation or a very different usage profile. Two common examples are the Nikkor 24-120 (bad grade, happy users) and the Cosina 19-35 (good grade, disappointed users). I feel that the majority of results still are mapping very well with user opinions and real world results. Possibly, the problem is that Photodo is free - people sometimes want to pay for advice ;-) Anders Svensson mail: anders.-.eivor.svensson@swipnet.se

From: Tony Polson tony.polson@btinternet.com Newsgroups: rec.photo.equipment.35mm Subject: Re: Is objective lens comparison possible? Date: Sat, 19 May 2001 Anders Svensson anders.-.eivor.svensson@swipnet.se wrote: > Photodo might not be perfect. However, the experts who > dismiss that effort as worthless seem to offer mighty few > alternatives to personal opinon. That's because there are *NO ALTERNATIVES* to personal opinion, based on a thorough and methodical test of the lens in question. And I mean *my* personal opinion about the lens *I* own, because that's the one I use, not the carefully selected review sample, or the one that got dropped by the mail man when delivering it to the reviewer. Anyone else's opinion, and that includes anything written in subjective lens reviews, the subjective overall ratings on the Photodo site, magazine reviews and review sites on the web, is automatically suspect. You only have to look at the conflicting reviews (often diametrically opposite) of similar lens designs to realise that. Anyone who believes otherwise is at the mercy of "lies, damn lies and sample variation". The more discerning buyer might look at many sources of information, including subjective and pseudo-objective tests and reviews, and make a buying decision based on an overall judgement of the reviews he/she has read. But that isn't where it stops; you are then faced with a single example of a lens that may or may not perform as well as the one tested or reviewed by someone else. At that point, all the buyer can do is test the lens they just bought, and return it if it doesn't measure up to their needs. Those buyers who place blind trust in reviews, and buy the reviewed lens without ever questioning whether it's any better or worse than the reviewer said, run the risk of getting one of the lenses the more discerning buyer returned to the dealer. I don't always agree with Bob Monaghan, but he has considerable experience of lens performance and this is one area where his views are right on the mark. -- Tony Polson

From Manual Minolta Mailing List: Date: Thu, 18 Jul 2002 From: Ulrich Olaf olaf.ulrich@siemens.com Subject: Test of 135 mm lenses Hi all, I just came across a 1978 issue of the German photo magazine "Color Foto" where about two dozen 135 mm telephoto lenses were tested. The brands tested include Canon, Konica, Minolta, Nikon, Rollei, Sigma, Soligor, Vivitar, Voigtländer, Yashica, and Zeiss. The winners were: - Minolta MD Tele Rokkor 135 mm f/2.8 (four-element version) - Zeiss Sonnar 135 mm f/2.8 - Zeiss Planar 135 mm f/2 - Nikon Nikkor-AI 135 mm f/3.5 The Nikon Nikkor-AI 135 mm f/2 surprisingly is even better at all apertures at the frame's center but drops significantly towards the edges so the overall rating is closely behind the top four. So is the Nikkor-AI 135 mm f/2.8---it has a more even performance across the frame than the f/2 (worse at the center but better at the edges) but still is slightly behind the f/3.5 version. The best 3rd-party lens tested is the Soligor C/D 135 mm f/2.8. The Vivitar "Series 1" 135 mm f/2.3 offers good sharpness and surprisingly low vignetting even at full aperture but contrast is mediocre, and the lens creates large halations around every highlight even when stopped down. Its overall ranking is some- where in the lower middle of the field. Unfortunately, a Leitz lens was not included in this test. Regards, Olaf -- Olaf Ulrich, Erlangen olaf.ulrich@onlinehome.de