Mr. Palij writes:
>In the course of reviewing the PC magazine article that
>Ms. Czyz referred to in her earlier posting to this list
>I came across a couple of inaccuracies that I thought
>should be made known.
Mr. Palij says:
>Ms Czyz misrepresents the ratings given in the PC Magazine
>article. First, each package was rated on three dimensions
>(Czyz statement implies there was one rating).
Systat certainly seemed to believe it came out on top. It has referred
to this Editor's Choice rating several times in its advertising
and promotional materials.
I did not keep a copy of the PC Magazine review (for reasons that will
become apparent later in this posting). It is my distinct recollection that
1. There was an overall rating (Editor's Choice?).
2. Systat and SPSS were co-winners.
So it appears Mr. Palij is engaged in a hair-splitting exercise here.
I cannot see how anyone might be misled by her comment.
While we're on the subject of accuracy, consider the following.
I have been led to believe, by comments in PC Magazine, that
Editor's Choice is a dichotomous award. You receive it or you do
not.
Yet Mr. Palij wrote:
>SPSS/WIN had the highest ratings
>and was presented first in the Editor's Choice column.
This appears to imply a rank ordering on both the "ratings" and the
Editor's Choice. Yet I assume SPSS was "presented first"
because it comes before Systat in alphabetical orderings.
Mr. Palij's statement might be considered misleading.
Perhaps he could clarify.
Mr. Palij has presented us with some interesting additional information. I
do not recall this information appearing in the original PC Mag
article, so it would be interesting to know his source. If true,
it certainly provides some food for thought.
Mr. Palij wrote:
>Systat was reviewed by Sheryl Canter and Charles
>Kadushin who, as far as I know, have no relationship to
>Mr. Blank. .......Canter and Kadushin also
>did the reviewing of the three SPSS packages reviewed.
>SPSS/WIN was also presented before Systat in the Editor's
>Choice column.
If Mr. Palij is correct, PC Magazine had one pair of reviewers review
SPSS and Systat, and an entirely different reviewer review Statistica.
The table below shows the experimental design and outcome.
------------------------------------------------------------------
Package SPSS Systat Statistica
------------------------------------------------------------------
Reviewer(s) Canter Canter Blank
Kadushin Kadushin
------------------------------------------------------------------
Rating High High Not so high
------------------------------------------------------------------
Perhaps Mr. Palij could clarify for us if he really feels this is an
appropriate way to design an objective review. I suggest we present
the table to a class of third year Psychology undergraduates in a
research methods class, to see if they find any flaws.
Yet Palij seems to feel there is no reason for Czyz to complain.
>Wouldn't one have to posit a conspiracy
>theory so that (a) Statistica would be rated poorly and
>(b) Systat be rated highly?
No conspiracy theory is necessary. Simple knowledge of the
principles of experimental design might suffice. Things like "rater
effects" come to mind immediately..... One does not have to posit
bad faith, or a bad review, on Grant Blank's part.
The simple point (apparently missed by Mr. Palij) is that the review
was not done correctly (according to elementary principles of
experimental design), AND Mr. Blank had assisted in the design of
part of Systat. Since Systat was involved in a public quarrel with
Statsoft, that MIGHT have compromised his objectivity toward Statistica.
Yet StatSoft had not contested the result.
These facts were posted by Maria Czyz in response to comments by
Alan Best (which I will reprint for below, in case the reader
is unaware of their tone and content).
Best had written:
>From: Best@Ruby.VCU.EDU (Al Best)
>Date: Tue, 23 Aug 94 12:18:14 GMT
>They lie....
>They make the market "smell".
>Statistica is the "used car salesman" of the stat marketplace.
>This is my last post on this subject.
Later he had added innuendo to the effect that
StatSoft was trying to influence and suppress reviews
through legal intimidation. :
>From: Best@Ruby.VCU.EDU (Al Best)
>Date: Thu, 1 Sep 94 13:00:40 GMT
>If there has been no review, one wonders why.
>I have heard through the grapevine of reviewers and
>editors that StatSoft has threatened legal action
>against publications considering such reviews because
>of negative comments that would appear in
>a review.
Does Mr. Palij feel that such postings are appropriate?
How would he feel if he were targeted by such materials?
Maria Czyz's posting had two purposes
(1) It challenged Best to support his allegations with facts,
(2) It presented a case where of a situation
where StatSoft might well have complained about an
unfair review, but did not.
Rather than damaging Czyz's case, Mr. Palij adds further support.
Specifically, if PC magazine had randomly assigned different reviewers
to different packages, there might be some faint claim that the
review is useful. Thanks to Mr. Palij, we find that, in fact, the
design of the review was unbalanced in an extremely interesting way.
Mr. Palij concluded:
>The above comments are misleading and should be retracted
>by Statsoft. Specifically, SPSS/WIN had the highest ratings
>and was presented first in the Editor's Choice column.
As I pointed out above, the ironic fact is that this comment is
misleading. It implies a hierarchy in a dichotomous category.
I conclude something quite different.
I believe that, unless Mr. Best can substantiate his "grapevine,"
it is he who should be doing the retracting. Moreover, I would urge
all participants to refrain from posting defamatory statements about
individuals or organizations on this forum. If Mr. Best wants to write
to all his friends and tell them his opinions, he may do so. This
forum should be reserved for interesting technical discussion.
Mr. Palij's posting has proven useful. I appreciate his providing us
with additional information about the PC Magazine review.
James H. Steiger (steiger@unixg.ubc.ca)
Dept. of Psychology
University of British Columbia
Vancouver, B.C., Canada V6T 1Z4