Reviewing the Reviewers at the Journal of Social Work Education

In spring 2011, I wrote a manuscript on the subject of attrition (i.e., dropping out or being eliminated) in a social work PhD program.  That manuscript appears in a separate post.

I submitted that manuscript to the Journal of Social Work Education (JSWE).  The procedures applied to my manuscript, by that journal’s editor, were interesting in themselves.  I have prepared a separate post on that aspect of the matter.

This was one of three manuscripts on social work education that I prepared and submitted to social work journals in spring 2011.  The two others, discussed in other posts, were on topics related to law and gender.

The present post focuses on the reviews given to my manuscript on PhD student attrition.  The following paragraphs begin with the text of those reviews, followed by a discussion.  As the discussion indicates, the reviews are quite interesting.  In some regards, they are remarkable.

This discussion will probably be of detailed interest to only a small number of individuals.  Preparing it has seemed to be one of those tedious but necessary activities that provides support for more general remarks.  As often happens, the relatively brief remarks in these reviews have required a rather lengthy effort for full rebuttal.

The Reviews

Reviewer 1

The topic of this paper, i.e., attrition within social work doctoral programs, particularly around the time of “qualifying” exams, is an important one, and one which lends itself well to a “researchable question.” However, the author has not chosen to investigate this question in an unbiased, methodologically sound manor [sic].  Rather, the author presents an extensive rationale for why telling his (he identifies himself as the “case” and uses the male pronoun) own story can be viewed as a justified research method.  In addition, the literature that is presented is only one side of a much larger picture about the role of qualifying, comprehensive, preliminary-type exams, and their role in a student’s doctoral experience.  In addition, conclusions are drawn about what may or may not be present in social work programs based on what occurred in disciplines he deems are fields closely aligned to social work, e.g., sociology, without data to support these comparisons.

The notion of looking at attrition in doctoral education from an ecosystemic perspective is an interesting one.  If the author had examined all literature about attrition, at various points in a student’s program, and then created a research project to investigate the model he presented, it could potentially make an interesting contribution to the literature.

Reviewer 2

This manuscript tackles a difficult and unexplored problem in social work higher education:  why so many doctoral students do not finish the PhD.  The author focuses on one phase, the qualilfying [sic] examination.  The literature review is interesting but does not distinguish social work from other doctoral disciplines, so it is difficult to know what is research in social work and what is inference from other disciplines.

The author uses one case example to draw conclusions about doctoral programs:  his own failure in the qualifying examination and his appeals process.  As a lawyer, the author weaves a skillful legal argument, apparently forgetting that a journal is not a court and no one has a chance to rebut his assertions.  Nor is a journal and its audience a court that will make a determination of “right.” The manuscript is essentially a well-written, amusing whine about why the doctoral program was unfair to the candidate.  His extensive attempts to appeal are described in excruciating detail.  There is no substance in the manuscript – we never learn why he was considered not passing – and no perspective on social work.  I agree that there are problems in doctoral education, but this manuscript sheds no light on them.  It is simply an attempt to self-justify failure to understand social work and to embarrass the doctoral program.

Discussion

Apparently the Journal of Social Work Education did not provide a rubric or otherwise structure or guide the review assignment (see Holden et al., 2007. p. 69).  For better and for worse, the reviewers were therefore obliged to rely upon their own knowledge and logic to structure their reviews.  The following paragraphs discuss the objections of each of these two reviewers in turn.

Reviewer 1

The first reviewer seemed to have three major objections and several minor ones, as discussed in the following sections.

First Objection:  Being Unbiased

Reviewer 1 begins with the assertion that my manuscript failed to proceed in an unbiased manner.  This assertion is not clear, and the reviewer unfortunately fails to explain it.

I assume the reviewer meant to be using “bias” in its ordinary dictionary-definition sense.  In that case, his/her complaint appears to be that the manuscript displayed “an inclination of temperament or outlook, especially a personal and sometimes unreasoned judgment; prejudice.”

The manuscript discusses a situation in which a PhD student’s qualifying paper appeared to receive unfair grading.  According to Reviewer 2, the manuscript “weaves a skillful legal argument.”  If that is so, then the manuscript is not biased in the sense of being unreasoning.  It provides a reasoned argument.

JSWE’s own submission guidelines do not require manuscripts to be free of every conceivable form of bias.  They proscribe only “racial, religious, gender, or ethnic bias.”  None of those are relevant to this manuscript.

Reviewer 1 seems to mean that the manuscript is biased, not by being unreasoning, nor by targeting a protected group, but just by displaying an inclination in favor of a certain view.  In that interpretation, the reviewer is suggesting that social work research generally is, and should be, neutral with respect to the subjects it investigates.

Neutrality is not the first word that comes to mind, when surveying social work publications.  Indeed, neutrality is not even a social work ideal.  To the contrary, social workers are expected to advocate against unfairness and abuse, on behalf of people in situations of disadvantage and powerlessness.  Given the manuscript’s specific description of a real situation of abuse, it is hard to see how the reviewer could justify an insistence on neutrality.

There is another problem.  Neutrality is hard to come by.  Indeed, when I saw the reviewer’s remarks, I wondered whether s/he had a PhD.  In my doctoral (even master’s-level) coursework, I have had repeated exposure to the reality that everyone has his/her own perspective.  Bias (more precisely, predisposition) is pervasive.  Not universal — there are things about which we can be relatively disinterested because we don’t have any personal stake in the outcome — but very common.

In this light, what Reviewer 1 appears to suggest is not that I should truly be neutral.  It is, rather, that I should pretend to be neutral.  Instead of going to the heart of the matter, I should adopt a research approach that would disguise my feelings.  I could design a questionnaire, choose interview subjects, or otherwise exert substantial influence over the outcome of my research, as well as the way in which I would write it up (or would perhaps choose not to).  And by thus laundering my preexisting views to make them less visible, I would supposedly be doing superior research.

That definitely is not the approach taken in the manuscript.  The manuscript is based, rather, on the premise that it is better to be honest about one’s relevant experience and preconceptions.  In that way, readers are not lulled; they can see right up-front what is happening, and are better able to make their own evaluations of the available information.

As I read the reviewer’s remarks, I got the sense that s/he may be unfamiliar with qualitative research.  As any doctoral student knows, qualitative research commonly proceeds in circumstances where the urge to avoid bias is unrealistic if not completely wrongheaded.  Qualitative research often involves the use of “personal experience, introspective, life story,” and other “relatively unstructured” sources of information that shed light on “problematic moments and meaning in individuals’ lives” in a way that “includes the subjective experiences of the researcher and the research participant as data” (Anastas, 2004, p. 58, quoting Denzin & Lincoln, 1994, p. 2).

Second Objection:  Using a Methodologically Sound Approach

Reviewer 1 asserts, next, that the manuscript was methodologically unsound.  The rationale for this assertion is that “the author presents an extensive rationale for why . . . [his] own story can be viewed as a justified research method.”

Perhaps a response to this assertion should begin with what the manuscript actually says:

From a lay perspective, this article presents a “case” as set forth in the first general sense identified in the Concise Oxford English Dictionary . . . . It does not, however, present a case from a research method perspective.

Those words come from the first two sentences of the manuscript’s Method section.  It appears that the reviewer’s reading did not carry him/her that far into the discussion of method.

The Method section acknowledges that the manuscript does not present a proper case for purposes of the case study method, and also does not present a case in the professional sense found in courts and clinics.  The Method section specifies that the manuscript is presenting a partisan (i.e., one-sided) case, and admits that such a presentation needs to be read critically.

As the manuscript repeatedly indicates, the reason for the partisan approach is that the other side — the school that has produced the seemingly unfair grading — is not responsive to requests for information or for adjustments that appear conducive to a fairer outcome:

If . . . one party engages in egregious practices, and then withholds information in order to avoid being held accountable for those practices, it might not be ideal to postpone investigation until the party someday changes its mind and makes that information available for use by researchers or other investigators.  On micro, mezzo, and macro levels, social workers, lawyers, and others often engage in advocacy and other interventions, on a partisan basis, to bring about change that seems to be needed, based upon “the best available evidence” as distinct from complete evidence.

The most palpable bias in this matter appears to be that of the reviewer him/herself.  As a social worker, s/he surely would not demand that research into an irresponsible corporation must wait until the corporation voluntarily opens itself to honest dialogue with employees, community members, or other stakeholders.  Social workers quite commonly pry into seats of power and privilege on the basis of incomplete but nonetheless troubling data.

Reviewer 1 is, most likely, a professor of social work.  As such, s/he probably believes in — has quite probably spoken up in defense of — the change-oriented mindset of social work.  And yet it appears, here, that Reviewer 1 thinks that professors of social work, themselves, should be granted immunity from prying, investigation, and change.

This does not necessarily imply that Reviewer 1 is him/herself guilty of the kinds of abuses identified in the manuscript.  But it does remain troubling that any social worker would defend the situation described therein, without recommending any practical remedy or response.  One might fairly wonder whether the reviewer would have reacted similarly if (a) the author had been identified as female or nonwhite or (b) had spoken up about a comparably oppressive situation arising outside the university.

Third Objection:  Using One’s Own Case as Research Method

Reviewer 1 states that the manuscript “presents an extensive rationale for why telling . . . [one’s] own story can be viewed as a justified research method.”  As noted above, the reviewer apparently failed to notice that the Method section denies presenting a case study in the research sense.  By characterizing the entire Method section as an extended elaboration upon a nonexisting claim, the reviewer appears to be underscoring his/her lack of familiarity with the contents of that section.  One might prefer not to see this sort of thing at one of the profession’s leading journals.

There is another problem.  In the remark just quoted, Reviewer 1 objects to the idea that telling one’s own story — no matter how carefully framed or parsed — can ever count as legitimate research.  Here, again, it becomes necessary to try to figure out what the reviewer meant to say.

One hopes that at least the reviewer is not rejecting single-case research.  The well-known research methods textbook by Rubin and Babbie (2005, p. 368) says,

Those who pooh-pooh single-case designs because of their small sample size perhaps overlook the important role they have played in the history of behavioral research, beginning early in the twentieth century with laboratory research on animal behavior.  Who, for example, would want to dismiss the importance of Pavlov’s dog as a single case in the development of learning theory? . . .

As discussed earlier in this book, significant scientific advances do not necessarily require the use of large-scale studies that attempt to verify hypotheses.  Important contributions also can be made by exploratory studies that use more flexible methods, including smaller samples, in efforts to discover new insights and generate hypotheses and theories for which generalizability can be tested later in more tightly controlled studies using larger probability samples.

With a high degree of internal validity, single­case experiments can identify those interventions that seem to work in one, perhaps idiosyncratic, context and can be tested for generalizability in subsequent studies. . . .

Accumulating findings of single-case experiments has value not only in advancing the scientific basis of particular interventions or of a particular practitioner’s effectiveness, but also in evaluating an entire agency or program.

As the manuscript painstakingly explains, it does not purport to provide a full-blown case study.  A research case study outcome would be nice, but it would require cooperation by Indiana University’s School of Social Work (IUSSW), where I was enrolled.  But if a research case study were feasible, it would be in line with these remarks by Rubin and Babbie.  Looking at the last sentence, the focus in the manuscript, and in a potential research study, would be upon evaluation of the qualifying exam phase of IUSSW’s PhD program.

Those remarks raise another problem.  Somehow, Reviewer 1 arrived at the conclusion that an account of a situation involving a large institution, wielding enormous power over a student’s career, is a private story about the individual student.  It is as if a social worker, confronted by a person living in poverty, were to conclude that the person just needs to do a better job of saving money.  That may be true in some cases.  But what we learn, in social work, is that problems often contain multiple dimensions, arising on several levels.  The manuscript does acknowledge that individual student issues can affect graduation rates.  But as the manuscript also points out, that individual level of analysis has been overdone, while analysis of institutional contributions to PhD student attrition have been neglected.  It seems that the reviewer is unwilling to contemplate that change of focus.  S/he seems to insist that the manuscript should remain fixated upon the individual level — and then s/he rejects the manuscript on the theory that it is really just a story about a single individual.

At a certain point, it begins to appear that Reviewer 1 is rather determinedly opposed to the idea that JSWE should report on research that its readers would find highly interesting.  There cannot be much doubt that an article highlighting serious abuses in IU’s school of social work would provoke a great deal of interest among readers in social work and beyond.

Of course, such an article would need to be approached carefully.  As the manuscript demonstrates, I am aware of the difficulties that would attend any such article.  The point here is just that this Reviewer is obviously not looking for creative ways to address the problem, to develop research from it, or in any other way to address what may be a real situation of abuse.  As discussed in an accompanying blog post, the Journal could have taken a variety of approaches to ameliorate those difficulties.  But this reviewer was not remotely inclined to suggest any.

Other Objections

The preceding paragraphs have addressed what were the first, and what appear to be the most significant, criticisms provided by Reviewer 1.  While the reviewer’s other criticisms could be examined in similar detail, that appears unnecessary.  A few brief comments may be sufficient.

One such criticism alleges that the manuscript’s literature review presents “only one side of a much larger picture” regarding qualifying exams.  This is an understatement.  As the manuscript indicates, there are not just two sides.  Research cited in the manuscript points toward widespread disagreement among university professors, spread across numerous theories as to the purpose and effect of the quals requirement, with precious little research to back up those views.  As in all other aspects of both of these reviews, Reviewer 1 does not bother to cite so much as a single source that would have supported her assertions or demonstrated some actual shortcoming in the manuscript.

Another criticism provided by Reviewer 1 is that the manuscript draws conclusions about social work PhD programs based on research in fields like sociology and psychology, “without data to support these comparisons.”  Yet the manuscript states — in precisely the section to which the reviewer alludes — that “Good data on PhD attrition in social work are not presently available.”  In that light, the reviewer’s point is not clear:  is s/he suggesting that I should put this manuscript on hold for some years, until I or someone else has managed to generate satisfactory data?  That section is cautiously phrased; it only suggests that there are grounds for concern that attrition in social work PhD programs may be at least as bad as in these other fields.  The reviewer provides no reason to doubt that.

Reviewer 1 concludes with the opinion that the manuscript should have examined attrition “at various points” in PhD programs.  That would be interesting too.  But why would I drop the focus on attrition at the qualifying phase, when the reviewer’s own first sentence acknowledges the importance of this particular topic?

By this point, the reviewer has repeatedly suggested ways in which a clear, focused investigation could be made less specific, less arresting.  The reviewer seems to think that, at least where schools of social work are concerned, a superior manuscript is one in which nothing too pointed and troubling appears.  That may be consistent with much that is published on the subject of social work education.  But it does not seem to be a route toward identification and resolution of problems.

Reviewer 2

The first paragraph provided by Reviewer 2 agrees that the manuscript addresses an important topic, and states that it could be improved by clarifying which sources pertain specifically to social work as distinct from other disciplines.  That is a good criticism.  It is less ambitious than the related argument advanced by Reviewer 1 (above).  The manuscript could indeed be improved in this regard.  There are limits to what one can hope for, however.  The available research, in the aggregate, may be too limited to warrant much hair-splitting.

The second paragraph provided by Reviewer 2 contains a number of remarks.  It is not clear whether they are intended to drive toward a single clear point.  If so, my reading is faulty; I don’t see it.  As far as I can tell, it is a paragraph without a topic sentence or other internal organization, comprised of an unstructured series of remarks that seem to have been jotted down on the fly.

The Emphasis on Failure

Despite apparent lack of structure, this second paragraph of the review does contain a dominant theme.  The statements in that paragraph refer repeatedly to failure:  “failure in the qualifying exam”; “not passing”; “failure to understand social work.”  It is difficult to tell from such brief remarks, but it seems that the reviewer’s fixation on failure may rest upon a logical fallacy.  Specifically, s/he seems to assume that a failing grade is an unquestionable truth.  But, in fact, the manuscript exists in part to cast doubt upon that grade.  It would not be surprising to find that someone is impatient with a paper that questions a grade, if s/he has already decided that the grade is unquestionable.

This interpretation could help to explain the reviewer’s otherwise baffling complaint, “We never learn why he was considered not passing.”  S/he may just not have read the “excruciating detail” that answers that particular question — such as the paragraphs discussing the fact that the graders themselves failed to explain the failing grade.  If s/he did read it, then s/he would seem to be faulting my paper because it did not provide explanations that do not exist.  And that would make no sense.

Then again, it is not clear why the reviewer wants an explanation of my failure to pass.  As just noted, s/he has already decided that my paper received a failing grade because I failed to understand social work.  Where s/he got that impression is not stated.  Unless s/he had conversations with faculty in my program, it appears s/he is expecting me to obtain information to which I do not have access, and yet is allowing him/herself to throw out random thoughts with no evidentiary support at all.

This is all very puzzling.  What kind of reviewer would just assume that a grade — especially a grade at the doctoral level — would be unquestionable?  The reviewer was, presumably, a social work academic.  Some may feel that this is a commentary on the quality of social work professors generally.  Others may share my concern that, for some reason, I drew someone who entertained a particular antipathy toward one or more aspects of the paper.

How did I wind up with this reviewer?  There is no way of knowing.  The Journal‘s reviewer selection processes were not transparent.  That is, there were no independent checks to verify that appropriate reviewers were assigned to this or to other manuscripts.

There are, in fact, reasons to suspect that my paper did not draw appropriate reviewers.  As described in the accompanying post, the JSWE’s editor and staff handled this manuscript in an unprofessional manner that appeared to depart from their normal procedure.  There was concern that one or more such individuals may have been in contact, regarding me, with faculty in my PhD program.  There were also indications — consistent with the brief and insubstantial nature of this reviewer’s remarks — that this second reviewer may have been selected on a last-minute, off-the-cuff basis.

The question of who this reviewer was, and how s/he was selected, becomes poignant in light of the larger political environment.  In an article entitled “A Call for Transparency,” former JSWE editor Deborah Valentine (2005, p. 4) states that the president of the Council on Social Work Education (CSWE), publisher of JSWE, chooses the people who appear on JSWE’s list of reviewers.  Apparently that choice rests upon political considerations, as in the selection of editors (see the accompanying post).  Needless to say, such individuals would tend to have pro-CSWE views.  But even if the slate of potential reviewers did include one or two who would tolerate criticism of a CSWE member school, it is not clear what, if anything, would keep the editor from assigning them to manuscripts that the editor wished to see in print.

If politics were not enough, there is the question of actual expertise.  Valentine (2005, p. 4) states that the list of reviewers available to review a particular manuscript is based upon “their self-identified areas of expertise.”  That is different from a reviewer’s record of research and publication in such areas.  According to Stoesz and Karger (2009, p. 105), as of 2006, one-third of the consulting editors at JSWE had not published so much as a single article in any of the field’s half-dozen leading journals.  The reviewers of the present manuscript may consider themselves experts on the topic of PhD student attrition at the qualifying stage.  But since hardly any social work professors have published much on it, these reviewers’ apparent inability to cite actual sources in support of their views suggests that they may not really be scholars in this area.

Yet such concerns, having to do with bias and apparent lack of expertise, do not explain why Reviewer 2 would adopt a snide, personally attacking tone.  Bear in mind:  this person is training future social workers.  Hopefully those whom s/he trains do learn, somehow, that gratuitous insults are not compatible with social work ethics.  For example, the Code of Ethics of the National Association of Social Workers states:

Social workers treat each person in a caring and respectful fashion . . . . Social workers should treat colleagues with respect and should represent accurately and fairly the qualifications, views, and obligations of colleagues.  Social workers should avoid unwarranted negative criticism of colleagues in communications with clients or with other professionals.  Unwarranted negative criticism may include demeaning comments that refer to colleagues’ level of competence.

Nor are belittling remarks compatible with best practices in peer reviewing.  For instance, Souder (2011, p. 64) quotes the International Committee of Medical Journal Editors:

In all instances, editors must make an effort to screen discourteous, inaccurate, or libelous statements and should not allow ad hominem arguments intended to discredit opinions or findings.

Above, I took the reviewer’s fixation on “failure” as an indication of an apparent belief that the professor is always right.  But when joined with the review’s characterization of my manuscript as an “amusing whine,” it appears that the reviewer may have been trying to hammer home an impression of my essential unworthiness.

Why would the reviewer do that?  Part of the answer may be simply that s/he can.  People do behave abusively when, as in this case, there are essentially no constraints.  It is possible that this reviewer’s other negative reviews have likewise indulged this kind of language.

Intense bias may be another part of the answer.  It is not inconceivable that Reviewer 2 entertains a personal hatred toward anyone who would criticize any school of social work.  For whatever reasons, such criticism may feel like a personal attack, even when it is directed at a school with which the reviewer has no connection.

Then again, it may not be safe to assume that the reviewer definitely had no connection with Indiana University.  I found it interesting that s/he closed with a reference to “the” doctoral program.  There are different ways of reading that.  It would be rather incredible if the editor decided to send my manuscript to someone on IU’s faculty, or to someone who knew (or who would be told) of my experience at IUSSW.  But the slack composition of the review does suggest the possibility that the reviewer let something slip, there, and did not bother doing a self-edit that might have caught it.  An undisclosed connection with IUSSW’s faculty would certainly explain the reviewer’s personal hostility.

Manuscript and Courtroom

In addition to an emphasis on failure, the remarks provided by Reviewer 2 repeatedly refer to such law-related terms as “unfair,” “skillful legal argument,” and “determination of ‘right.'”  The essential point here seems to be in the statement that “a journal is not a court and no one has a chance to rebut [the author’s] assertions.”  In the eyes of Reviewer 2, the manuscript is fatally flawed insofar as it advances only one side of the story.

In other words, the reviewer does have an answer to a key question presented in the manuscript and quoted above.  The question is, what should a social worker do when one side is all that is available — when there appears to be an abuse, and when the party responsible for it withholds information essential to a balanced presentation, such that the complaining party’s side is the only perspective being presented openly?  The reviewer’s approach is to deny the existence of abuse and to withhold publication until both sides have decided to disclose relevant information.

As noted in the accompanying post regarding the editorial processes surrounding this manuscript, there were ways to address both sides.  They were beyond my power, but they were not beyond the power of the editor.  If information from IUSSW was the barrier to publication, the Journal did have the ability to seek it.  Dedication to the improvement of social work education would seemingly have compelled related inquiries.  A journal may not ordinarily be inclined to engage in such pursuits.  But it does seem reasonable to suggest that a claim of abuse at the pinnacle of social work education — at, that is, the PhD level — is particularly worthy of attention in the Journal of Social Work Education.  At the very least, the possibility should have been given some explicit consideration.

The real-life situation in this case is that people at IUSSW who were in a position to disclose important information failed to do so.  In this sense, the reviewer seems to be advocating on behalf of potentially corrupt arrangements in a wide variety of institutions, where the people with the information necessary to correct a problem are allowed to withhold such information for the sake of personal comfort and advancement.  A position more compatible with social work values would be to highlight potential abuses and press for appropriate disclosures and responses.

In advocating a seemingly unethical approach to reports of abuse, the reviewer is not accurately presenting the approach taken by JSWE itself.  The Journal does not, in fact, require a full and relatively impartial presentation of perspectives on an issue.  There are many examples from articles in JSWE.  Here are a few:

  • Funge (2011) observes that CSWE requires SSWs to provide education in social justice, but that there is no agreement on what social justice is.  Funge does not elaborate on the fact that, as stated by Dover (2009, pp. 507-508) in the context of primary and secondary education, social justice education is often criticized as “unnecessary, ill-suited to the goals of public education, and politically biased.”  I, myself, am inclined to favor various concepts commonly lumped together under the rubric of social justice.  But the point remains that Funge was obliged to proceed with an ideologically skewed discussion due to CSWE bias.
  • Greene (2010, p. 294) recommends that SSWs create “a human behavior course based on information gleaned from a study of Nazi Holocaust survivors.”  According to the rule stated by Reviewer 2, this article should not have been published without a presentation of Nazi views.  Since that would be ludicrous, one may reasonably infer that JSWE sees no problem with publishing a one-sided presentation where there cannot be serious disagreement that the behavior in question was inappropriate or worse.
  • Postmus, McMahon, Warrener, and Macri (2011, p. 303) offer an article pertaining to “survivors of violence,” with no sex specified.  But the article begins with this sentence:  “Millions of women experience physical, sexual, and other forms of violence every year in the United States.”  In a footnote, the authors explain this sex bias:  “Women disproportionately represent victims . . . of violence” (p. 304).  Possibly I am misunderstanding.  The readily available Statistical Abstract of the United States (2012, tables 313 and 316) indicates that there are more than three (for blacks, about five) male homicide victims for every female homicide victim, and that males are also 16% more likely to be victims of other crimes of violence.  In this instance, JSWE published a manuscript with a sexist bias justified by ignorance of the data.  In my case, at least the missing information as to the other side of the story is due to no fault of my own.

That article by Postmus et al. is consistent with the general hostility toward males that permeates social work (see Kosberg 2002).  It appears to be worse than the discrimination that previously existed toward women in schools and workplaces, insofar as it is practiced against male students by some male as well as female faculty.  For present purposes, it is perhaps worth noting that some research has found female reviewers much more likely to reject articles written by males than articles written by females (Lloyd, 1990).

In the several examples just cited, and in many others, the editor and reviewers at JSWE do not seem to have been especially concerned with a balanced presentation.  It does seem reasonable to question the sincerity of their concern for fairness.  That concern appears to be raised when it is convenient — notably, in the present case, to deflect criticism of their own behaviors and affiliations.  Reviewer 2 essentially admits that s/he is interested in protecting the reputation of IUSSW.  His/her last remark is that I sought “to embarrass the doctoral program.”  Is that the test of publishability in social work — that we might embarrass the Nazis, the abusers, or the plutocrats?  If IUSSW has done things whose disclosure would be embarrassing, then (a) it would be shrewd of IUSSW to cover its tracks rather than amplify the issue and (b) it is important to take steps to insure that others are protected from a recurrence, at IUSSW or elsewhere.

Focus on a Particular Case

A third theme in the critique provided by Reviewer 2 appears to be that a focus on a single case is inappropriate or unpublishable.  This is the least coherently presented of the three themes pieced together in this rebuttal.  But it does appear that the reviewer was having at least some passing thoughts along these lines, when s/he suggested that the manuscript “uses one case example to draw conclusions about doctoral programs” and that the manuscript provides “no substance” and “no perspective on social work” and “sheds no light” on problems in social work doctoral education.

If one begins with the premise that a single case cannot provide substance or insight, then of course one would conclude that the manuscript, presenting a single case, is unenlightening.  But it is not clear how to respond to a social work professor who rejects single-case research.  Yin’s (2003, pp. 40-42) widely known book enumerates five major rationales for case study research.  Several could be used in the situation presented in my manuscript.  In particular, the manuscript can be construed as leading toward a “revelatory” case, wherein the researcher “has an opportunity to observe and analyze a phenomenon previously inaccessible to scientific investigation.”  Yin offers the example of important case studies of unemployed men, conducted in the 1960s, where the relevant people and circumstances (like the environment of a social work PhD program) had been around for years, but had not previously been explored in specific terms.

Reviewer 2 apparently believes that readers would have no interest in a manuscript describing egregious abuses by social work faculty.  That seems unlikely.  The situation seems to be, rather, that JSWE suppresses such material precisely because it would be controversial.  The reviewer does not provide examples of previous articles along these lines, with an indication of their lack of impact or any other reason to think they were a waste of pages.

In other professions (e.g., law, medicine, psychology — or, for that matter, history), the experience or condition of one individual very commonly gives rise to large amounts of careful thought and analysis.  Social work itself has long had case-oriented dimensions, and cases have often been used in social work education (see Egan, Combs-Orme, and Neely-Barnes, 2011).  Moreover, the profession borrows from cases in other professions — from, for instance, Supreme Court case decisions, from clinical reports in psychology, and from accounts of various experiences of people like Mohandas Gandhi and Rosa Parks.  Some of the most profound and influential writings of any era appear in anecdotal and even fictional writings (e.g., Anne Frank, Upton Sinclair, Charles Dickens).  One need not even be literate to know that the story of a solitary person against an institution can shed light on the condition being confronted.

Possibly Reviewer 2 objected to a case approach precisely because it was reminiscent of law in particular.  There is almost a sense that my being a lawyer might somehow make my manuscript less publishable.  The lesson from this reviewer’s remarks appears to be the opposite, however.  If one is going to argue on behalf of a certain view, one ought to be able to present evidence and provide a coherent and persuasive interpretation of that evidence.  In other words, I do appreciate that Reviewer 2 felt that my supposedly “legal” argument was “skillful,” but frankly I am afraid I cannot return that particular compliment.

Consider, for example, the claim that the manuscript “uses one case example to draw conclusions about doctoral programs.”  Where are these conclusions?  They do not appear in the Conclusion section of the manuscript.  The statements made there are cautious and limited; they speak only of “some” schools.  Such a claim raises, again, the question of whether the reviewer actually read the manuscript.  There is, for instance, a section that contrasts the doctoral programs at IUSSW and at the University of Washington.  The latter looked fairly good in that comparison.  Plainly, there was no blanket critique of doctoral programs on the basis of a single case.

To recap, there is nothing intrinsically wrong with single-case research.  As stated by Gustafson, Nassar, and Waddell (2010, p. 18), “Single-case designs have a long, well-accepted history in scientific disciplines.”  Such research may be the best and/or only option in many situations worth studying and reporting.  Other professions use it with considerable effect.  This particular case is valuable.  Its publication would stimulate much discussion.  It could contribute to the badly needed reform of social work education.

In this regard, the argument against the manuscript appears to be that JSWE simply does not publish single-case research.  This will appear to be true if one searches the Journal for law- or medicine-style articles focusing on the travails of a single individual.  As Yin (2003, pp.22-23) indicates, a case may indeed use the individual as its unit of analysis.  But other things can also comprise a case, for purposes of case study.  Examples include events, decisions, programs, processes, institutions, and entire industries.  Rubin and Babbie  (2005, pp. 132-134) likewise point out that the unit of analysis for research, be it case study or otherwise, may be a group, a collection of social objects, or a type of social interaction.

With this understanding, it becomes clear that the pages of JSWE are littered with single-case studies.  They may not look like case studies because they involve sampling a number of people, sometimes with the aid of quantitative methods.  But for purposes of generalizability across multiple settings, a study using quantitative methods to identify apparent facts about a single situation is comparable to a study that would use quantitative methods to identify aspects of one person’s thought processes.  That is, such research is not valuable for providing generalizable insights applicable across a variety of other cases, though it may be very useful for providing ideas.  Here are several examples from two recent issues of JSWE:

  • Averett, Carawan, and Burroughs (2012) studied an admittedly unusual field placement setting.  That is, the setting was the unit of analysis.  Individual student views were polled to inform the impression of that setting.  The authors admit that the study has limited generalizability beyond that single case.
  • Wiechelt and Okundaye (2012) sought to evaluate a certain course module for its effectiveness in achieving certain outcomes.  They present their findings in generalized terms.  For example, they say that “the module . . . is effective at increasing HBSE students’ knowledge” and that “faculty who teach HBSE and practice courses would likely find this module to be useful.”  Yet these conclusions are based only upon the study of students within a single program.  There is no way of knowing, from this study, whether different instructors in a different school, teaching students of a predominantly different race, culture, or socioeconomic status, would achieve remotely similar outcomes.
  • Grady, Powers, Despard, and Naylor (2011) sought to develop an instrument to measure factors within what CSWE calls the “implicit curriculum” (i.e., the learning environment within which students take courses).  To assist in this process, they conducted a preliminary study of students within one MSW program.  They do not claim to have reached generalizable findings.  To the contrary, they appropriately describe their work as “a beginning step.”

Ultimately, the concern with single-case studies is a red herring.  JSWE often publishes articles that are not research pieces at all, but are rather presentations of theory, individual experience, or opinion.  In this light, a manuscript (like mine) that presents a single case may enjoy a stronger empirical base than many other pieces that do get published.  Readers can examine and discuss the particulars supporting various conclusions in a manuscript of the type that I submitted, whereas it is usually infeasible to suggest that someone’s experience has not actually been what they claim.

Conclusion

I submitted a good manuscript to the Journal of Social Work Education.  It drew bad reviewers.  Due to political precommitments, that may have been all that the Journal could offer.  My manuscript was probably doomed at that journal in any case — mine, and an unknown number of others that have likewise identified problems at one or more CSWE member schools.

Some of the reviewers’ remarks are incredible.  The reviews contain several indications that the reviewers — social work professors, most likely, with responsibilities for grading their own students’ papers — did not even read whole sections of the manuscript.  It appears that the conclusion to reject the manuscript was reached without regard to what it said.  It is not clear whether the editor or someone else on the staff of the Journal conveyed any such predisposition to either of the reviewers.  For whatever reason, there is an impression, especially but not only in the case of the second reviewer, that the review was dashed off in rather scattered fashion, as if while multitasking.

Failings in journal peer review are nothing new.  The odds of two irresponsible reviews on a single manuscript are hopefully not high.  The particular content and context of this manuscript present a substantial likelihood, supported by the reviewers’ own words, that it was rejected because it said things that CSWE, the American social work accrediting body, did not wish to see in print.  Whether these circumstances facilitated unethical direct communications between JSWE and IUSSW remains unknown.

Advertisements
Trackbacks are closed, but you can post a comment.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: