Abuses and Mysteries at ASWB

[Note: a revised version of this article, missing some things presented here and containing some things not presented here, was published on March 3, 2015 in Research on Social Work Practice under the title “Abuses and Mysteries at the Association of Social Work Boards,” and may also be available through Sci-Hub. The accepted version of that publication is available for download from 1 2 sources. At present, Sage appears to offer open HTML and PDF access to the published version.]

* * * * *

As of July 2014, the Association of Social Work Boards (ASWB) administered the national exams that people were required to pass, in order to become licensed to practice social work in most states in the United States and in parts of Canada. At that time, there were hundreds of thousands of social workers working with millions of clients in the U.S., commonly at taxpayer expense. ASWB claimed to be qualified to determine which social work graduates had the requisite knowledge for that work.

Money

ASWB’s social work exams were not cheap: roughly $250 per person per exam. As a monopoly, ASWB had considerable freedom to charge whatever it felt like charging. When asked why the exams had become so expensive, ASWB’s then-Executive Director Donna DeAngelis blamed a recent price hike on “increases in examination development, security, and administration costs.” That was an interesting way to phrase the fact that, according to its IRS Form 990 filings, ASWB increased its net assets by $1 million in 2011. That was a lot of money, but apparently not enough. It seems this allegedly not-for-profit organization wished to return to the $1.5 million annual increase in its net assets that it had achieved the previous year. The price increase did the trick, and then some: net assets rose more than $1.6 million in 2012, leaving ASWB with more than $10 million in, essentially, cash. (Note that ASWB is not the American Red Cross. It is a small organization with a narrow purpose.) The Form 990 indicates that an unspecified amount of that $10 million was invested in, of all things, financial derivatives. ASWB’s total income, provided overwhelmingly from exam fees, rose 61% in just four years, from 2008 to 2012.

In that interview, DeAngelis forgot to mention that her own salary needed to keep rising too, at a rate of about 10% per year. In 2012, DeAngelis took a salary of $303,547. A way to think of that: each year, approximately 15,000 test-takers (i.e., more than the entire number taking the Masters exam) contributed $20 each to her personal paycheck. Nonprofit CEO compensation exceeding $300,000 is atypical in any case: according to Charity Navigator (2014, p. 4), median CEO compensation in an organization of this size, in ASWB’s geographical location, was about $170,000. A report by the Grant Thornton (2012, pp. 12, 18) consulting firm conveyed additional points of comparison — reporting, for example, that New Jersey state regulations cap salaries at $141,000, and Florida at $130,000, for executives at certain state-affiliated nonprofits. That report also noted that median annual chief executive salary increases at nonprofits were around 3% — not 10%. As some have put it, if you want for-profit compensation, go to a for-profit corporation.

In addition, since her salary accounted for about two-thirds of total compensation paid to all of ASWB’s key employees, DeAngelis was presumably getting a fair chunk of the $278,000 that ASWB was paying per year for employee benefits (2012, p. 10). But at yearend 2012, DeAngelis retired to “begin traveling with her husband” (Annual Report, p. 23). Apparently she would retire in comfort: ASWB had been putting another $170,000 per year into its pension plan.

Bear in mind that none of this compensation was for the actual process of developing and administering exams; those functions were budgeted separately. This was just payment for being the queen bee. DeAngelis was undeniably growing the organization’s bottom line. But in terms of the quality of ASWB’s work — its contributions to something other than itself — the following paragraphs demonstrate that this high level of executive pay was inappropriate; it may, in fact, have inadvertently appealed to a kind of chief executive who would value a self-centered accumulation of money over the provision of needed services at reasonable prices.

The outsized compensation paid to people like DeAngelis (other examples: Indiana University School of Social Work Dean Michael Patchner, also paid over $250,000, and the head of NASW, taking $461,919) brought to mind the words of Sowell’s Poverty Pimps’ Poem: “Let us celebrate the poor . . . There’s a market for their pain / Votes and glory and money to gain.” The poor, in this context, might include not only clients but also social work students and many practitioners themselves. ASWB — which had the temerity to define itself as a Public Charity in its Form 990 filings — seemed to be looting people who were trying to make the world a better place in the face of a discouraging job market and difficult working conditions. Under such circumstances, it was rather remarkable to see an article in which DeAngelis (2006, p. 11) expressed interest in whether state social work licensing boards were doing their job by requiring applicants to attest to their own “good moral character.”

Validity

In an interesting study, Albright and Thyer (2010) gave first-year MSW students an ASWB Clinical practice test. According to ASWB, the practice test accurately represented the kinds of questions and answers that test-takers would encounter in the actual Clinical exam. (Apparently the practice test questions were real-life test questions that had previously been used on ASWB exams.)

For these MSW students, Albright and Thyer removed the questions, so that the students could see only the four multiple-choice answers. That is, they had to guess which of the four might be the right answer to a question that was not available to them. Random guesses would have resulted in a correct-guess rate of 25% (i.e., one out of four). But these students did not get 25%. They did much better than that: they averaged 52%. How did they beat the odds? Simple: they were able to guess which answer was the right one, more than half the time, just from the way the answers were worded. In other words, the test did not provide an accurate measure of test-takers’ knowledge. It represented a failure in test design. On this basis, Albright and Thyer (p. 229) concluded that the ASWB Clinical exam “cannot justifiably be claimed to be a valid assessment of competence to practice social work.”

That was embarrassing for ASWB because, at about the same time, DeAngelis was co-authoring an article (Marson, DeAngelis, & Mittal, 2010, pp. 98-99) in which she claimed this:

The central focus of this article has been to illustrate reliability and validity processes, absolutely essential when the successful completion of such an evaluation instrument determines whether or not someone can work in a profession. These very high stakes require careful, responsible, and ethical construction of such instruments. ASWB has consistently shown over the years that it executes its responsibility regarding test construction and administration according to established psychometric theory and practice, and with adherence to social work ethics.

One might be curious about the reliability and validity of such assurances, not to mention the ethics: all three of those authors had conflicts of interest. DeAngelis, as already noted, was ASWB’s executive director; Marson was senior editor at Journal of Social Work Values and Ethics, published by ASWB; and Mittal was employed by ACT Inc., which was deeply involved in ASWB test construction.

This was hardly the first time, before or during DeAngelis’s years at the helm, that ASWB had been warned that its exam processes were badly deficient. Among others, Collins et al. (2002, p. 209, citations omitted) had said this:

Concern has been raised about the validity and reliability of licensing exams. Although the Association of Social Work Boards reports that the exams are valid and reliable, the psychometric properties of these exams are not widely available. Criticism of licensing exams is disturbing. For example, several authors have challenged their content and discriminative validity. Others also argue that written tests for social work licensing fall short in measuring knowledge unique to the profession. For example, one study suggests that passing licensing exams is correlated more with the candidate’s personal therapy experience rather than fieldwork, work experience, graduate or undergraduate education, or clinical training. Many also claim that the exams fail to predict practice competence. Moreover, [one study] found that social work experience, education, and number of credit hours in social work have no significant correlation to examination scores. A further disturbing issue is whether exams discriminate on the basis of race and ethnicity.

Concerns about ASWB exams reached the point of mutiny in the late 1990s, when the state of California defected from the fold, rejecting ASWB exams in favor of a home-grown clinical exam, due to concerns that the ASWB exam was based on sampling unsuited for California’s demographics and that the exam at that point had a pass rate of 89% (Rhine, 2010, p. 109). A review of the situation in 2008 yielded the conclusions that the ASWB Clinical exam was vague and/or weak on important aspects of practice — that it “appears to be a theory-driven examination versus the California LCSW exam which is practice-driven, relying on application and analysis of critical knowledge” (Montez, 2010, p. 113). California’s recent decision to resume using the ASWB exam appears to have been made, not due to any faith in that exam’s superiority, but for reasons related to the state budget and participation in certain federal programs (see Rhine, 2010, p. 108).

As emphasized in the foregoing quote by Marson et al., ASWB’s social work exams are high-stakes in a double-edged sense: they are all-or-nothing for the test-taker, and they also embody enormous claims, by ASWB, regarding its expertise in deciding who qualifies to join the next generation of social workers. Even the most ethical and competent testing organization would find it difficult to live up to such claims, as illustrated by my separate discussion of the use of high-stakes testing in social work practice. As demonstrated in a separate post on the manner in which ASWB develops its tests, ASWB does not in fact live up to such pretensions.

Concealment of Testing Data

In the research just cited, Albright and Thyer (2010, p. 233) concluded that “The professional stakes are too high for anything less than the scientific transparency provided by both internal and independently conducted and published psychometric evaluations of ASWB examinations.” They called upon ASWB to provide full disclosure of the testing data that independent examiners would need, in order to verify that ASWB exams really measured what they were supposed to measure.

Among other things, Albright and Thyer apparently wanted researchers to be able to look at the ASWB data reporting what happened during trial uses of specific questions. This was an obvious need, in the wake of their study: it appeared that the average test-taker would have tended to guess right when answers were worded one way, but might not be able to do so with other experimental wordings. There would be questions as to whether, and how, ASWB performed such testing.

It seemed obvious that ASWB would not need Albright and Thyer to provide basic instruction in the proper construction of standardized tests. ASWB had been at it for a long time. As noted above, ASWB had its own journal; and in 2010, as disclosed on its Form 990 (above), ASWB employed Phillip J. Gullion as its Director of Examinations at a salary of $88,407, along with a half-dozen others pulling down salaries ranging from $85,000 to $128,000. Gullion apparently would have grossed around $125K from ASWB in 2010, had he not left the organization in September, after just 16 months there. (For some reason, there was no Director of Examinations in 2012, and these half-dozen others were reported at that point as receiving zero compensation. ASWB did not provide any information in the Independent Contractors section of the 2012 Form 990. The other post contains the results of further investigation into these matters.)

With people being paid at those levels, you’d be hiring nothing but the best. Not to knock Gullion’s graduate certificate from the University of Massachusetts — Amherst, but a salary of $125,000 for someone two years out of grad school would be hefty even for a PhD in statistics or research methods. It seems something odd was happening there. Be that as it may, ASWB could have consulted people at Pearson VUE, which was handling its statistical issues, as well as any number of statistically trained professors.

So this is not a case where ASWB made a technical error on an esoteric detail. This is a situation where, after many years in business, ASWB was charging thousands of people millions of dollars, each year, to test them with amateurish materials that were grossly unsuited for the task. The problem that Albright and Thyer identified went to the heart of the organization’s business — and as noted by Collins et al. (above) among numerous others, it was a problem that had been around for a long time.

Of course, these experts could have been mistaken. That would have been easy enough to prove: just release the testing data. Consistent with those earlier remarks by Collins et al. (2002), ASWB remained unwilling to do anything of the sort. As of this writing, ASWB’s website still withholds its testing data from researchers. There is a question, then, of whether ASWB develops and retains coherent data of that nature. But as discussed in the middle of that other post, there is actually more to the mystery than that.

Concealment of Ordinary Information

The discussion of that research by Albright and Thyer illustrates a general problem in ASWB’s public communications. A reputable nonprofit organization, possessing tons of money, a boatload of data on social work licensing examinations, and a desire to serve the public, would be pouring out reams of information intended to inform the profession and to improve the licensing of practitioners. There would definitely be a place for advanced researchers, but that sort of expertise wouldn’t be absolutely necessary: anybody who had that data and who knew how to use a spreadsheet could provide enormously helpful guidance on a plethora of issues.

You would know if you were encountering a website like that. There would be a passionate commitment to make sure that members of the public, social work clients, and social work practitioners were not being inappropriately harmed by inadvertent (never mind deliberate) manipulation or withholding of important information.

Unfortunately, I detected no such passion or commitment in ASWB’s website.  It would have been impressive and reassuring to find, for example, that ASWB was indicating how many people took its graduate-level exams across a number of recent years; how many passed in each year; which states and schools they were from; what their undergraduate majors and GRE scores had been; whether people from different regions, sexes, ethnicities, or income levels had scored differently; what kinds of questions seemed to be best and worst suited for relevant purposes; how ASWB’s own exams might have changed from year to year; what had prompted such changes; and so forth. There would be data tables to download, reports to read, forums and discussions. It would be a public service.

Nothing could be farther from what the user encounters on ASWB’s website as of mid-July 2014. There is virtually no information of that sort. What is provided appears to constitute the bare minimum. If ASWB is not actually trying to hide something, it certainly seems to be doing everything it can to give that impression.

The withholding of information is rather extreme. For instance, ASWB’s annual reports do not even state the numbers of people who took its different exams (e.g., Masters, Clinical): the 2012 annual report provides only percentages, and the 2010 annual report did not even do that. As of this writing, ASWB’s website does not report pass rates for any year other than 2013. So older links to ASWB’s reports of pass rates in previous years — even including links found in old issues of ASWB’s own Association News, such as the issue from March/April 2010 (vol. 20, no. 2), referring to 2009 pass rates — now produce an error message: “Sorry, there is no page or article at this location.”

You could look up previous years’ pass rates in old copies of its annual reports but, oops, all annual reports other than the most recent year’s have also been removed from ASWB’s website. (I had previously saved a copy of the 2010 annual report, else I would no longer have access to it.) If you go to the web address where the 2010 annual report used to be located, you get that same error message.

Searches (both via Google and using the ASWB website’s own search box) confirm that previous years’ annual reports are gone. This is unusual. If you go to the websites of other major social work organizations, or do site-specific Google searches (of e.g., NASW, CSWE), you can pull up their prior years’ annual reports. But at ASWB, if you want the kind of information that annual reports contain, you are out of luck, unless you happen to know that it is possible to download the IRS Form 990 filings cited above — and, even then, as noted above, information seems to be missing from those forms as well. It certainly appears that ASWB is trying to discourage comparisons between its present and previous annual reports, and to minimize what one can deduce from any such comparisons.

Yet another example of providing no more than the bare minimum amount of information: consider what happened when I browsed ASWB’s website in search of statements that would distinguish the purposes of its several exams. Among other things, I was not able to find any clear distinction as to content, between the Masters and the Advanced Generalist exams. Bizarrely, ASWB’s About the Exams page said virtually nothing about the exams. Its Exam Content Outlines page did lead to an Advanced Generalist Content Outline – but there, it launched immediately into a long list of topics covered, again lacking any overall statement of the exam’s purpose, or of the concepts that would unite those disparate topics. The closest I could get was a very brief and obvious set of statements about the career phase at which one is allowed to take such exams — indicating, for instance, that the Masters exam was simply “The examination that is intended for individuals who hold an MSW degree, but who do not have post-degree supervision.”

In that example, it appeared that ASWB had discovered that everyone was willing to treat its several different exams as though they did have distinct and important purposes, despite the reality of an inordinately unfocused profession, its practitioners scattered among myriad careers. What, indeed, could be more ridiculous than the search for a specific focus in an “Advanced Generalist” exam? The situation seemed to be, not that ASWB’s exams were rationally developed within rigorous guidelines, but rather that, consistent with the priorities at schools of social work, ASWB was simply trying to provide whatever would keep the money flowing and give the appearance of appropriate professional testing.

Concealment of Schools’ Pass Rates

One aspect of the foregoing discussion deserves additional attention. Somehow, in our corrupt era, it is considered tolerable for ASWB, which holds itself out as a public servant, to discover clear and extreme differences in ASWB exam pass rates, from one MSW program to another — and yet to withhold that information from the public and, more specifically, from future social work students who are trying to decide which program to attend. In a sign of the times, this is just the sort of organization that would be honored by today’s social work profession.

In this regard, Thyer’s (2010) article on LCSW exam pass rates offered a contrast between social work and law. In the legal profession, he pointed out, the average bar exam pass rates achieved by graduates of various schools are openly and directly compared against one another, and schools with especially poor performance records are at risk of closure. In social work, by contrast, Thyer observed that ASWB provides information on a specific school’s ASWB exam pass rate only to officers at that school. Furthermore, such reports are available only at a steep price. A broad comparison across multiple schools would be essentially impossible, unless all such schools publicly shared the information that they might have obtained from ASWB.

Fortunately, Thyer found an exception. Under the sunshine laws of his state of Florida, it was possible to request LCSW pass rates from all Florida MSW programs for the previous several years. (A separate post in this blog provides a start of a compilation of similar data for other states and for individual schools.)

Thyer’s inquiry produced dramatic results. In his data from 2007, for example, pass rates varied from 42% at Barry University to 78% at the University of South Florida (USF). That contrast would obviously steer students toward USF. Then again, rates at USF dropped from that high of 78% in 2007 to a low of 67% in 2008. If that decline continued, there could be a question of whether USF had made a change for the worse in some aspect of its MSW program after 2007. Thyer also observed that Florida programs as a whole did not compare well against the average national pass rate. These sorts of facts and comparisons are potentially valuable in identifying and rectifying problems — but only if the data are made available.

It goes without saying that data can be misused and misconstrued. For instance, it could be, not that Barry University was doing anything wrong, but simply that for some reason it tended to attract inferior students. Indeed, Barry University could get a bum rap for taking a chance on students who would not even have gotten into USF. To quote CSWE (2013), “ASWB exams . . . should not be used as the sole outcome measure for an educational program.”

At the same time, to quote Thyer (2009), “[T]hese issues do not obscure the simple fact that licensure is an exceedingly important influence in the professional practice of our field. One might even characterize it as a driving influence.” The persistently low scores achieved by graduates of the Barry University social work program, in Thyer’s data, are simply not appropriate for suppression by ASWB, when graduates of other programs scored persistently higher. If Barry University has something to explain, it should explain it — not leave potential applicants in the dark.

ASWB’s website does not deign to discuss such matters. It simply continues to assert that its reports on pass rates will be made available, not to the public, but “only to the individual school of social work.” Hence, from the perspective of the typical exam applicant, the validity of social work education and of professional licensing exams are left in the hands of institutions and agencies that collaborate in withholding crucial and potentially life-changing information from the individuals who are paying their salaries.

Ethical Issues

This post has identified a number of regards in which ASWB appears to promote and depend upon unethical behavior. First, there are several large-scale and continuing violations of the NASW (2008Code of Ethics:

  • The Code requires social workers to advance the profession’s knowledge and improve its integrity “through appropriate study and research, active discussion, and responsible criticism” (5.01(b)).
  • The Code requires social workers to “contribute to the knowledge base of social work and share with their colleagues their knowledge” and “to contribute to the profession’s literature” (5.01(d)).
  • The Code requires social workers to “facilitate evaluation and research to contribute to the development of knowledge” (5.02(b)).
  • The Code requires that “Social workers engaged in evaluation or research should be alert to and avoid conflicts of interest” (5.02(o)).

In addition, ASWB’s behavior in these regards appears to violate widely understood principles of research ethics. Those principles, summarized by Resnik (2011), include the following:

  • Strive for honesty in all scientific communications. Honestly report data, results, methods and procedures, and publication status.
  • Disclose personal or financial interests that may affect research.
  • Act with sincerity.
  • Share data, results, ideas, tools, resources. Be open to criticism and new ideas.
  • Help to educate, mentor, and advise students. Promote their welfare and allow them to make their own decisions.
  • Respect your colleagues and treat them fairly.
  • Strive to promote social good and prevent or mitigate social harms through research, public education, and advocacy.

(See also e.g., the American Psychological Association’s (2010Code of Conduct and links provided by Scientists for Global Responsibility.)

Of course, ethical concerns are not limited to ASWB.  The NASW Code makes clear that others who facilitate or condone the occurrence of such unethical behavior are themselves culpable. Among other things, the Code says, social workers elevate service to others above self-interest; they promote ethical practices on the part of the organizations with which they are affiliated; and they take adequate measures to prevent, expose, and correct unethical behavior by other social workers.

The damage done by ASWB’s unethical behavior over the years is potentially huge, and it cannot be undone. As noted above, in an attempt to reduce the damage going forward, in a separate post, I have commenced an effort to identify and share pass rate information that various schools and state authorities have been able to obtain from ASWB.

Conclusion

At this writing, the general situation appeared to be this: ASWB managed to establish itself as the monopolist of social work professional testing; it used that position to charge exorbitant prices of people who could ill afford to pay them, directly enriching its own chief officers; it retained its monopoly position with the apparent indifference or active assistance of state regulators and other social work organizations; it concealed essential information from the public that it claimed to serve; and for many years it ignored reasonable calls for change and improvement.

Taking the scheme as a whole, there appeared to be protections for the schools of social work, protections for ASWB, and protections for the licensed professionals, but little to no protection for students, applicants, clients, and the public. In such regards, ASWB stood as one more illustration of the sharp contrast between the corruption of today’s social work profession and the alternative of genuine social work. It was perhaps not surprising that, in such an ambiance, ASWB would become the provider of licensing exams that appeared to serve the interests of insiders rather than of the public.

Advertisements
Trackbacks are closed, but you can post a comment.

Comments

  • Emma  On April 19, 2016 at 9:52 PM

    I signed up to take the ASWB exam and paid $85 for the practice exam and about $35-$40 for the study guide. Because I had 12 months to take the exam, I thought I had 12 months to use the practice test as well. When I tried to access it, I was unable to. I called and they said my time to take the practice test had expired after 30 days. I said, “So basically you charged me $85 but will not give me anything in return.” They were rude and basically said if I wasn’t smart enough to read the fine print, they couldn’t help me (and there is validity to this argument but it smacks of money-grubbing scammery if you ask me). I paid another $85 to get the exam a second time.

    I took the exam today and got 133/150 (hence I passed). Having now seen the actual test, I can tell you that there is one word that is used over and over on the exam and is almost always (seriously- I would estimate 85% of the time this is the case) in the correct answer (when that word is part of one of the answers- there were many questions where this was not the case). I will not say what that word is for fear of repercussions.

    I work at the VA and if I hadn’t passed this exam, I would be ineligible for promotion. How is this okay? How is this not a monopoly? Very, very frustrating.

  • Ray Woodcock  On February 7, 2016 at 3:24 PM

    On an issue somewhat distinct from the topic of this post, FairTest.org provides a sobering list of problems with Pearson tests, going back years. Recent items on that list include a $5.7 million settlement with Minnesota, due to disruptions in Pearson exams in that state, and a New York State decision to drop Pearson altogether.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: