Designing Rubrics for Assessing Higher Order Thinking
by William Peirce

This is the text version of a workshop presented at AFACCT
Howard Community College
Columbia, MD, on January 13, 2006.

Introduction

Professors who teach thinking skills such as arguing, analyzing, synthesizing, drawing conclusions, solving problems, making decisions, and evaluating need to know how well their students can use these skills. Using rubrics that describe several different levels of student performance
Rubrics can be used to evaluate programs, courses, and individual student assignments and projects. For example, to assess student thinking in a multi-section course, faculty would assign the same task requiring thinking to all students (essays, projects, performances, portfolios, etc.), and normed raters would score a random sample of student work using rubrics. Unless multiple choice questions are designed very well and ask about a novel situation, multiple choice tests are not good indicators of critical thinking because they ask for recall of thinking described in the lectures or textbook. More information and advice about assessing courses at PGCC can be found in the Course Assessment Handbook. Advice on designing writing assignments that promote thinking can be found in an article on the RAC web site, along with a good many other articles on teaching thinking.

After describing some content-free critical thinking tests as a result of the Delphi project, this article describes  holistic and analytical rubrics developed by various practitioners, ending with examples that provide useful feedback on student thinking without using rubrics.

The Delphi Project of the American Philosophical Association

In the late 1980's a subcommittee of the the American Philosophical Association assembled 46 experts to reach consensus on a definition of critical thinking and how to assess it. The result is the Delphi Report (Facione, 1990); the Executive Summary is available online. It defined critical thinking as having two dimensions: cognitive skills and affective dispositions. The cognitive skills the group included are skills and subskills in
  1. interpretation; subskills:
  2. analysis; subskills:
  3. evaluation; subskills:
  4. inference; subskills:
  5. explanation; subskills:
  6. self-regulation; subskills:
Affective dispositions to critical thinking were divided into two categories (1) approaches to life and living in general, and include such traits as inquisitiveness, trust in reason, and fairmindedness, and (2) approaches to specific issues, questions or problems, and include such traits as clarity in stating the question, diligence in seeking relevant information, and persistence although difficulties are encountered.

As a result of the Delphi project, Facione and his associates developed the California Critical Thinking Skills Test (CCTST), a 34-item multiple-choice test in three versions, and the California Critical Thinking Dispositions Inventory (CCTDI), 75 "agree-disagree" items.

Content-Free Critical Thinking Tests to Assess Programs and Courses

Several commercially available tests attempt to assess critical thinking in a content-free way; that is, they do not assess thinking in nursing or biology or business management courses but instead assess the student's recognition of the use of evidence to support a claim, the validity of reasoning, logical fallacies, soundness of interpretations, drawing conclusions, and the like. A review of critical thinking tests can be found at the web site of the National Postsecondary Education Cooperative (US Department of Education) at http://nces.ed.gov/pubs2000/2000195.pdf. Among the more widely used are the Watson-Glaser Critical Thinking Appraisal and several different tests available from Insight Assessment, such as the CCTST and the CCTDI. Often such tests are used by departments to assess whether their programs or courses have improved students' critical thinking. Departments typically use the A version as a pre-test before students begin the program or course and the B version as a post-test. For example, Van Gelder, Bissett, and Cumming (2004) used the CCTST to see if deliberate practice in informal reasoning using the Reason!Able software program improved university students' critical thinking (it did);  McCarthy, Schuster, Zehr, and McDougal (1999) used both the CCTST and CCTDI to determine whether nursing students' critical thinking improved during a baccalaureate nursing program (it did); and Nokes, Nickitas, Keida, and Neville (2005)  used the CCTDI to determine whether a 15-week service-learning intervention improved critical thinking (it didn't; scores were lower after 15 weeks) .

Critical thinking occurs in the context of a course, so there is a a trend for developing context-specific thinking tests. For example, the National League for Nursing has developed tests to measure critical thinking applied to nursing situations for RNs and LPNs. Insight Assessment has a test that measures reasoning in the health sciences. 

Holistic Critical Thinking Scoring Rubric

Peter Facione and Noreen Facione have developed the four-level Holistic Critical Thinking Scoring Rubric to assess the critical thinking skills and some of the dispositions identified by the Delphi project as these skills are demonstrated by by students in essays, projects, presentations, clinical practices, and such. The Facione and Facione Holistic Scoring Rubric (1994) is copied below and is available free, with a page of instructions, at http://www.insightassessment.com/pdf_files/rubric.pdf

4  Consistently does all or almost all of the following:
Accurately interprets evidence, statements, graphics, questions, etc.
Identifies the salient arguments (reasons and claims) pro and con.
Thoughtfully analyzes and evaluates major alternative points of view.
Draws warranted, judicious, non-fallacious conclusions.
Justifies key results and procedures, explains assumptions and reasons.
Fair-mindedly follows where evidence and reasons lead.
3  Does most or many of the following:
Accurately interprets evidence, statements, graphics, questions, etc.
Identifies relevant arguments (reasons and claims) pro and con.
Offers analyses and evaluations of obvious alternative points of view.
Justifies some results or procedures, explains reasons.
Fairmindedly follows where evidence and reasons lead.
2  Does most or many of the following:
Misinterprets evidence, statements, graphics, questions, etc.
Fails to identify strong, relevant counter-arguments.
Ignores or superficially evaluates obvious alternative points of view.
Justifies few results or procedures, seldom explains reasons.
Regardless of the evidence or reasons maintains or defends views based on self-interest or preconceptions.
1  Consistently does all or almost all of the following:
Offers biased interpretations of evidence, statements, graphics, questions, information, or the points of view of others.
Fails to identify or hastily dismisses strong, relevant counter-arguments.
Ignores or superficially evaluates obvious alternative points of view
Argues using fallacious or irrelevant reasons, and unwarranted claims.
Regardless of the evidence or reasons, maintains or defends views based on  self-interest or preconceptions.
Exhibits close-mindedness or hostility to reason.

Analytical Critical Thinking Scoring Rubrics

Analytical rubrics provide more information than holistic rubrics. The holistic rubric illustrated above combines five different kinds of thinking into a single category. Instead of the holistic rubric's lumping of several different traits into one category, an analytical rubric separates them.  Although they take more time to score because the raters sometimes have to examine the essay, project, or performance more than once, analytical rubrics can be useful to departments assessing student's thinking skills in assignments and projects in multi-section courses to determine which areas of student thinking need more attention in the course.

WSU Critical Thinking Rubric

The Washington State University Critical Thinking Project has produced an analytical rubric that assesses seven thinking skills. The WSU rubric specifies only the highest and lowest levels of performances, leaving it to faculty adapting it to describe the intervening levels.
WSU Critical Thinking Rubric

1) Identifies and summarizes the problem/question at issue (and/or the source's position).

Emerging

Mastering

Does not identify and summarize the problem, is confused or identifies a different and inappropriate problem.

Does not identify or is confused by the issue, or represents the issue inaccurately.
 

Identifies the main problem and subsidiary, embedded, or implicit aspects of the problem, and identifies them clearly, addressing their relationships to each other.

Identifies not only the basics of the issue, but recognizes nuances of the issue.

2) Identifies and presents the STUDENT'S OWN hypothesis, perspective and position as it is important to the analysis of the issue.

Emerging

Mastering

Addresses a single source or view of the argument and fails to clarify the established or presented position relative to one's own. Fails to establish other critical distinctions.

Identifies, appropriately, one's own position on the issue, drawing support from experience, and information not available from assigned sources.

3) Identifies and considers OTHER salient perspectives and positions that are important to the analysis.

Emerging

Mastering

Deals only with a single perspective and fails to discuss other possible perspectives, especially those salient to the issue.

Addresses perspectives noted previously, and additional diverse perspectives drawn from outside information.

4) Identifies and assesses the key assumptions.

Emerging

Mastering

Does not surface the assumptions and ethical issues that underlie the issue, or does so superficially.

Identifies and questions the validity of the assumptions and addresses the ethical dimensions that underlie the issue.

5) Identifies and assesses the quality of supporting data/evidence and provides additional data/evidence related to the issue.

Emerging

Mastering

Merely repeats information provided, taking it as truth, or denies evidence without adequate justification. Confuses associations and correlations with cause and effect.

Does not distinguish between fact, opinion, and value judgments.


 

Examines the evidence and source of evidence; questions its accuracy, precision, relevance, completeness.

Observes cause and effect and addresses existing or potential consequences.

Clearly distinguishes between fact, opinion, & acknowledges value judgments.

6) Identifies and considers the influence of the context* on the issue.

Emerging

Mastering

Discusses the problem only in egocentric or sociocentric terms.

Does not present the problem as having connections to other contexts--cultural, political, etc.

Analyzes the issue with a clear sense of scope and context, including an assessment of the audience of the analysis.

Considers other pertinent contexts.

7) Identifies and assesses conclusions, implications and consequences.

Emerging

Mastering

Fails to identify conclusions, implications, and consequences of the issue or the key relationships between the other elements of the problem, such as context, implications, assumptions, or data and evidence.

Identifies and discusses conclusions, implications, and consequences considering context, assumptions, data, and evidence.

Objectively reflects upon the their own assertions.

Contexts for Consideration

  1. Cultural/Social
    Group, national, ethnic behavior/attitude
  2. Scientific
  3. Conceptual, basic science, scientific method
  4. Educational
    Schooling, formal training
  5. Economic
    Trade, business concerns costs
  6. Technological
    Applied science, engineering
  7. Ethical
    Values
  8. Political
    Organizational or governmental
  9. Personal Experience
    Personal observation, informal character

Source: Washington State University Critical Thinking Project Critical Thinking Rubric

The WSU critical thinking rubric has been adapted by faculty in a variety of ways, some illustrated at the WSU Critical Project web site in its 50-page Resource Guide.

The WSU Rubric Adapted to Assess the General Education Outcome in Critical Thinking

Mid-South Community College (West Memphis, AR) has designed a four-level variation of the WSU rubric to assess its general education outcome number 7 - Apply critical thinking skills to solve problems, make informed decisions, and interpret events.

Critical Thinking Rubric
GEO #7 - Apply critical thinking skills to solve problems, make informed decisions, and interpret events.

Rubric Component 

4 3 2 1
Identifies and summarizes the problem/question at issue. Accurately identifies the
problem/question and provides a well-developed
summary. 		Accurately identifies the
problem/question and provides a brief summary.		 Identifies the problem/
question and provides a poor summary or identifies an inappropriate problem/question. 		Does not identify or
summarize the problem/question accurately if at all.
Identifies and assesses the quality of supporting data/evidence Provides a well-developed
examination of the evidence and questions its accuracy, relevance, and completeness. Clearly distinguishes between fact and opinion.		 Examines evidence and
questions the quality. Distinguishes between
fact and opinion.

Merely repeats information
provided. Does not justify position or distinguish between fact and opinion.

 Does not identify or
assess the quality of supporting evidence.
Identifies and considers the influence of the context* on the  issue

Accurately identifies and
 provides a well-developed explanation of contextual issues with a clear sense of scope.

 Accurately identifies
and provides an explanation of potential contextual issues.		 Does not explain
contextual issues;
provides inaccurate information; or merely provides a list.

Does not identify or
consider any contextual issues.
Demonstrates higher level
thinking by interpreting the author’s meaning or the potential bias

Accurately identifies the author’s meaning and/or potential bias and provides a well-developed
explanation.

Accurately identifies meaning and/or bias and provides a brief explanation.

Does not explain, provides inaccurate information, or merely lists potential bias or inferred meanings.
Identifies and evaluates
conclusions, implications, and consequences

 

 Accurately identifies conclusions, implications, and consequences with a well-developed explanation. Provides an objective reflection of own assertions. Accurately identifies conclusions, implications, and consequences with a brief evaluative summary. Does not explain, provides inaccurate information, or merely provides a list of ideas; or only discusses one area. Does not identify or evaluate any conclusions, implications or consequences.
      * Context may include cultural/social, scientific, educational, economic, technological, ethical, political, and personal experience issues.

The WSU Rubric Applied to a Single Course


At Brenau University the WSU rubric has been adapted to a course in women and the law by retaining the description of the lowest and highest levels and inserting a 5-position scale (presumably corresponding to a grade of A, B, C, D, F).

The Critical Thinking Rubric For LE 495, Women & The Law
Critical thinking and writing are skills essential to effective communication, problem solving, and analysis in the humanities, and, indeed, to any profession and field of study. To attain these abilities requires ongoing practice and critical review by peers, mentors, and perhaps most importantly, yourself. The forum and format of your work at Brenau University that can be evaluated for critical thinking skills can vary widely, ranging from informal dialogues to formal, graded research projects. In any event, the same fundamental principles will assist you to create tighter, better reasoned, and more compelling analyses and arguments. This rubric represents a brief overview of the main points to bear in mind as you prepare one of the topics for your written topic presentations and classroom oral presentations. A written topic presentation which shows critical thinking includes the following seven criteria as it--

1) Identifies and summarizes the problem/question at issue (and/or the source's position).

1------------------------------2----------------------------------3---------------------------------4----------------------------------5

Scant

                                                       Substantially Developed

Does not identify and summarize the problem, is confused or identifies a different and inappropriate problem.

Does not identify or is confused by the issue, or represents the issue inaccurately.

Identifies the main problem and subsidiary, embedded, or implicit aspects of the problem, and identifies them clearly, addressing their relationships to each other.

Identifies not only the basics of the issue, but recognizes nuances of the issue.

2) Identifies and presents the STUDENT'S OWN perspective and position as it is important to the analysis of the issue.

1------------------------------2----------------------------------3---------------------------------4----------------------------------5

Scant

                                                       Substantially Developed

Addresses a single source or view of the argument and fails to clarify the established or presented position relative to one's own. Fails to establish other critical distinctions.

Identifies, appropriately, one's own position on the issue, drawing support from experience, and information not available from assigned sources.

3) Identifies and considers OTHER salient perspectives and positions that are important to the analysis of the issue.

1------------------------------2----------------------------------3---------------------------------4----------------------------------5

Scant

                                                       Substantially Developed

Deals only with a single perspective and fails to discuss other possible perspectives, especially those salient to the issue.

Addresses perspectives noted previously, and additional diverse perspectives drawn from outside information.

4) Identifies and assesses the key assumptions.

1------------------------------2----------------------------------3---------------------------------4----------------------------------5

Scant

                                                       Substantially Developed

Does not surface the assumptions and ethical issues that underlie the issue, or does so superficially.

Identifies and questions the validity of the assumptions and addresses the ethical dimensions that underlie the issue.

5) Identifies and assesses the quality of supporting data/evidence and provides additional data/evidence related to the issue.

1------------------------------2----------------------------------3---------------------------------4----------------------------------5

Scant

                                                       Substantially Developed

Merely repeats information provided, taking it as truth, or denies evidence without adequate justification. Confuses associations and correlations with cause and effect.

Does not distinguish between fact, opinion, and value judgments.

Examines the evidence and source of evidence; questions its accuracy, precision, relevance, completeness.

Observes cause and effect and addresses existing or potential consequences.

Clearly distinguishes between fact, opinion, & acknowledges value judgments.

6) Identifies and considers the influence of the context * on the issue.

 1------------------------------2----------------------------------3---------------------------------4----------------------------------5

Scant

                                                       Substantially Developed

Discusses the problem only in egocentric or sociocentric terms.

Does not present the problem as having connections to other contexts--cultural, political, etc.

Analyzes the issue with a clear sense of scope and context, including an assessment of the audience of the analysis.

Considers other pertinent contexts.

7) Identifies and assesses conclusions, implications and consequences.

1------------------------------2----------------------------------3---------------------------------4----------------------------------5

Scant

                                                       Substantially Developed

Fails to identify conclusions, implications, and consequences of the issue or the key relationships between the other elements of the problem, such as context, implications, assumptions, or data and evidence.

Identifies and discusses conclusions, implications, and consequences considering context, assumptions, data, and evidence.

Objectively reflects upon the their own assertions
    *Contexts for Consideration
    1. Cultural/Social
      Group, national, ethnic behavior/attitude
    2. Scientific
      Conceptual, basic science, scientific method
    3. Educational
      Schooling, formal training
    4. Economic
      Trade, business concerns costs
    5. Technological
      Applied science, engineering
    6. Ethical
      Values
    7. Political
      Organizational or governmental
    8. Personal Experience
      Personal observation, informal character
      9.
Source: http://intranet.brenau.edu/assessment/content/ct/default.asp. Click on "Frank Adaptation."

An Example of a Generic Analytical Rubric Using Terms from Bloom's Taxonomy

Faculty comfortable with Bloom's taxonomy of the cognitive domain might prefer to use or modify this rubric from North Hennepin Community College to evaluate the thinking displayed by students in their essays, projects, presentations, performances, portfolios, and other tasks:

CRITICAL THINKING RUBRICS

Based on a draft from Elaina Bleifield and the Paulus CT Group

CATEGORY ONE:  KNOWLEDGE AND COMPREHENSION (understanding the basics)
4—The work consistently demonstrates clear, accurate, detailed and comprehensive understanding of the relevant facts / data / theories/ terms as well as the ability to organize the information for application, presentation, documentation, and/or further examination.
3--The work demonstrates an adequate understanding of the relevant facts / data / theories/ terms as well as the ability to organize the information for application, presentation, documentation, and/or further examination
2-- The work demonstrates an uneven and shaky understanding of the relevant facts / data / theories/ terms as well as a limited ability to organize the information for application, presentation, documentation, and/or further examination.
1-- The work demonstrates an inadequate understanding of the relevant facts / data / theories/ terms as well as a limited ability to organize the information for application, presentation, documentation, and/or further examination.

CATEGORY TWO:  APPLICATION AND ANALYSIS  (attaining the concept)
4—The work demonstrates confident ability to work with the key concepts / information / process / theory  -- applying or extending them to a wide variety of new problems  or contexts, making predictions, recognizing hidden meanings, drawing inferences, analyzing patterns and component parts, communicating insightful contrasts and comparisons.
3--The work demonstrates adequate ability to work with the key concepts / information / process / theory -- applying or extending them to a variety of new problems  or contexts, making predictions, recognizing hidden meanings, drawing inferences, analyzing patterns and component parts, communicating insightful contrasts and comparisons.
2-- The work demonstrates uneven and shaky ability to work with the key concepts / information / process / theory -- applying or extending them with mixed success to new problems  or contexts, making predictions, recognizing hidden meanings, drawing inferences, analyzing patterns and component parts, communicating insightful contrasts and comparisons.
1-- The work demonstrates extremely limited ability to work with the key concepts / information / process / theory -- applying or extending them with very limited success to new problems  or contexts, making predictions, recognizing hidden meanings, drawing inferences, analyzing patterns and component parts, communicating insightful contrasts and comparisons.

CATEGORY THREE:  SYNTHESIZING AND EVALUATING (going beyond the given)
4—The work demonstrates surprising/insightful ability to take ideas / theories / processes / principles further into new territory, broader generalizations, hidden meanings and implications as well – as well as to assess discriminatively the value, credibility and power of these ideas (etc.) in order to decide on well-considered choices and opinions.
3-- The work demonstrates adequate ability to take ideas / theories / processes / principles further into new territory, broader generalizations, hidden meanings and implications as well – as well as to assess discriminatively the value, credibility and power of these ideas (etc.) in order to decide on well-considered choices and opinions.
2-- The work demonstrates uneven and superficial ability to take ideas / theories / processes / principles further into new territory, broader generalizations, hidden meanings and implications as well – as well as a limited ability to assess discriminatively the value, credibility and power of these ideas (etc.) in order to decide on well-considered choices and opinions.
1-- The work demonstrates little ability to take ideas / theories / processes / principles further into new territory, broader generalizations, hidden meanings and implications as well – as well as a limited and superficial ability to assess discriminatively the value, credibility and power of these ideas (etc.) in order to decide on well-considered choices and opinions.
Source: http://www.nh.cc.mn.us/mydocs/1000007/ThinkRubric.doc

Using Primary Traits to Assess Thinking in Individual Assignments

So far in this presentation, the descriptions of different kinds of thinking (for example, identifying conclusions or synthesizing and evaluating) have been generic descriptions applicable to almost any discipline or course. Analytical rubrics that specify kinds of thinking  are especially valuable to students and faculty if they go beyond generic descriptions and instead specify traits tailored to a specific assignment. For example, if the assignment is to write a persuasive argument, a primary trait of one kind of thinking needed is selecting evidence appropriate for the intended audience. The rubric for scoring an assignment will describe other important traits in addition to thinking traits. Primary traits are observable indicators of performance on that particular task; for example, for a task to design a graph, the primary traits would be: graph has a title, axes are labeled, scale is appropriate, correct variables on axes (Benander, 2000). Two excellent guides to using primary trait assessment to design rubrics, with many examples, are Barbara Walvoord and Virginia Anderson's Effective Grading (1998) and Linda Suskie's Assessing Student Learning (2004); a shorter guide is an article by Craig A. Mertler (2001) available online. His article includes the template below.
Template for Analytic Rubrics
 

Beginning
1

Developing
2

Accomplished
3

Exemplary
4

Score

Criteria #1

Description reflecting beginning level of performance

Description reflecting movement toward mastery level of performance

Description reflecting achievement of mastery level of performance

Description reflecting highest level of performance

  

Criteria #2

Description reflecting beginning level of performance

Description reflecting movement toward mastery level of performance

Description reflecting achievement of mastery level of performance

Description reflecting highest level of performance

  

Criteria #3

Description reflecting beginning level of performance

Description reflecting movement toward mastery level of performance

Description reflecting achievement of mastery level of performance

Description reflecting highest level of performance

  

Criteria #4

Description reflecting beginning level of performance

Description reflecting movement toward mastery level of performance

Description reflecting achievement of mastery level of performance

Description reflecting highest level of performance

  
Source: Mertler (2001).

The table below shows my adaptation of the generic template above for an assignment in an English 101 class to write a 350-word summary of an argument they will use as a source in their research paper. Only two primary traits are shown on the rubric below as a sample.

Analytic Rubric Template Adapted to Summary Assignment (Excerpt)

Summarizes argument accurately

1. Misses the point; does not get claim, reasons, and support.

2. Partially identifies claim, reasons, and support.

3. Identifies claim, reasons, and support; perhaps not maintaining author's emphasis.

4. Accurately identifies claim, reasons, and support without oversimplifying or distorting.

 

Excludes nonessential information

1.  Does not distinguish introductory information  from argument; includes unneeded examples.

2.  Does not distinguish introductory information  from argument; perhaps includes unneeded examples.

3.  Includes a little background or an unneeded example or two.

4. Excludes introductory information and unneeded examples.

 

Why Use a Rubric When a List of Grading Criteria Might be More Helpful to Students?

To assess a course, a rubric scored by normed raters is essential to get reliable information on which to base decision on how to improve the course. But  to provide feedback to individual students on an assignment, a list of grading criteria might be more helpful than a rubric because the list allows the instructor more flexibility in providing comments. Below is the list of grading criteria I use, rather than a rubric, to provide feedback on the EGL 101 summary assignment; to provide individual comments tailored to each essay, I write evaluative comments beside each criteria item.

GRADING CRITERIA: SUMMARIZING AN ARGUMENT
Content
1. You meet minimum requirements (or your paper gets an F): follows instructions; argument approved in advance; rough draft and final deadlines met on time; at least 350 words; outline of reasons and support is included; article is included; nothing is plagiarized.
2. Argument is summarized well:
a.Presents the basic claim or thesis in a sentence or two
b.Accurately presents the major reasons and support for the thesis without oversimplifying or distorting the argument
c.Paraphrases the author primarily in your own words, quoting occasionally or not at all
d.Excludes introductory and background information
e.Excludes multiple examples and other repetition
f.Excludes evaluative comment
Overall Organization and Coherence
3.The introduction gives useful background information on the issue, clearly identifies the source (author, title, date, where published), and precisely states the thesis.
4. It is suitably arranged to support the thesis; organized clearly, with separate major reasons and their support in separate paragraphs.
5. There are helpful transitions between and within paragraphs.
Paragraphs
6. Each paragraph has a clear topic sentence stating its main point.
7. Each paragraph fully supports its topic sentence with reasons and explanations.
Sentence Structure
8. Sentences are primarily in your own words, quoting occasionally or not at all
9. Sentences are well-written: varied beginnings and patterns; condensed; subordinated well without awkwardness; words are selected well; tone is appropriate.
Grammar and Punctuation
10. You use correct spelling, punctuation, and grammar.

Another way to present grading criteria is to provide a table, rather than a list. Below is a table (slightly modified) designed by Stephen Paulone of University of Maryland University College to provide feedback to his online students on a business economics assignment. 

Content: Key items of the assignment

Comments:

80 Percent  Percentage Earned:

The paper is 1050-1400 words in length.

 

The paper summarizes an economics issue presented in a newspaper, magazine, journal, TV report, or on the Internet.

 

The paper identifies subjects covered in our text.

 

The paper identifies the thesis of the article.

 

The paper explains how the role of economics is involved.

 

The content is comprehensive and persuasive.

 

The paper links theory to relevant examples of current experience and industry practice and uses the vocabulary of the theory correctly.

 

The paper indicates which economic theories apply to the situation.

 

Major points are stated clearly and are supported by specific details, examples, or analysis.

 

The paper includes a citation for the article or other media used as references.

 

Organization / Development

Comments:

20 Percent   Percentage Earned:

 

The paper has a structure that is clear, logical, and easy to follow.

 

The paper develops a central theme or idea, directed toward the appropriate audience.

 

The introduction provides sufficient background on the topic and previews major points.

 

The conclusion is logical, flows from the body of the paper, and reviews the major points.

 

Transitions between sentences/paragraphs/sections aid in maintaining the flow of thought.
The tone is appropriate to the content and
assignment.

 

Conclusion

We started with a generic holistic rubric for assessing the thinking showed on any task that requires thinking (essay, clinical practice, performance, project, demonstration, portfolio, etc.), continued with generic analytical rubrics, showed examples of holistic and analytical rubrics designed for specific courses, and ended with examples of criteria lists that might provide better feedback to students than rubrics.

References and Resources for Assessing Thinking

Articles and Books 

Aretz, A. J., Bolen, M. T., & Devereux, K. E. (1997). Critical thinking assessment of college students. Journal of College Reading and Learning, 28(1). Retrieved September 2, 2005, from Expanded Academic Index ASAP database (A20053524). 

Baughn, J. A., Brod, E. F., & Page, D. L. (2002). Primary trait analysis: A tool for classroom-based assessment. College Teaching, 50(2), 75-80. 

Benander, R., Denton, J., Page, D., & Skinner, C. (2000). Primary trait analysis: Anchoring assessment in the classroom. Journal of General Education, 49(4). 279-302. 

Brookfield, S. (1997). Assessing critical thinking. New Directions for Adult and Continuing Education, 75, 17-29. San Francisco: Jossey Bass. 

Critical Thinking Faculty Learning Community, Indiana University Southeast. (2004). Beyond course content: Changing hearts and minds. Workshop presentation. Available http://ilte.ius.edu/pdf/ctflcworkshoppresentation.pdf 

Facione, P. A. (1990). The delphi report: Executive summary. Millbrae, CA: California Academic Press. Available http://www.insightassessment.com/pdf_files/DEXadobe.PDF 

Facione, P. A. & Facione, N. C. (1994). Holistic critical thinking scoring rubric. Millbrae, CA: California Academic Press. Available http://www.insightassessment.com/pdf_files/rubric.pdf

Halpern, D. (1993). Assessing the effectiveness of critical thinking instruction. Journal of General Education, 42(4), 238-254. Reprinted in JGE, 50(4), 270-286.

Lawson, T. J. (1999). Assessing psychological critical thinking as a learning outcome for psychology majors. Teaching of Psychology, 26(3), 207-209. 

McCarthy, P., Schuster, P., Zehr, P., & McDougal, D. (1999). Evaluation of critical thinking in a baccalaureate nursing program. Journal of Nursing Education, 38(3), 142-144.

Mertler, C. A. (2001). Designing scoring rubrics for your classroom. Practical Assessment, Research, and Evaluation, 7(25). Available http://PAREonline.net/getvn.asp?v=7&n=25

National Postsecondary Education Cooperative (US Department of Education). (2000) The NPEC Sourcebook on Assessment, Volume 1: Definitions and Assessment Methods for Critical Thinking, Problem Solving, and Writing, vol. 1. Available  http://nces.ed.gov/pubs2000/2000195.pdf.

Nokes, K. M., Nickitas, D. M., Keida, R., & Neville, S. (2005). Does service-learning increase cultural competency, critical thinking, and civic engagement? Journal of Nursing Education, 44(2), 65-70.

Peirce, W. (2002). Course assessment handbook. Prince George’s Community College. Available http://academic.pgcc.edu/assessment/Handbook.doc

Sorrell, J. M., Brown, H. N., Silva, M. C., & Kohlenberg, E. M. (1997), Use of writing portfolios for interdisciplinary assessment of critical thinking outcomes of nursing students. Nursing Forum 32(4). Retrieved September 2, 2005, from Expanded Academic Index ASAP database (A20304283).

Suskie, L. (2004).  Assessing student learning: A common sense guide. Bolton, MA: Anker Publishing. 

Van Gelder, T., Bissett, M., & Cumming, G. (2004). Cultivating expertise in informal reasoning. Canadian Journal of Experimental Psychology, 58(2), 142-152.

Walvoord, B. E. & Anderson, V. J. (1998). Effective grading: A tool for learning and assessment. San Francisco: Jossey Bass.

Wolcott, S. K. (2004). Using rubrics for assessing critical thinking skills. Available: http://www.planning.iupui.edu/conferences/national/National/2004/Handouts/Wolcott/Wolcott04.pdf

Rubrics 

Brenau University. Critical Thinking Rubric for LE 495, Women & The Law
http://intranet.brenau.edu/assessment/content/ct/default.asp  Click on "Frank Adaptation." 

Evergreen State College. Evergreen Thinking Rubric http://www.evergreen.edu/institutionalresearch/pdf/CriticalThinkingRubric.pdf 

North Hennepin Community College. Critical Thinking Rubrics
http://www.nh.cc.mn.us/mydocs/1000007/ThinkRubric.doc

Mid-South Community College (AR). Critical Thinking Rubric
http://assessment.midsouthcc.edu/pdf/criticalthinking.pdf

Washington State University. The Critical Thinking Rubric
http://wsuctproject.wsu.edu/ctr.htm 

Resources for Rubrics 

Indiana University Kokomo. Links to Educational Resources about Rubrics
http://www.iuk.edu/~koctla/assessment/rubrics.shtml

Winona University. Assessment and Research website. Over 100 rubrics.
http://www.winona.edu/air/rubrics.htm

To the Annotated List of Documents from PGCC

Back to the MCCCTR homepage