Assessing Critical Thinking in Writing

Our Writing Initiative rubric for evaluating critical thinking in student writing is very simple. It has three competencies that we feel should be made evident in the assignment, and that should be evidenced in the resulting papers. There are many aspects of critical thinking that might have been used as measures, but we chose three that we felt were common to almost all good writing assignments.

The three competencies are:
1) Find appropriate evidence and accurately interpret it
2) Identify pertinent arguments pro and con and rebut those arguments that counter to your thesis
3) Draw a thoroughly justified conclusion(s)

We ask instructors to have students indicate when they submit assignments to their portfolios which assignments best address critical thinking competencies. Part of that request is because we want students (and faculty!) to be very conscious of critical thinking elements in the assignment.

At the start of the Initiative in 2008, we realized early on that instructors too quickly said that "all my assignments use critical thinking" but were then unable to actually point out where that critical thinking occurred.

The assessment of this area in writing samples has shown improvement the past five years. Students are most capable in achieving competency in evidence gathering and interpretation and in drawing conclusions. This can partially be explained because these skills are addressed through research in not only WI classes but in other courses as well. The competencies we stress in our information literacy component also reinforce these two areas. The development of argument and drawing conclusions are a focus in Composition I & II essays, and are stressed in the College Writing Exam essays.

The weakness that is evident in our analysis is one that keeps many papers in the “competent” part of our rubric rather than receiving a rating of “accomplished.” We observed in earlier years and have addressed in subsequent Faculty Institutes, that many instructors create assignments that require a single argument rather than the more “accomplished” criteria of addressing multiple and counter arguments.

That also affects the third competency, since a writer addressing only one point of view will most naturally arrive at only one conclusion.

In some writing situations, such as in the sciences and on the College Writing Exam, multiple arguments are actually seen as “going off topic.” We have made very conscious efforts this past year in our training and when working with faculty on assignment creation to have them create assignments which allow and encourage multiple perspectives on an argument.

In our analysis of portfolio samples from the fifth year, we again found improvement in all three competencies but competency 2 (argument) both showed the greatest improvement and still remains the weakest area. 

Our conclusion is that this is an area that needs to be addressed by more courses using writing across the curriculum. The problem may be more in the assignments that are given, than in students' ability to accomplish the task.


All of our rubrics, including the critical thining rubric, are available online.

Writing Initiative Faculty Institute - January 2012

The Writing Initiative will be holding the 13th Faculty Institute for faculty developing new writing intensive course sections or taking on a WI section for the first time on January 4 and 5, 2012.

This is a collaborative seminar-style event where we will be discussing overall Initiative goals, sharing existing WI materials and best practices, creating new assignments and materials and looking at the pedagogy, tools and resources available for writing intensive courses.

Following the Writing Initiative Handbook, our topics of discussion include:

• Writing Initiative goals and requirements
• Formal and informal writing
• A review of existing course materials (syllabus and assignment samples) both for regular sections and for any existing WI sections.
• Creating new WI assignments - examples & using templates
• Student portfolios: objectives, hands-on use of the eFolio product, sample student portfolios, grading strategies
• Critical thinking assignments – template, rubrics
• Information literacy assignments – template, rubrics
• Responding to and grading student writing - for content and for writing
• Using the Writing Center
• Using eTutoring
• Online resources for online & face-to-face course sections
• Initiative assessment of WI Courses
• Hands-on session using LibGuides as an editor and adding links, resources, media objects etc.

Writing Across Disciplines Day

This full day event is for faculty who want to incorporate writing into their courses. 

The writing across the disciplines approach is designed to give students additional academic writing experiences, but also to have them practice with the language conventions and specific formats typical of that discipline. For example, the lab report differs in content, format and style from literary criticism or the annual business report.

Based on the best practices that have emerged during the four years of the PCCC Writing Initiative, we will be discussing how to use, assign and evaluate writing that occurs in different disciplines.

There will be a series of topics discussed including:

• A brief overview of the goals of the Initiative and our writing intensive courses 
• What do you do with writing in your classes now?
• Using formal and informal writing
• Creating effective writing assignments
• Incorporating information literacy and critical thinking
• Responding to and grading student writing - for content and for writing
• Writing Resources available to you and your students at PCCC

January 6, 2012  8:30am - 3:30 pm  The Writing Center at PCCC (Paterson Campus)  A113

Space is limited, so if you’re interested in participating, please respond to writingcenter@pccc.edu by December 23rd. Please include your department and the courses you will be teaching in the spring. For more information, go to http://pccc.libguides.com/writing/.

Questioning the Socratic Method and Academic Integrity

After I read an article on insidehighered.com, I wondered about the state of the Socratic method in American classrooms. The article chronicles the case of Professor Steven Maranville at Utah Valley University. He teaches a capstone business course and his style is to ask questions even if students don't volunteer to answer.

It seems that students complained about that (and pedagogical choices) and that those complaints contributed to the university denying him tenure.

I won't attempt to judge his employment status or his classroom teaching, but the article points to other cases where the power of student evaluations and opinions on faculty rehiring was significant. Professors at Louisiana State University and  at Norfolk State University are noted. I also noticed a number of online posts recently about an article headlined "NYU Prof Vows Never to Probe Cheating Again—and Faces a Backlash."

Maranville's style of pedadgogy is described as Socratic, "engaged learning" in which he "pushed for students to go beyond lectures" and created teams for assignments outside class. I don't hear anything very radical in any of that. If done well, it sounds quite admirable.

The Socratic Method as a style of teaching is named after the classical Greek philosopher Socrates who used this method of inquiry. It is covered in chapter one of the textbook I use in my critical thinking course, and I can't really imagine teaching a class face-to-face or online without using some form of this questioning technique.

This kind of inquiry and debate between individuals is popular in law schools and probably used more in the humanities and the social sciences. Using opposing viewpoints and asking and answering questions stimulates critical thinking and often illuminate ideas.

It is a dialectical method, and so it can often generate oppositional discussion and the defense of one point of view is up against the defense of another. I suppose that opposition can, if the discussion lacks discipline, lead to out and out arguing.

Maybe the Socratic method seems negative because it uses a method of hypothesis elimination. The stronger hypotheses are found by identifying and eliminating those that lead to contradictions or that cannot be defended.

As the article points out, one advantage of this kind of teaching is that it challenges students "to learn how to think on their feet." In general, I find that students at the undergraduate and graduate level don't like that challenge, especially if they have been not been asked to do it in other classes.

It wasn't Socratic teaching that got the NYU prof mentioned above in trouble. It was his attempt to be more aggressive about cheating and plagiarism in his introductory information technology class. Perhaps feeling safer because he had tenure, he decided to use the Blackboard course-management system and Turnitin's plagiarism-detection software together for the first time. Those integrated software packages allows assignments that are submitted to be automatically checked for matches to materials online.

He found that plagiarism was "pervasive" and 22 of the 108 students admitted cheating by the end of the semester.

The connections to the earlier Socratic case? Students, especially those in certain majors, are simply not asked to do much writing. (It is one of the reasons the Writing Initiative at PCCC was launched.) And faculty in those disciplines often feel unprepared to evaluate writing "as writing" as opposed to grading based on subject matter content. Students have not been asked to do this kind of writing, and instructors are not prepared or unwilling to "police" writing for plagiarism.

Introduce some powerful software that detects it automatically, even on papers or students that you would not have suspected, and an ugly truth is revealed.

Of course, the other connection in these two cases is that the teachers paid for their actions with bad student reviews. Professor Ipeirotis dropped to a below-average score of 5.3 (of 7.0) from his usual 6.0 to 6.5. He claims that "The Dean’s office and my chair ‘expressed their appreciation’ for me chasing such cases (in December), but six months later, when I received my annual evaluation, my yearly salary increase was the lowest ever, and significantly lower than inflation, as my teaching evaluations took a hit this year."

As anyone who has pursued an academic integrity case at a college with the administration probably discovered, it can take up many hours away from your actual teaching or research. Sadly, when he posted on a blog that he had decided to no longer pursue cheating instances, that too was met with a backlash.

Lesson learned? Well, I completely agree with what he says: "Rather than police plagiarism, professors should design assignments that cannot be plagiarized."  I am actually presenting tomorrow at a writing across the curriculum faculty roundtable here at PCCC on just that topic. I hope to get a positive response from faculty, and will report back here soon.

This is a cross-posting from my blog at Serendipity35.net

Writing Support Online

Back in 2009, I read that the University of Arizona was developing a one-credit online writing course that will be used to supplement their three-credit GenEd (general education) classes.

It's one way to address a problem a problem that occurs on campuses where enrollment is growing and the number of staff and the facilities to support them have not increased.

This is true of many writing centers, and they often have problems meeting the increased demand. For better and for worse, online versions are often seen as an economically feasible solution.

At PCCC, we use eTutoring, but we don't have anything like an online writing center. Since our center only opened in 2009, we are fortunate that our roll out was planned in phases. Since the Center was built to support the writing intensive courses, our student clients increase with each semester. This semester we are running 30 sections of WI courses. By the end of this academic year, we will have about 1000 college-level students in the Center student database.

When the grant ends in September 2012, the Center is designed to become the college-level writing center supporting about 4000 students. (PCCC also has basic skills, EOF and ESL labs.)

An online writing course could be viewed as a form of writing across the disciplines. At UA, the course were introduced as a one-credit supplement to the typical three-credit general education class. It is intended to provide an interactive and self-paced online environment in which students' writing skills are diagnosed and improved.

According to an article on the UA course:

"...the courses will not replace gen-ed classes, but instead will support them with needed writing instruction that is not available in the typical 50 minute lecture period...The online course will offer tutorials on topics in writing not ordinarily covered by professors, such as grammar, drafting a thesis and style and craft. Writing proficiency will be tested by a diagnostic system that will, depending on the student's score, direct him or her signed to target a given problem area. These modules will feature flash animation and other interactive software tailored to the specific skill level of the student.

Thomas Miller, English professor and associate provost of academic affairs, pointed out that the online course will help deal with problems in writing essays before it's too late. He said that students all too often realize they have significant problems in writing only after their papers are returned with a poor grade. Miller added that research on writing pedagogy shows that "students do not read teachers' comments on their papers. They often do not understand comments they read and do not apply them." The online course is intended to remedy this problem by developing students' writing skills before a paper is even assigned to them.The course will "take them through the writing process," Miller said. "It will help them draft a research question or thesis and will include strategic visits to the writing center."

It's an interesting idea. At PCCC, our approach is to try to incorporate these skills into the GenEd courses. We have designed 23 distinct courses so far across al disciplines as writing-intensive. We also strive to better equip those faculty to support their students' writing, as well as sending students to our writing center for face-to-face help and sending them online to use eTutoring.

One reason that we chose this path is because we wanted to also include faculty in the learning process. A good part of our initiative effort goes to professional development. We are trying to help faculty improve their ability to create writing assignments, facilitate assessment and utilize technology to do it.

PCCC, like many other colleges, is looking at putting more courses online each year. Since the Writing Center needs to support them as well as students on two smaller satellite campuses at Wanaque and Passaic, we will also be looking at supplementing writing instruction online.

Critical Thinking and Writing in Science and Math

It was just announced this week that Passaic County Community College has won a $4.1 million federal grant to improve college students' success in science and math and get high school students engaged in those fields. The funds come from the U.S. Education Department's Hispanic-Serving Institutions Program. (More than half of PCCC's students are Hispanic.) The Education Department has sought to boost training in what are known as the STEM fields (science, technology, engineering and math) in order to help American students compete in an increasingly high-tech, global marketplace.

In our writing initiative, we have now redesigned six courses in science and math as writing-intensive. Those courses are: College Math, Statistics, Environmental Science, Biology I, Biology II and Microbiology. Part of our approach to WI course redesign is to include the explicit use of critical thinking (CT) assignments and techniques in the writing activities. Scientific critical thinking is different than the CT found in a literature class. Though certain standards, such as open mindedness, relevancy and accuracy, certainly go across disciplines, a familiarity with the nature of science, the experimental method and the knowledge base of a particular science changes most teachers approach to CT.

 Critical thinking skills that are often noted as essential in STEM areas include:

• Distinguishing correlation from cause and effect
• Distinguishing analysis from description
• Understanding reductionism and knowing when to use it.
• Recognizing important variables in complex phenomena
• Isolating and testing variables
• Knowing the value of prediction (versus retrodiction) 
• Distinguishing scientific evidence, reasoning and rationalizing
• Exercising sound reasoning in understanding
• Making complex choices
• Understanding the interconnections among systems
• Framing, analyzing and solving problems

We can look at retrodiction as an example of a skill that is probably unfamiliar to many people. Retrodiction is the act of making a "prediction" about the past. That's an idea that is hard for students to grasp at first. You would speculate about uncertain events in the more distant past so that you can "predict" a known event in the less distant past. If greenhouse gases had been capped at a certain level in 1950, what effect would it have on the current readings? If a murder victim's body had been refrigerated for a month before it was discovered, how would it affect an autopsy findings? It is used in archaeology, climatology, evolutionary biology, financial analysis, forensic science, and cosmology.

Hands-on, project-based math and science curriculum activities are often the nest places for students to think critically about the use of math and science in solving problems. In the taxonomy of higher level thinking skills, this deepens their knowledge of the basics.

This approach is used in many K-12 curriculum too. WGBH offers teachers a number of classroom resources at http://www.teachersdomain.org.

The problem based learning approach (PBL) can be seen in "PBL in Action" activities such as The Wing Strength Design Project and Geometry in the Real World: Students as Architects offered by sites such as  http://www.edutopia.org and http://www.21stcenturyskills.org.

Further Reading
Developing Reflective Judgment (Jossey-Bass Education Series)

Writing and Critical Thinking

There seem to be as many definitions of critical thinking as there are courses that include it. We have a writing intensive section of our critical thinking class, CT 101, here at PCCC. I have taught one of those sections since last spring.

There is at least general agreement in what I read about using critical thinking (CT) that these components are part of what using critical thinking in an academic setting includes:

• Mindful, conscious, reflective thinking about the task before making decisions about what to believe or do.
• Being able to assess the authenticity, accuracy, and the value of information (evidence) and arguments.
• Self-directed inquiry, analysis and critique.

The reason why critical thinking fits so well into a writing initiative is fairly obvious when you look at those elements. They already look very much like what we are teaching in traditional writing classes when we teach the essay or research writing. By that, I mean writing/critical thinking elements such as:

1. Using a process to evaluate a thesis (proposition, hypotheses, judgment) and then supply well-supported evidence. 
2. Determine if the evidence found is valid
3. In that process (of decision-making, research, problem-solving) generate options, looking at opposing viewpoints and make discriminating judgments.
4. Arrive at a conclusion(s) that is the most reasonable based on the evidence.

In our own Initiative faculty development, we have now trained more than 50 full-time and part-time instructors in teaching WI courses which includes writing pedagogy, critical thinking and information literacy. It is sometimes surprising to instructors how well those three areas work together,

Perhaps the biggest myth we have to dispel is that "every course already has critical thinking."  Though certainly every course has the potential for CT and students are always "thinking", unless the teacher and the students are aware that they are using CT techniques and are asked to reflect on that process, there is little resulting improvement in those skills.

One aspect of this that we have found in our analysis of the Initiative is that our students are better at finding evidence than they are at discriminating appropriate evidence and being able to accurately interpret it. This is no new finding. The use if the Internet has made all of us better at finding information, but has no positive effect (possibly a negative effect) on evaluating and interpreting what we find. Those two latter skills need to be consciously taught. They are not learned by simply being asked to find information or "do research."

There are many sites online that deal with critical thinking and also CT and writing. Our CT101 course site and sites like criticalthinking.org are good starting places. There are other colleges working on these initiative. We have crossed paths at several conferences with the folks at Georgia State who are doing a CT and writing program.

Spring Roundtable

The College Writing Committee Presents:
Writing Across the Curriculum Spring Roundtable

Wednesday, March 23, 2011

3:30-5:00 PM

Paterson Room

This roundtable forum is focused on improving writing across the curriculum at PCCC!

 Topics will include:

Text Rendering: Reading to Write and Writing to Read—

M. Hillringhouse

Using Reflection to Enhance Student Learning—

E. Nesius

Reading and Questioning: Questions from the Text that Generate Writing—

David Bolger

Bring your own writing ideas to share!

Feel free to DROP IN!

Sponsored by The Title V Writing Initiative and The Professional Development Committee.

Writing Across the Curriculum

Elizabeth Nesius and Ken Ronkowitz from the Writing Initiative attended the International Writing Across the Curriculum (WAC) Conference this month held at Indiana University.

Featured speakers included Art Young, Clemson University, speaking on "WAC Today and Tomorrow: Composing in Many Modes and Media" and Terry Myers Zawacki, George Mason University, on "Researching the Local/Writing the International: Developing Culturally Inclusive WAC Programs and Practices"

The 2010 Conference was hosted by the Campus Writing Program at Indiana University in Bloomington, Indiana. This biannual conference, the only U.S. conference dedicated exclusively to writing across the curriculum (WAC) and writing in the disciplines (WID), is typically of interest to people who are concerned with using writing to improve teaching and learning—faculty, administrators, and students from post-secondary institutions, as well as faculty and administrators from secondary schools.

Writing Across the Curriculum is a philosophy that maintains that writing is a valuable learning tool that can help students to synthesize, analyze, and apply course content; clear writing helps to develop clear thinkers in any curriculum.

Our own Writing Initiative philosophy is that All Teachers Teach Writing. While English teachers are traditionally charged with teaching "learning to write," we promote other disciplines using "writing to learn" techniques.

Some useful links for more information on WAC & WID efforts:

• WAC Clearinghouse  and  their journal (also at Colorado State) is at  http://wac.colostate.edu/atd/ 
• The WAC program at George Mason University  and their Writing Intensive course requirements.
• Writing Across the Curriculum in Middle and High Schools
• Writing and Reading Across the Curriculum
• WAC for the New Millennium: Strategies for Continuing Writing-Across-The-Curriculum-Programs
• Content-Area Writing: Every Teacher's Guide