Title: Process Writing and Computer Correction
Authors: Yi-chien Yao, Feng China University, Taiwan and Clyde A. Warden, The overseas Chinese College of commerce, Taiwan
Source: http://www.lc.tut.ac.jp/callej/Warden1.html
The goal of this research and the motivation behind the research goal (problem) are clearly stated in the beginning of the introduction. The research goal is set up by the authors' observation that most language teachers in EFL (English as a Foreign Language) settings have difficulty applying 'a-process-approach-to-writing' technique to actual classroom instructions due to some constraints which were normally encountered in most of EFL settings, such as large class size and standardized exam preparation which focuses on 'a product writing'. Thus, this article aims to show that the application of computers, used for creating feedback on grammar and mechanical errors, can be effective not only in relieving teachers of heavy teaching loads but also in making a process approach technique successfully available even in a large class size.
With regard to 'a process approach', why the process approach to writing is preferred over the product-approach technique is well covered in the section of 'literature review', including the description of what is meant by 'a process approach' and 'a product approach to writing'. The researchers also cited some of the previous researches over the issue of 'the computer grammar checking software' to talk about the appropriateness of computer software for writing correction. There are a lot concerns over the computer's writing correction because of its imperfectness. For the reason, the authors carefully selected QBL software (see description at bottom of this section) to minimize the effect of the software inefficiency on writing correction since QBL is relatively better than the other commercially available grammar checking software. All of those concerns are reviewed in various contexts of other literature in the 'literature review' section.
*The QBL (Quick Business Letters) system is adopted in this research. The QBL tool is a window based program that allows students to send writing electronically to the teacher. It can automatically find errors, track students' progress, and print feedback for each student on his/her errors.
a) The number of subjects are 173 who are first year university students in Taiwan and Non-English major students from four different departments. All of them are Chinese students who received little or no formal writing training in 'the process approach'. All of the subjects were taking a Freshmen English course of 3 credit hours (See table A below).
With the constraints in EFL settings in mind, the authors selected Chinese students in an EFL setting as a sample. Its population, thus, would be all Chinese students who have no formal training concerning process-approach writing in Taiwan.
| Department: | Insurance | Finance | Chemical Eng. | Intl. Trade |
|---|---|---|---|---|
| Label: | A | B | C | D |
| Number of students: | 38 | 37 | 48 | 50 |
Concerning the description of the subjects, a detailed information on the subjects would be usually helpful(maybe not necessary in this case) in making all other variables as equal or similar as possible except one or two employed variables, thereby decreasing the effect of extra variables on the research outcomes. This is because researchers need to ensure if the concerned variables are solely responsible for its outcome before coming to any conclusion regarding the employed variables. The information about the subjects, for example, would include the subjects' characteristics, attitudes toward this research, gender, and the subjects' previous level of writing skills, all of which could be another factors influencing the results of this study.
b) With regard to scoring of data (errors in this case), the authors did not fully describe how to calculate the mean errors. Were those errors based on the number of mechanical errors such as punctuation or the number of paragraphs which are incoherent, poorly organized in terms of logic? I can guess that the three-handed assignments might be graded according to criteria seen on the Assignment Checklist returned to each student (See Appendix 1) but still unclear about which measurement scale was employed to calculate those errors.
c) Although it is assumed that there is only one female teacher involved, the number of instructors needs to be clearly specified. This is because the number of the instructors triggers the issue of scoring agreement. Concerning scoring agreement which is required to avoid the scorer's bias, it is unclear as to whether the single instructor was the only one involved in scoring or there are any other scorers who didn't get involved in teaching. Due to these reasons, how the scoring system was conducted is a bit unclear.
a) Design of study
If the researchers assigned the subjects to two different groups to see the effectiveness of the computer involvement in correction, the researchers might be in a better position to corroborate their claims based on the findings; for example, groups with computer correction access(experiment group) and groups with no computer access (control group) or other different variables such as access to human teachers. In this research, however, the computer was simply set up as a single variable that the researchers are interested in in terms of the error correction. This loosen design is likely led to a weak corroboration of its finding.
b). Variables
It is clear that the dependent variable is 'mean errors' since the authors looked at the mean errors to see the rate of decrease. With regard to independent variables, however, it was not established because one treatment (computer's involvement) was dealt with. Concerning constant variables, all subjects were given the same instructions (constant variable) along with the computer's correction.
Another thing to be considered for extraneous variables in this kind of writing-error-correction research is that the different topics of writing could work as important variables in contributing to producing different errors. In other words, it is less likely for students to make mechanical mistakes if the easy topic is chosen such as introduction of oneself than difficult topic such as discussion of a certain political issue for several reasons (attention). It means that there is a possibility that the kinds of topic given to students could contribute to the error decrease rather than computer involvement in error correction. (Students were given two different topics which unknown in this paper.)
c) Results
This research concluded that all four classes showed declines in their rate of errors. Looking at each class on its own, it is quite clear that each assignment brings a statistically significant reduction in total errors. The researchers said that the reduction in errors is quite pronounced and clearly shows that students had responded to the computer generated feedback. However, unless the other variables that might contribute to error decreases should be clearly excluded, this result would likely show a weak conclusion. In other words, it is not sure to say if students' error decreases are really caused solely by QBL although QBL can be assumed to be a positive factor in this research.
a) Validity
Concerning validity issue, if math test fails to test students' math-related ability by presenting inappropriate test items such as social or health related questions, it is said that the math test has no or weak validity in its testing design(method). In other words, the test materials should be designed good enough to achieve the test goal to well maintain validity. Likewise, if the authors of this research aim to show if the computer's correction is helpful in decreasing students' mechanical errors in their writing, the test items (writing assignment in this case) to see its effectiveness of the computer intervention should be designed to ensure its goal. It means that only when students make less errors in three different writings over the period while being solely supported by the continuous aid from the computer feedback, it may be safe to claim that the decrease might be caused by the computer correction. However, in this study, a revision of the students' first assignment was used again as the third assignment, while the second writing is a different subject from the first. The authors allowed the students to edit the first writing and asked them to return the revision of the first for the third assignment. Then, the third assignments were used to count the number of errors. In this case, errors in third writing assignment are likely to decrease regardless of any other variables involved (even computer). Unless the students were expected to revise the whole content of the first one quite differently from the first one, the third writing would be almost the same as the first writing. Thus, it is assumed that the rewrite of their first writing could contribute greatly to the reduced errors as well. In other words, the familiarity of the same writings checked previously by the QBL would likely help the students make less mistakes in the third writing. It is easier to write something based on the completed draft than start writing from scratch. Thus, using three independent writings as test items over the time would be more helpful in maintaining validity than reusing the first writing as the third one.
b) Reliability
It is unclear that the authors allowed the other observers except for one instructor who get involved in teaching', to grade the same assignments in order to seek scoring agreements for reliability issue. The scoring agreement is suggested as one of effective method to avoid scorer's bias, thereby ensuring reliability of the research ('How to design and evaluate research in education', 1996). The scoring process (Instrument) also needs to be considered to make sure reliability as well as internal validity since scoring procedure can be changed due to the ambiguousness of established criteria( what is meant by 'organization', 'style'), which may allow different interpretations of the results among different testers. (instrument decay; Jack R. Fraenkel ,1996). In this research, the scoring process would pose little threat to reliability issue if only mechanical errors are targeted, whose correction criteria is quite general. However, the final grades were reflected by the integration of what QBL checked and what the teacher manually did with its focus more on the the evaluation of students' paragraph development and organization of ideas, all of which need a bit strict criteria.
The authors in this study present two aspects of the employment of computer in a writing class. One is whether or not the computer is helpful in decreasing the mechanical errors. The other is whether or not the computer can, accordingly, free teachers to spend more time on the process aspects. Concerning the first one, although the data shows a positive response to what is claimed by the researchers, other significant variables are not considered which might possibly affect the outcome. For example, concerning punctuation and spelling issues in writing, the subjects' characteristics or attitudes (carelessness) can be considered as possible variables which might contribute to their writing errors without regard to the computer's help. Because of the presence of those possible variables other than computer's feedbacks, the fact that the decrease in errors is quite distinctive doesn't necessarily mean that students had responded to the computer generated feedbacks.
Concerning the latter one, the researchers also showed their interest in the reduced teachers' load with computer mediation. However, the conclusion about the interest was not made through being put into the research design but was derived from the observation of the process of the whole research. For this reason, many comments on the relationship between computer program and process approach has little to do with what was actually conducted in the study. This research design itself was centered on the first interest, computer correction not the process approach. Thus, such comments as the inclusion of computer software allowed process to be stressed, needs to be based on more relevant research founding. This is because the founding of error decreases in this research setting doesn't give us much information about whether or not the inclusion of computer software also allowed the process to be stressed and made adoption of a process approach more viable. Although the results clearly showed that teacher can have more time to focus on the process aspects because of computer involvement, it doesn't necessarily mean computer had directly contributed to the process approach.
In conclusion, in EFL settings like Taiwan where a lot of constraints exist for the process approach to be employed, this study result may be valuable in that computer can be one of good mediums that takes over partly what teachers have to do, thereby freeing teachers to spend more time on the process aspects. However, the study still needs a bit tightening research design to make its result more general to broader situation.