安 徽 大

硕 士 学 位 论

题目 The Validity Study of Cloze Tests in Chinese Context

专 业 英语语言学

 

2004年4月20

The Validity Study of Cloze Tests in Chinese Context

Presented in Partial Fulfillment of the Requirements

For the M.A Degree in English Language and Literature

In the School of Foreign Studies of Anhui University

By

Fan Lixia

Under the Supervision of

Professor Zhu Yue

Anhui University

Hefei

April, 2004

Acknowledgements

I am extremely indebted to my supervisor, Professor Zhu Yue, for his warm-hearted encouragement and valuable directions, especially for his insightful comments on my paper. His critical and careful reading of my paper has saved me from a lot of errors. Without his guidance and encouragement, my paper would have never come out in the present form.

I am grateful to all those who have taught me courses in the School of Foreign Studies of Anhui University. I have benefited a lot from their work and spirit. Special thanks are extended to Prof. Hong Zengliu, Prof. He Gongjie, Dr. Zhang Xuemei, Dr. Chen Bin. Each of them has given me great help in my study.

I also want to express my appreciation to my colleagues, classmates and friends for their great support and understanding of my work.

Abstract

The cloze tests originated in the 1950s as a means of assessing the difficulty of a reading test for native speakers. Some studies have indicated that the cloze tests might also be valid ways of measuring native-speaker reading ability. The validity coefficient between cloze tests and standardized reading examinations in these studies range from 0.61 to 0.95 (Brown, J. D. 1983: 238). It is also suggested that the cloze tests could be used for assessing learners’ progress of second or foreign languages. Many researchers believe that the cloze test is a valid and economical way to test reading comprehension ability.

The validity of the cloze procedure as measurement of reading skills has long been argued in the field of pragmatics. Much empirical work on the cloze tests has been focused on determining whether or not cloze tests can measure subjects’ reading skills and whether or not the use of cloze tests can improve subjects’ reading ability. Heaton (1986) claims that the most common purpose of a cloze test is to measure reading comprehension. Henning (1987) holds that the test of reading comprehension by using cloze tests may turn out to be a test of grammar or vocabulary. Researches regarding strategies for taking cloze tests are interested in determining whether cloze tests actually measure global reading skills as they are commonly purported to. One of the studies most worth mentioning is carried out by Homburg and Spaan (1981). They have administered a rational-deletion cloze with 23 blanks to 39 EFL subjects. In verbal report studies, they have found that nearly 20% of the subjects do not use the co-text for clues to fill the blanks. Cohen (1984) has studied the strategies for taking the cloze tests and found that perhaps only a quarter of non-native respondents read the entire cloze passage before responding. In China, Yang Huizhong (1998) and Zou Shen (1998) also show that the cloze test can test the abilities which are tested by other examination types.

This paper is divided into five chapters. The first chapter is an introduction to the cloze test and a survey of the cloze literature. The second chapter is a study of the face validity of rational-deletion cloze test and the multiple-choice cloze test in Chinese context. The third chapter is devoted to the discussion of the content validity of the cloze procedure as measurement of reading skills in Chinese context. The fourth chapter focuses on the response validity of cloze procedure as measurement of reading skills in Chinese context. Then comes the conclusion of the paper.

Through questionnaires and interviews, the paper aims to prove whether the cloze test is a valid and economical way for testing reading comprehension ability and which type of cloze test is more reliable in the testees’ eyes by making the validity research on the cloze tests in Chinese context. The originalities of this research lie in the discoveries that firstly, the face validity of multiple-choice cloze test is more reliable than rational-deletion cloze test, secondly, the content validity of the cloze procedure as measurement of reading ability is insufficient and finally the Subjects’ responses can not support the idea that cloze procedure measures global reading ability.

Key words: Cloze procedure Face validity Content validity

Response validity Rational-deletion cloze Strategy

Multiple-choice cloze Chinese context

内容摘要

完形填空又叫完形程序。这一测试方法是1953年由威尔逊·泰勒创设, 用来测试以英语为本族语的人所阅读的文章的难度,此后, 拉德尔等人多次在研究中发现完形填空是测试阅读能力的一种灵敏度很高的测试手段。他们的研究表明, 完形填空题考试和多项选择阅读理解题考试有很高的相关性。许多语言学家也认为完形填空是一种检测外语语言能力的一种既有效又经济的手段。

完形填空,作为考查学生阅读技能的一种题型,一直是语用学领域引起争议的问题。国外在这方面争论的焦点是完形填空这种题型是否测试学生的阅读技能,是否这种题型的采用有助于提高学生的阅读理解能力。Heaton(1986)认为完形测试的最终目的是考查学生的阅读能力。Henning(1987)认为完形填空的阅读理解是一种语法和词汇的测试,至少是这些技能的综合测试。对于学生在测试中所用策略的研究,其争论的重点是:完形填空考查学生的全面阅读技能还是局部阅读技能。其中最引人注目的是Homburg 和Spaan(1981)的研究。他们对39位学生进行了一次完形填空测试,测试后逐一对学生进行面谈,了解学生做题时所反应的策略。通过对学生会谈报告的研究,他们发现有大约20%的学生在做题的过程中没有使用上下文的线索,只是在原句的基础上猜测。在国内,杨惠中(1998)和邹申(1998)的研究表明完形填空所测试的内容与其它题型测试的内容有很高的相关。

本研究论文包括五个章节的内容。第一章介绍了完形填空题型的发展,并对多年来完形填空的研究作了详细的调查。第二章对合理删词完形填空和多项选择完形填空进行了研究和比较。第三章探讨完形填空作为考查阅读理解技能的一种题型,其内容是否有效。第四章着重从测试者的反应来研究完形填空题所测试的阅读技能。

本研究有三个发现:第一,选择完形填空与合理删词完形填空相比,其表面效度更可信一些;第二,完形填空作为检测阅读能力的一种题型, 其内容是不充分的;第三,测试中,学生的反应和他们所使用的策略不能证明完形填空是考查学生的全面阅读能力。

关键词: 完形填空 效度 表面效度 内容效度 反应效度

合理删词完形填空 多项选择完形填空 策略 中文语境

Contents

Acknowledgements -----------------------------------------------------------------------------I

Abstract (English) ------------------------------------------------------------------------------II

Abstract (Chinese) -----------------------------------------------------------------------------IV

Chapter One A Survey of Cloze Test Researches ---------------------------------------1

1.1Cloze Test ------------------------------------------------------------------------------------1

1.1.1 Definition of Cloze Test ---------------------------------------------------------1

1.1.2 Types of Cloze Test --------------------------------------------------------------2

1.2 Validity -------------------------------------------------------------------------------------- 3

1.2.1 Definition of Validity ----------------------------------------------------------- 3

1.2.2 Types of Validity ---------------------------------------------------------------- 4

1.3 A Survey of the Cloze Literature -------------------------------------------------------- 6

1.3.1 Domestic Validity Research on Cloze Test ------------------------------------ 6

1.3.2 Foreign Validity Research on Cloze Test -------------------------------------- 7

Chapter Two Face Validity of Rational-deletion Cloze and Multiple-choice Cloze

in Chinese Context---------------------------------------------------------------------------10

2.1 Subjects-------------------------------------------------------------------------------------10

2.2 Questionnaire Design ---------------------------------------------------------------------15

2.3 Data Collection and Analysis ----------------------------------------------------------- 15

2.4 Summary------------------------------------------------------------------------------------17

Chapter Three Content Validity of Cloze Test as Measurement of Reading Skills in Chinese Context ---------------------------------------------------------------------------19

3.1 The Comparative Study ----------------------------------------------------------------- 19

3.1.1 Study on the Judgements from English Teachers----------------------------------- 20

3.1.2 Further Study --------------------------------------------------------------------------- 22

3.2 Summary----------------------------------------------------------------------------------- 25

Chapter Four Response Validity of Cloze Test as Measurement of Reading Skills in Chinese Context--------------------------------------------------------------------------- 26

4.1 Interview Procedure ---------------------------------------------------------------------- 26

4.2 Data Collection and Analysis ----------------------------------------------------------- 27

4.3 Summary----------------------------------------------------------------------------------- 29

Chapter Five Conclusion ------------------------------------------------------------------ 30

Bibliography ---------------------------------------------------------------------------------- 32

Appendixes ------------------------------------------------------------------------------------ 35

Tables and Figures

Tables

Table 1 Subjects for the Questionnaire----------------------------------------------------- 11

Table 2 Anxiety Degrees of R.D Cloze and M.C Cloze --------------------------------- 16

Table 3 Percentages of Anxiety Degrees----------------------------------------------------16

Table 4 Identification of the Skills---------------------------------------------------------- 22

Table 5 Percentages of the Identification--------------------------------------------------- 22

Table 6 Identification of the Skills for Further Study-------------------------------------23

Table 7 Percentages of the Identification----------------------------------------------------23

Table 8 Results of the Interview Study------------------------------------------------------27

Table 9 Results of the Interview Study------------------------------------------------------27

Figure

Figure 1 Illustration of the Content Validity------------------------------------------------24

Chapter One A Survey of Cloze Test Researches

This paper aims to prove that the cloze test is not a highly valid way to test reading comprehension by doing validity researches on the cloze test, and that rational-deletion cloze test is superficially more invalid than multiple-choice cloze test through the study on subjects’ psychological processes during the cloze testing.

Language testing is an essential part of language teaching. It helps to set standards and provide useful information for educational administration, for language teaching and learning and for research. With the development of technology, computers are used to do the statistical analysis of the test scores, which helps the interpretation of the scores. For instance, the pace of teaching can be adjusted according to the achievements of the students in the test.

The cloze test has enjoyed a short but varying history. Researches on the cloze test began with an investigation of its applicability for measuring reading ability of native English speakers by W . Taylor in 1953 (Brown, J.D.1983: 238). After three years, Taylor suggested that it could be used to test reading comprehension. Then it was accepted that it could be used for assessing the progress of second and foreign language learners.

In recent years, many Chinese researchers have done researches on the cloze test. Most of them have focused on solution methods and scoring principles. Other researchers have studied the reliability and validity of the cloze test. They believe that the cloze test is a valid way to test foreign language reading ability.

1.1 Cloze Test

1.1.1 Definition of Cloze Test

The cloze test is called the “gap filling” test. It was invented by W. L.Taylor in 1953. The principle of the cloze testing is based on the Gestalt theory of ‘closure’(closing gaps in patterns subconsciously). Heaton (1988: 16) in his book “Writing English Language Tests” has discussed two types of cloze procedures. Firstly, the words for deletion are selected subjectively consisting largely of structural words. Secondly, the words are deleted systematically.

Cloze tests measure the reader’s ability to decode ‘interrupted’ or ‘mutilated’ messages by supplying the most acceptable choices from all the contextual clues available. In a cloze test, words are removed from a reading passage at regular intervals. For example, every 5th word may be removed. The reader must read the passage, try to guess the missing word and complete each gap in the text, using the most appropriate word. The chosen passage should be at a difficult level appropriate to the people who are to take the test. The text should be of the style appropriate to the kind of language ability being tested.

1.1.2 Types of Cloze Test

Chinese researchers, Liu Runqing and Han Baocheng (2000: 66-78), in their book “Language Test and Its Methods”, divide the cloze test into four types:

The fixed-ratio cloze. It means every nth word is deleted fixedly.

The rational-deletion cloze. It means you delete the words you want to delete. For instance, if you want to assess the mastering of prepositions, then all the prepositions can be deleted.

The multiple-choice cloze. It can be used to test the reading comprehension that is easy to score. Look at the following example:

I find that train travel is restful and 1 . There ‘s always something to see. I particularly enjoy 2 through Switzerland and Italy. You can get a very good 3

of what a country is like from a 4 ; you don’t go too fast and you stop quite often. 5 rail travel is very fade and I am never sick on a train 6 I am on a ship! Then there are always people to 7 if you feel like a chat.

1. A interesting B tiresome C lonely D cheap

2. A to travel B traveling C to drive D driving

3. A idea B looking C drawing D painting

4. A bike B plane C car D train

5. A Also B Yet C But D However

6. A when B while C as D and

7. A talk to B talk C speak about D speak

(From Shu Yunxiang, 1999:135)

C-test. It was created by Klein-Braley, C. in 1985. It can be used to test the students’ lexis ability. The principle is that the second half of every second word is deleted, such as “ China is a ve beau and pros country.

It is essential in cloze tests to draw upon a subject which is neutral in both content and language variety used. It is always advantageous to provide a ‘lead-in’, thus no deletions should be made in the first few sentences so that the students have a chance to become familiar with the author’s style and approach to the subject of the text. Of the four types of cloze test, rational-deletion cloze and multiple-choice cloze are most widely used in all kinds of foreign language tests of our country.

1.2 Validity

1.2.1 Definition of Validity

Henning (1987: 89 ) defines validity as follows:

Validity in general refers to the appropriateness of a given test or any of its component parts as a measure of what it is purported to measure. A test is said to be valid to the extent that it measures what it is supposed to measure. It follows that the term valid when used to describe a test should usually be accompanied by the preposition for. A test then may be valid for some purpose but not for others.

However, if a test is to be used for any purpose, the validity of use for that purpose needs to be established and demonstrated. The validity of a test is the extent to which it measures what it is supposed to measure. Every test, whether it be a short, informal/classroom test or a public examination, should be as valid as the constructor can make it. The test must aim to provide a true measure of the particular skill which it intends to measure. If it measures external knowledge and other skills at the same time, it will not be a valid test.

Invalid tests are those that have undesirable contents mixed in the desirable content. Since validity has been defined as the capacity of a test to measure what it is purported to measure, a variety of considerations may bring about a reduction of this validity.

Invalid application of tests

Inappropriate selection of content

Imperfect cooperation of the examinee

Inappropriate referent or norming population

Use of invalid constructs

1.2.2 Types of Validity

Validity can be established in a number of different ways, which leads most writers on the topic to talk of different types of validity. This paper uses the terms internal validity and external validity: the internal validity relates to the studies of the perceived content of the test and its perceived effect, while the external validity relates to the studies of comparing students’ scores with their ability gleaned from outside the test. There are many ways of assessing the internal validity of a test, but three of the most common ways are: face validity, content validity and response validity.

1. Face validity

Ingram (1977: 18) defines face validity as “ the test’s surface credibility or public acceptability”. A test is said to have face validity if it looks as if it measures what it is supposed to measure. Language tests which have been designed primarily for one country and adopted by another country may lack face validity. A test which does not have face validity may not be accepted by candidates, teachers, education authorities or employees. It may simply not be used; and if it is used, the candidates’ reaction to it may mean that they do not perform on it in a way that truly reflects their ability. Face validity is important in testing. For one thing, tests that do not appear to be valid to users may not be taken seriously for their given purpose; For another, if test takers consider a test to be face valid, they are likely to perform to the best of their ability on the test and respond appropriately to items.

Face validity is a kind of impressionistic reaction to the examination on the part of examinees. If a test has good face validity, the students’ motivation is maintained, for most students will try harder if the test looks sound. If, on the other hand, the test appears to have little relevance in the eyes of students, it will clearly lack face validity. Data on face validity may be gathered by the teachers interviewing students or asking them to complete a questionnaire about their attitudes and reactions to a test they have just taken.

2. Content validity

A test is said to have content validity if its content constitutes a representative sample of the language skills, structures, etc. In order to judge whether or not a test has content validity, a specification is needed to include the relevant skills or structures, etc, which the test is meant to cover. Such a specification should be made at a very early stage in a test construction. It is not to be expected that everything in the specification will always appear in the test, otherwise there may simply be too many of them in a single test.

What is the importance of content validity? First, the greater a test’s content validity is, the more likely it is to accurately measure what it is supposed to measure. A test in which major areas identified in the specification are under-represented—or not represented at all—is unlikely to be accurate. Second, such a test is likely to have a harmful backwash effect. Areas which are not tested are likely to become areas ignored in teaching and learning.

Content validation involves the experts’ making judgements in a systematic way. In content validation, judgements are gathered from the people who are familiar with language teaching and testing but who are not directly concerned with the production of the test in question. A common way is to analyze the content of a test and to compare it with a statement of what the content ought to be.

3. Response validity

Response validity, an increasingly common aspect of test validation, is to gather information on how individuals respond to test items. The processes they go through and the reasoning they are engaged in when responding are important indications of what the test is testing, at least for the test-takers. Hence there is a considerable current interest in gathering the accounts (from the learners or test takers) of their test-taking behaviors and thoughts.

The way to measure response validity is to gather introspective data. After candidates have taken a test, they can be interviewed about the reasons why they produced the answers they did. For example, introspection on a cloze task will show whether a student has to answer an item by using the range of reading skills intended by the test designer, or whether all that is needed is some knowledge of the grammatical structure of the phrase in which the item appears.

1.3 A Survey of the Cloze Literature

The study of language testing as a new area of applied linguistics has developed quickly at the coming of the new century. In this field, there have been extensive researches done. Among them, the validity is the most important question of all in language testing. The validity of the cloze test as measurement of reading comprehension is a controversial issue which many linguistics and foreign teachers have concerned themselves about for years. The cloze test originally intended to measure the reading difficulty level of a text. It was thought to be a reliable means of determining whether or not certain texts were at an appropriate level for a particular group of students (both native speakers and non-native speakers). However, this test has long been argued in the field of pragmatic linguistics.

1.3.1 Domestic Validity Researches on Cloze Test

In our country, researches on cloze test have developed quickly in recent years. The writer of this paper conducts a survey about cloze literature produced from 1994 to 2003. Most of the researches concentrate on design principles, scoring problem, solution techniques and context analysis. Many researches on reliability and validity have been done from a variety of perspectives. Yang Huizhong and C.Weir (1998:118) claim: “ If the passage is well chosen and the test item can be well designed, cloze test can test the global language proficiency of the testees effectively.”

In their work, the internal correlation coefficient of CET-4 is shown as follows:

 

LC

RC

VS

CL

WR

Listening .C

1.0000

       

Reading .C

.563

1.0000

     

Vocabulary & structure

.539

.6147

1.0000

   

Cloze

.476

.5310

.6264

1.0000

 

Writing

.3870

.3594

.4698

.4042

1.0000

Total

.7922

.8915

.8022

.7076

.5813

(from Yang Huizhong and C.Weir, 1998:118)

The data shows that the cloze test has a high correlation coefficient with the other subtests. That means cloze tests can test the ability that is tested by other test items basically.

Zou Shen ( 1998: 216-229 ), in his work “ Validation Study on Test for English Majors (TEM)”, has done the validation study on TEM, which also shows that the cloze test item can test the abilities that are tested out by other test items.

 

WR

Dictation

LC

Cloze

G&V

RC

Writing

1.0000

         

Dictation

.4042

1.0000

       

Listening .C

.3809

.7039

1.0000

     

Cloze

.3440

.5229

.5402

1.0000

   

Grammar &Vocab.

.3671

.5536

.5776

.5566

1.0000

 

Reading .C

.3309

.5512

.6030

.4924

.5302

1.0000

Total

.5616

.8496

.8384

.6997

.7620

.8032

(from Zou Shen, 1998:226)

In this table, cloze test items show high correlation coefficient with other test items. The data ranges from .3440 to .6997, which means that there is a high internal correlation coefficient of the test. The author has presented some suggestions for the further language teaching, for example, pay attention to the lexis, the usage of the collocation of the phrases and the grammar teaching.

In the work “ Study on the Reliability and Validity of M.C Cloze and O.E Cloze”, Xu Ang(1998: 26-28) has studied the cloze papers produced by 55 postgraduates and indicated that M.C cloze tests are more reliable than O.E cloze when the cloze passages are short ( about 250 words). In his work, he proves the validity of cloze test, but no detailed analysis is given. After him, Hu Fengming (2002: 72-74) studies the standards of evaluating a test’s quality from the angles of validity, reliability, difficulty, sector length, feasibility and feedback of teaching. Through the analysis of cloze designing aims and principles, his research shows that cloze is an economical but efficient means in testing students’ general language ability and reading ability.

1.3.2 Foreign Validity Researches on Cloze Test

The validity of cloze test has long been argued in foreign researchers. Heaton (1988:107), in his book “ Writing English Language Tests”, claims that the most common purpose of the cloze test is to measure reading comprehension. It has been argued that cloze procedure measures textual knowledge: an awareness of cohension in a text, involving the interdependence of phrases, sentences and paragraphs within the text. He believes that the cloze test (mechanical deletion of words in the text) is generally used to measure global reading comprehension, although insights can undoubtedly be gained into particular reading difficulties. However, it is also argued that cloze test measures an underlying global linguistic ability (and indeed even knowledge of the world ) rather than simply those skills associated with reading comprehension. In the work “A Guide to Language Testing: Development, Evaluation and Research”, Henning (1987:92) claims that our test of reading comprehension through cloze examination may turn out to be a test of grammar or vocabulary, or at least a few such items may be “mixed in”. The Japanese researcher Sandra S. Fotoshad (1993:64-66) has showed that cloze can be used to test global language abilities in his paper “Is Cloze Test a Test Way for Testing Integrative Abilities”.

Researchers regarding strategies for taking cloze tests are of interest in whether such tests actually measure global reading skills as they are commonly purported to. Cohen(1984:70-81) studies the strategies for taking cloze tests and has found that perhaps only a quarter of non-native respondents read the entire EFL cloze passages before responding. Homburg and Spaan (1981:25-33) have administered a rational-deletion cloze with 23 blanks to 39 EFL subjects from three levels. In their verbal report studies, it is found that nearly 20% of the respondents do not use the preceding or following sentences for clues to fill in the blanks but rather guess on the basis of the immediate context. Kleiman (1986:104) has made a case study which has shed some light on the issue of reading the text before completing a L1 cloze test. Verbal protocal data is provided by a 7th grade (12-year-olds) Brazilian girl filling in two cloze passages. What has emerged is that she does not use global clues until she has completed a substantial number of blanks. However, contrary to the claims of these researchers, Chihara (1977: 63-73) and Oller (1985: 149) have maintained that cloze assesses global reading.

The survey made above shows that firstly, opinions about what cloze test measures are different; secondly, Chinese studies on validity of cloze test are too general, with almost no consideration given to different ways of assessing the validity of a test; thirdly, the data of study on cloze test strategies comes from investigation in a foreign language environment.

In order to conduct effective studies on the validity of cloze test, it is necessary to study it from different angles of assessing the validity. Thus, at the end of this part, the writer would like to present three questions to those who are interested in this problem to share with:

How do subjects respond to multiple-choice cloze and rational-deletion cloze, which are used in almost all kinds of tests of our country?

Are the contents of cloze test valid to measure reading comprehension skills?

What strategies are used by cloze testees in the Chinese environment?

In the following three parts, the writer conducts a detailed study respectively on the face validity, content validity and response validity of cloze test, which might throw some light on the above questions.

Chapter Two Face Validity of Rational-Deletion Cloze and Multiple-Choice Cloze in Chinese Context

Rational-deletion cloze is the cloze in which the words for the deletion are selected subjectively (consisting largely of structural words in tests of grammar and key context words in vocabulary). This type of cloze test emphasizes the purpose and practical use of cloze testing. In order to avoid the differences in evaluation, both the rational-deletion cloze and multiple-choice cloze are designed in the same form. This is a new pattern of cloze test - multiple choice cloze. The rational-deletion cloze and the multiple-choice cloze are two important types of cloze test in China. They can often be seen not only in college proficiency tests, but also in college entrance examinations and adult tests. Cloze test has always been a part of the Band-Four and Band-Six tests. Before 2002, rational-deletion cloze was used in the National Professional Title test. But in recent two years, rational-deletion cloze has been changed into multiple-choice cloze. Cloze test is experiencing changes nowadays. More and more multiple-choice cloze patterns are selected by test designers in all kinds of tests. Many teachers and learners consider that the rational-deletion cloze is too difficult for foreign language learners, and the multiple-choice pattern is more appropriate. So in this part, the face validations of rational-deletion cloze and multiple-choice cloze will be analyzed with the specific data obtained directly from the questionnaire study on anxiety degrees displayed by the 71 English major college students when they are sitting for the two types of cloze test.

2.1 Subjects

The subjects of this questionnaire are 71 English major college students. They are three-year students from Chuzhou Teachers College and classified into two groups. Table 1 is the detailed description of the subjects.

Table1 Subjects for the Questionnaire

Subjects

Number

Male

Female

Total number of subjects

71

22

49

Group one

38

12

26

Group two

33

10

23

Students in Group one are required to finish two rational-deletion cloze tests designed by Sparkenglish selected from Band-Four Model Tests of Forecast for 2003. Students in Group two are required to finish the same two passages of multiple-choice cloze test. When tested at the same time, the subjects did not know the test purpose. The whole test process is observed by their extensive English teachers in their regular class. The following Cloze 1 and Cloze 2 are two rational-deletion cloze passages, while Cloze 3 and Cloze 4 are two multiple-choice cloze passages.

Cloze1

Directions: There are 20 blanks in the following passage. You should fill the blanks with the best appropriate word.

We were about to gather up our belongings and return to our car when a man appeared. He looked very 1 indeed and asked us angrily 2 we realized that the grounds were private 3 . Father looked very confused and the man pointed to a 4 which said that camping was 5 forbidden. Poor father explained that he had not seen it and did not know that camping was not allowed. 6 father apologized, the man did not seem 7 and asked him for his name and 8 . All the way home, we were so upset that hardly 9 spoke a word. This unpleasant event had 10 a wonderful day in the country. 11 the rest of the week, we wondered what 12 . The following Sunday, we stayed at home 13 it was a fine day. About noon, a large and very expensive car stopped outside our house. We were 14 when we saw several people preparing to have a picnic in our garden. Father got very angry and went out to ask them what they 15 they were doing there. You can imagine his 16 when he recognized the man who had been 17 to us the week before! Both men 18 out laughing and father welcomed the strangers into the house. 19 we become good friends ---but we learned a 20 we have never forgotten.

Cloze2

Directions: There are 20 blanks in the following passage. You should fill the blanks with the best appropriate word.

One day, in Philadelphia, three men walked into the Thirtieth Street Station. One of them went to the Information Center in the main lobby(门厅) to check the departure-arrival listings. There he 1 out that the 3 o’clock train to New York was going to leave an hour late that say.

When he told his friends about the 2 , they suggested that they all 3 to a nearby bar to wait. So 4 they went.

About an hour 5 they returned to the station laughing and joking and relaxed. When they arrived at the right track, they saw the train was already moving slowly 6 the station.

All three men ran 7 the train shouting, “ wait! wait! Come back!” Two of them 8 up with the last car, 9 The railing(栏杆) of the rear 10 , pulled themselves up, and boarded the moving train. 11 his time, the third man had stopped running. As he hopelessly watched the train 12 out the station he could still see his friends wave and 13 them call. At first he had a confused look 14 his face. Then after a brief moment, he turned 15 and slapped(掌击)his knee as he laughed.

A man 16 nearly, who had seen it all, turned to him and asked, “What’s so 17 , fellow? After all, you r two friends made 18 and you didn’t.” The man was still laughing so hard 19 he had tears in his eyes as he replied, “You don’t understand! Those two guys came to see me 20 !”

Cloze3

Directions: There are 20 blanks in the following passage. You should fill the blanks with the best appropriate word.

One day, in Philadelphia, three men walked into the Thirtieth Street Station. One of them went to the Information Center in the main lobby(门厅) to check the departure-arrival listings. There he 1 out that the 3 o’clock train to New York was going to leave an hour late that say.

When he told his friends about the 2 , they suggested that they all 3 to a nearby bar to wait. So 4 they went.

About an hour 5 they returned to the station laughing and joking and relaxed. When they arrived at the right track, they saw the train was already moving slowly 6 the station.

All three men ran 7 the train shouting, “ wait! wait! Come back!” Two of them 8 up with the last car, 9 the railing(栏杆)of the rear 10 , pulled themselves up, and boarded the moving train. 11 this time, the third man had stopped running. As he hopelessly watched the train 12 out the station he could still see his friends wave and 13 them call. At first he had a confused look14 his face. Then after a brief moment, he turned 15 and slapped(掌击) his knee as he laughed .

A man 16 nearly, who had seen it all, turned to him and asked, “What’s so 17 , fellow? After all, you r two friends made 18 and you didn’t.” The man was still laughing so hard 19

he had tears in his eyes as he replied, “You don’t understand ! Those two guys came to see me 20 !”

1. a worked

2. a. delay

3. a. went

4. a. on

5. a. late

6. a. into

7. a. on

8. a .caught

9. a. grabbed

10. a. stage

11. a. By

12. a. pulled

13. a. hear

14. a. in

15. a. to

16 .a. standing

17. a. terrible

18. a. it

19. a. what

20. a. to

b. made

b. accident

b. would go

b. in

b. later

b. away

b. in

b. brought

b. held

b. board

b. Till

b. pulls

b. listen to

b. at

b. away

b. stood

b. mad

b. that

b. that

b. out

c. sought

c. relay

c. go .

c. off

c. after

c. off

c. out

c. marched

c. took

c. platform

c. At

c. pull

c. watch

c. of

c. out

c. was standing

c. interesting

c. this

c. as

c. off

d. found

d. lateness

d. gone

d. away

d. before

d. out of

d. after

d. went

d. shook

d. landing

d. Since

d. be pulled

d. make

d. on

d. up

d. had stood

d. funny

d. way

d. so

d. through

Cloze4

Directions: There are 20 blanks in the following passage. You should fill the blanks with the best appropriate word.

We were about to gather up our belongings and return to our car when a man appeared. He looked very 1 indeed and asked us angrily 2 we realized that the grounds were private 3 . Father looked very confused and the man pointed to a 4 which said that camping was 5 forbidden. Poor father explained that he had not seen it and did not know that camping was not allowed. 6 father apologized, the man did not seem 7 and asked him for his name and 8 . All the way home, we were so upset that hardly 9 spoke a word. This unpleasant event had 10 a wonderful day in the country. 11 the rest of the week, we wondered what 12 . The following Sunday, we stayed at home 13 it was a fine day. About noon, a large and very expensive car stopped outside our house. We were 14 when we saw several people preparing to have a picnic in our garden. Father got very angry and went out to ask them what they 15 they were doing there. You can imagine his 16 when he recognized the man who had been 17 to us the week before! Both men 18 out laughing and father welcomed the strangers into the house. 19 we become good friends ---but we learned a 20 we have never forgotten.

1. a. annoyed

2. a. if

3. a. property

4. a. notice

5. a. surely

6. a. Though

7. a. critical

8. a. age

9. a. nobody

10. a. hurt

11. a. At

12. a. had happed

13. a. because

14. a. curious

15. a. wanted

16. a. excitement

17. a. rude

18. a. broke

19. a. During time

20. a. knowledge

b. worried

b. where

b. sign

b. sign

b. seriously

b. Since

b. satisfied

b. occupation

b. no one

b. spoilt

b. After

b.would happen

b. unless

b. amused

b. thought

b. anger

b. unkind

b. came

b. In time

b. relationship

c. troubled

c. why

c. profit

c. paper

c. strongly

c. If

c. angry

c. address

c. one

c. damaged

c. For

c. happened

c. while

c. frightened

c. supposed

c. surprise

c. doubtful

c. burst

c. On time

c. friendship

d. disappointed

d. how

d. advantage

d. mark

d. strictly

d. When

d. understood

d. unit

d. anyone

d. harmed

d. On

d. is to happen

d. when

d. astonished

d. prepared

d. hesitation

d. unfriendly

d. brought

d. At time

d. lesson

2.2 Questionnaire Design

In order to know how subjects respond to the rational-deletion cloze and multiple-choice cloze, the questionnaire is to survey language test anxiety when subjects are engaged in an English cloze test. Designed on the basis of a test-anxiety study conducted by Jones-Madsen, the questionnaire consists of 3 questions. Each has 5 objective choices A, B, C, D and E: choice A, strongly agree; choice B, agree; choice C, no view; choice D, disagree; choice E, strongly disagree. The questions for the survey are like those in the questionnaire paper below. The questionnaire is designed in Chinese, in order to avoid students’ misunderstanding of the questions. The work is to be finished in class immediately after the cloze test.

Questionnaire paper

1). 我喜欢做这一题型。

A. 完全同意 B. 同意 C. 说不来

D. 不同意 E. 完全不同意

2). 这一题型很难。

A. 完全同意 B. 同意 C. 说不来

D. 不同意 E. 完全不同意

3). 做这一题型时我心情愉快,平静。

A. 完全同意 B. 同意 C. 说不来

D. 不同意 E. 完全不同意

2.3 Data Collection and Analysis

Data collection of test anxiety consists of collections of test anxiety indicated by rational-deletion test takers and multiple- choice test takers. Teachers responsible for different groups collect the students’ questionnaire papers, and bring them together. The 38 questionnaire papers for rational-deletion cloze test and 33 questionnaire papers for multiple-choice cloze test are collected. After the questionnaire papers are corrected, the anxiety degrees of rational-deletion cloze test takers and multiple-choice cloze test takers are analyzed respectively. The results of the survey are indicated in Table 2 and Table 3.

Table 2 Anxiety Degrees of R.D Cloze and M.C Cloze

Group one (38 subjects)

Rational-deletion cloze

Group two (33 subjects)

Multiple-choice cloze

A

B

C

D

E

A

B

C

D

E

Question 1

2

5

9

12

10

3

13

14

3

0

Question 2

6

21

8

2

1

0

4

8

20

1

Question 3

1

3

14

13

7

3

23

5

2

0

Table 3 Percentages of Anxiety Degrees

Questions

Question 1

Question 2

Question 3

Mental experience

I do not like the test

(choice D and E)

Hard to say

(choice C)

The test is

difficult

(choice A and B)

Hard

to say

(choice C)

I do not feel relaxed

(choice D and E)

Hard

to say

(choice C)

Rational-deletion cloze

60%

24%

71%

21%

52%

37%

Multiple-choice cloze

9%

42%

12%

24%

6%

15%

From Table 2, it can be seen that the subjects who choose D and E in the first question for rational-deletion cloze and multiple-choice cloze are respectively 22 out of the 38 and 3 out of the 33. Table 3 indicates that the subjects who do not like the test of rational-deletion cloze and multiple-choice close are respectively 60% and 9%. 60% of the students do not like the rational-deletion cloze and 24% of the students hold the opinion that it is hard to say, so the subjects who like this pattern are only16%. This difference shows that the most of the students are not interested in the rational-deletion cloze test.

In question 2, 27 out of the 38 students doing the rational-deletion cloze and 4 out of the 33 students doing the multiple-choice cloze test choose A and B, which means that the test is difficult. The percentages are respectively 71% and 12%. The comparison of the data shows that the rational-deletion cloze test is more difficult than the multiple-choice cloze test.

In Table 2 and Table 3, 20 out of the 38 students doing rational-deletion cloze and 2 out of the 33 students doing multiple-choice cloze do not feel relaxed in the tests. The percentages of the subjects who do not feel relaxed and have no views in rational-deletion cloze test are respectively 52% and 37% and the percentages who do not feel relaxed and have no views in multiple-choice cloze test are respectively 6% and 15%. It means that 11% of the students doing rational-deletion cloze and 79% of the students doing multiple-choice cloze feel relaxed in the tests

Comparing the different percentages and data for the two patterns of cloze test, the writer observes that the anxiety degrees in rational-deletion cloze are higher than the anxiety degrees in multiple-choice cloze, so cloze procedure, especially rational-deletion cloze procedure is rather difficult for Chinese students, although in some countries where English is used as a first language or second language, cloze procedure is now an important item in language testing (especially in the United States). The use of this difficult test may place much pressure on Chinese students and prevent students from having interest in learning English. On the other hand, the anxiety degrees of the three questions in the multiple-choice cloze is less than the rational-deletion cloze, so the multiple-choice cloze is more acceptable and appropriate for Chinese learners. If a test can not be accepted by the subjects, candidates, teachers, education authorities or employees, it does not have face validity and can not measure the students’ ability and improve their English. The data of test anxiety indicates that the rational-deletion cloze may be less valid to Chinese users. The multiple-choice cloze, however, may be face-valid, and learners are more likely to perform the test to the best of their ability and to respond appropriately to the items involved.

2.4 Summary

The chapter compares the face validations of rational-deletion cloze test and multiple-choice cloze test. We know that the rational-deletion cloze and the multiple-choice are two important types of cloze test in China. In order to understand which kind of cloze tests is more appropriate for Chinese learners, a questionnaire is conducted to 71 subjects of third-year students after a cloze test which includes two passages of rational-deletion cloze and the same passages of multiple-choice cloze. Students are classified into two groups – 38 for two rational-deletion cloze passages and 33 for two multiple-choice cloze passages. After the students have finished the two types of the cloze test, they are provided with a questionnaire which falls into three questions and five choices for each question. The questionnaire is designed on the basis of a test-anxiety study conducted by Jones Madsen. The students are asked to give their own choices for the three questions which are related to how confident they are about their answers to the blanks of the cloze test.

The average anxiety value of each question in rational-deletion cloze is higher than its corresponding value in multiple-choice cloze. The anxiety degrees of the three questions in rational-deletion cloze are higher than the anxiety degrees in multiple-choice cloze. By the comparison of the results, the multiple-choice cloze test is more accepted by the students than the rational-deletion cloze test. Therefore, it is concluded that the rational-deletion cloze may be less valid to Chinese users, while the multiple-choice cloze is face-valid.

Chapter Three Content Validity of Cloze Test as Measurement of Reading Skills in Chinese Context

The validity of cloze procedure as measurement of reading comprehension is a controversial issue which many linguistics and foreign teachers have concerned themselves about for years. The focus of the argument is whether the cloze test is to measure reading skills and whether this kind of tests contributes to examining subjects’ reading comprehension ability. In this part, the writer attempts to analyze the content validity of cloze procedure in measuring reading ability in comparison with reading skills and skills used in cloze test .

3.1 The Comparative Study

Heaton (2000: 105-106), in his book “ Writing English Language Tests”, defines reading skills as the ability to:

-- recognize words and word groups, associating sounds with their corresponding graphic symbols;

--deduce the meaning of words;

--understand explicitly stated information;

--understand relations within the sentence;

--understand relations between parts of a text;

--perceive temporal and spatial relationships and also sequences of ideas;

--understand conceptual meaning;

--anticipate and predict what will come next in the text;

--identify the main idea and other salient features in a text;

--generalize and draw conclusion;

--understand information not explicitly stated by making references;

--skim and scan (looking for the general meaning and reading for specific information);

--read critically;

--adopt a flexible approach and vary reading strategies according to the type of material being read and the purpose for which it is being read.

3.1.1 Study on the Judgements from English Teachers

In chapter two, the subjects in group one and group two are required to finish two rational-deletion cloze passages and two multiple-choice cloze passages. Then, the papers from group one are selected for further study. 22 English teachers from the English Department of Chuzhou Teachers College are required to correct the rational-deletion test papers (See Chapter 2: Cloze 1 and Cloze 2) of the students in group one. The total scores are 40 points for the 40 items involved. The two cloze passages with right answers are provided to the teachers in order that they can be used to get the valid correction and judgements.

Cloze1

Directions: There are 20 blanks in the following passage. You should fill the blanks with the best appropriate word.

We were about to gather up our belongings and return to our car when a man appeared. He looked very (1) annoyed indeed and asked us angrily (2) if we realized that the grounds were private (3) property . Father looked very confused and the man pointed to a (4) sign which said that camping was (5) strictly forbidden. Poor father explained that he had not seen it and did not know that camping was not allowed. (6) Though father apologized, the man did not seem (7) satisfied and asked him for his name and (8) address . All the way home, we were so upset that hardly (9) anyone spoke a word. This unpleasant event had (10) spoilt a wonderful day in the country. (11)For the rest of the week, we wondered what (12) would happen. The following Sunday, we stayed at home (13) while it was a fine day. About noon, a large and very expensive car stopped outside our house. We were (14) astonished when we saw several people preparing to have a picnic in our garden. Father got very angry and went out to asks them what they (15) though they were doing there. You can imagine his (16) surprise when he recognized the man who had been (17) unfriendly to us the week before! Both men (18) burst out laughing and father welcomed the strangers into the house. (19) In time we become good friends ---but we learned a (20) lesson we have never forgotten.

Cloze2

Directions: There are 20 blanks in the following passage. You should fill the blanks with the best appropriate word.

One day, in Philadelphia, three men walked into the Thirtieth Street Station. One of them went to the Information Center in the main lobby(门厅) to check the departure-arrival listings. There he (1) found out that the 3 o’clock train to New York was going to leave an hour late that say.

When he told his friends about the (2) delay , they suggested that they all (3) go to a nearby bar to wait. So (4) away they went.

About an hour (5) later they returned to the station laughing and joking and relaxed. When they arrived at the right track, they saw the train was already moving slowly (6) out of the station.

All three men ran (7) after the train shouting , “ wait! wait! Come back!” Two of them (8) caught up with the last car, (9) grabbed the railing(栏杆)of the rear (10) landing , pulled themselves up, and boarded the moving train. (11) By this time, the third man had stopped running. As he hopelessly watched the train (12) pull out the station he could still see his friends wave and (13) hear them call. At first he had a confused look (14) on his face. Then after a brief moment, he turned (15) away and slapped(掌击)his knee as he laughed.

A man (16) standing nearly, who had seen it all, turned to him and asked, “What’s so (17) funny , fellow? After all, your two friends made (18) it and you didn’t.” The man was still laughing so hard (19) that he had tears in his eyes as he replied, “You don’t understand! Those two guys came to see me (20) off !”

The procedure of skill identification has been conducted in two steps. First, the students’ papers are corrected by 22 English teachers from Chuzhou Teachers College. Before doing this, they have known the contents of reading skills. Then, after a hard, careful and painstaking process, the skills involved in the cloze tests have been identified. Table 4 and Table 5 show the results of the identification.

Table 4 Identification of the Skills

Score

Skills

Average

scores

Total

(40points)

Percentage

Using context to guess meaning of words

13.68

34.2%

Understanding relations between the sentences or between parts of text

6.14

15.35%

Identifying referent of pronounce

1.4

3.5%

Perceive temporal and spatial relationships and sequences

3.73

9.3%

Grammar

5.63

14%

Vocabulary

8.06

20%

Others

1.36

3.4%

Table5 Percentages of the Identification

Skills

Scores

Total(40points)

Percentage

Reading skills

24.95

62.5%

Other skills

15.05

37.5%

Table 5 indicates that the percentage of reading skills measured in the cloze tests is 62.5%, while other skills account for 37.5 %. The table shows what cloze test measures is more than reading comprehension. The content validity is defined as “the extent to which it measures what it supposed to measure and nothing else”. In the definition, “nothing else” means “other skills” at the same time. Based on the definition, the two cloze tests may not be valid tests. The score of reading skills measured in the two rational-deletion cloze passages is only 24.95 out of 40 points. 15.05 points out of the total belong to other skills, so as measurement of reading comprehension, the content of rational-deletion cloze test is not valid.

3.1.2 Further Study

The previous study chooses two rational-deletion cloze passages, and perhaps it is not enough to reach the conclusion that cloze procedure as measurement of reading skills is not valid, so in the further study, the writer selects ten multiple-choice cloze passages to conduct a continuing study. Out of the ten, five passages (see Appendix 1) are selected from Wang Chang-xi Model Tests of Forecast for 2003 CET-4 and other five passages (see Appendix 2) are selected from English Test 2002 for National Professional Title. The total points for the Band-Four Test are 100, and the total points for the Professional Title Test are 75. After the skills measured in cloze tests are compared with the contents of reading skills, the relevant data and percentages are shown in Table 6 and Table 7.

Table 6 Identification of the Skills for Further Study

Score

Skills

Band-Four

test (100points)

Professional Title test (75points)

Percentage

Using context to

deduce meanings of words

21

14

20%

Understanding relations

Between the sentences or

between parts of text

17

6

13.14%

Understanding conceptual meanings

5

3

4.57%

Perceiving temporal

and spatial relationships

and sequences

5

5

5.71%

Understanding information

by making inference

18

14

18.3%

Grammar

13

17

17.14%

Vocabulary

21

16

21.14%

Table 7 Percentages of the Identification

Skills

Scores

Total (175points)

Percentage

Reading skills

108

61.72%

Other skills

77

38.28%

The percentage of reading skills shown in table 7 is 61.72%, approximate to 62.5% shown in table 5. When the statistical data in table 5 and table 7 are compared, the percentages regarding vocabulary and grammar used in cloze tests do not appear to be very different. It indicates that the result of the previous study is reliable. From table 6 and table 7, it can be observed that some skills of reading comprehension have been used in the cloze test. When the contents used in the tests are studied against the Heaton’s definition of some specific skills involved in reading, it is discovered that only about 35.7% of the reading skills have been included in the measurement of the test. To explain this comparison clearly, Figure 1 is provided. Let us assume for the sake of illustration that the largest rectangle in Figure 1 represents the complete understanding of reading content as a general domain. The tiny circles distributed throughout Figure 1 represent items selected to represent specific reading skills. The four smaller rectangles represent domains responding to cloze test measurement, each containing different representative items. The fifty items of the large rectangle are said to comprise a content-valid proficiency test, in the sense that they are representative and comprehensive. However, only 36 of these items are indicated in the cloze tests, and 50 of these items are found to lie outside of the area of the given reading content, therefore, such a test certainly does not have content validity as a measurement of reading comprehension.

Figure 1 Illustration of the Content Validity

3.2 Summary

This chapter firstly reviews the contents of reading skills defined by Heaton (2000: 105-106). We understand what reading skills students should acquire. They need to be able to scan the text for particular bits of information, skim to get a general idea and read for detailed information. In order to know whether the cloze test contributes to examining testees’ reading comprehension ability, a comparative study is conducted with two steps. The first step is the study on the judgements from English teachers. The second step is the further study by selecting ten multiple-choice cloze tests. Comparing what the ten cloze tests measure with the contents of reading skills, the writer discovers that the cloze test measures more than reading skills, and many other skills, for example, vocabulary and grammar are mixed in. So it is concluded that the cloze test is not content-valid in measuring reading comprehension.

Chapter Four Response Validity of Cloze Test as Measurement of Reading Skills in Chinese Context

The theories on strategies used in cloze procedure have been well developed. Many new ideas in this field are emerging. However by now there has been little done in relating strategies with response validity of cloze test. Although many researchers’ studies on strategies for taking cloze tests have mentioned suspicion about cloze test as measurement of reading skills, they still give no demonstration about response validity. So this paper conducts interviews after the cloze tests and tries to provide specific evidence and data through subjects’ responses and their strategies used in cloze tests.

4.1 Interview Procedure

In chapter two, group-one subjects (38 students) and group-two subjects (33 students) are selected to finish respectively two rational-deletion cloze passages and two multiple-choice cloze passages. The 33 students who are asked to finish multiple-choice cloze tests have performed better than the students who are asked to finish rational-deletion cloze tests. In order to know the students’ psychological process of finishing the tests, face-to-face individual interviews are conducted with the 33 students who take the two multiple-choice cloze passages after the cloze tests. The interviews are to be conducted by asking the students why they have produced such answers to 40 blanks. Through the strategies used by the interview subjects, it can be discovered whether or not the skills used in the cloze tests are the skills reading comprehension is designed to measure. The following questions are the interview questions designed by the writer.

Interview Questions

1) Do you read quickly the entire passage to get a general meaning before choosing the answers?

yes or no

2) Where do you like to look for the right answers?

A. In immediate sentences

B. In forward and backward reading

3) When do you begin to use forward or backward reading to look for the answers?

A. From the beginning of responding

B. When you have filled in a large number of blanks

4) How many answers do you choose by the use of fixed collocation? (students’ self-statements about why they choose the answers for each blank)

5) How many answers do you choose on the basis of grammar? (students’ self- statements about why they choose the answers for each blank)

4.2 Data Collection and Analysis

The whole process of interview is to be recorded and the results are shown in the following tables (Table 8 and Table 9).

Table 8 Results of the Interview Study

Questions

Answers(40)

Students(33)

Percentage

yes

7

21.2%

no

26

78.8%

In immediate sentences

12

36.4%

In forward or backward context

21

63.6%

From the beginning of responding

5

15.2%

When you have filled in a large number of blanks

28

84.8%

Table 9 Results of the Interview Study

Items

Blanks(40)

Percentage

Chosen by the use of fixed collocation

13

32.5%

Chosen on the basis of grammar

5

12.5%

In the multiple-choice cloze tests, the number of students who have not read the entire passage before choosing the answers is 26, and only 7 students have read the passage to get the general idea before responding. 12 out of 33 students do not look for the right answers with reference to context, but rather decide in immediate sentences. Only 5 out of 33 students can use the context to look for the answers from the beginning of responding; other 28 students use the context when they have finished filling in a large number of blanks. There are some answers chosen with the knowledge of fixed collocations and grammar(with the ratios of 13/40 and 5/40 respectively).

The study on strategies for taking cloze test as in Table 8 and Table 9 has shown that only 21.2% of the students read the entire cloze passage before responding. It is found that 36.4 percent of the students do not use the forward and backward context for clues to fill in the blanks but rather guess on the basis of sentence-level reading. Most of the students (with a percentage of 84.8) do not use global clues until they have completed a large number of blanks in the cloze. In other words, it is easier to read the cloze passage once it has been partially completed and the respondents have some idea of what the text is about. The students’ statements show that, in deciding some answers, students do not need to read the context but to guess by using fixed collocations or to identify the correct items according to grammar. The percentage of using vocabulary and grammar is about 45%. It is well known that success in reading comprehension is significantly associated with success in getting the main idea and understanding important information calling for forward reading and backward reading. Thus, the research on strategies in taking cloze tests suggests that such tests assess sentence-level reading more than they measure global reading ability. A lot of strategies used in cloze procedure are related to vocabulary and grammar, so cloze procedure measures more than reading comprehension.

Judging from the subjects’ response, a conclusion can be drawn that the validity of cloze procedure as measurement of global reading skills is not sufficient.

4.3 Summary

In this chapter, the writer conducts a study on the response validity of the cloze test as measurement of reading skills by analyzing the strategies used by testees in the cloze tests. In order to understand the testees’ psychological process of filling the blanks, the writer has experimented with 33 students. In addition, face-to-face interviews of these subjects with recordings are also resorted to. The study on the strategies for taking the cloze tests has shown that, first, only 21.2% of the students read the entire cloze passage before responding; second, 36.4% of the students do not use the forward and backward context for clues to fill in the blanks; third, 84.8% of the students do not use global clues until they have completed a large number of blanks; fourth, 45% of the blanks are filled by using fixed collocation and grammar identification. The data analysis of the subjects’ strategies concludes that the validity of the cloze test as measurement of global reading skills is invalid.

Chapter Five Conclusion

The paper firstly reviews the development of the cloze test and the definition of the validity. Then the writer conducts a survey on the related researches on cloze procedure. The cloze test originated as a means of assessing the difficulty of a reading test and many researchers believe that the cloze test is a valid and economical way to test reading comprehension ability.

Rational-deletion cloze test and multiple-choice cloze test are usually seen in all kinds of tests in China. Which kind of cloze tests is appropriate for Chinese learners is very different in the test users’ eyes, therefore, the paper makes the face validity researches on the rational-deletion cloze test and multiple-choice cloze test from the testees’ angles in chapter two. In this chapter, 71 students are selected to finish the two rational-deletion cloze passages and the same multiple-choice cloze passages. After the cloze tests, a questionnaire is provided to the test takers to study language test anxiety.

The validity of cloze test as measurement of reading skills has long been a controversial topic in the field of pragmatics. In order to understand whether the cloze test is a highly valid way to test reading comprehension, the writer does researches by conducting a comparative study on the judgements from English teachers in Chapter three. 22 English teachers are asked to correct the papers of the rational-deletion cloze testees and during correcting the papers, they conduct the comparative judgements. The further study selects ten multiple-choice cloze tests to conduct continuing researches on the content validity of the cloze test.

Foreign researches regarding strategies for taking cloze tests concentrate on whether the cloze tests actually measure global reading skills. In order to understand what the cloze tests measure in Chinese context, in Chapter four, an interview questionnaire is designed to study on the strategies and responses used in the cloze tests by the students.

The researches have obtained three main results after analyzing the data collected. First, the quantitative analyses and comparisons have shown that, in China, rational-deletion cloze is not well accepted by the students, while the face validity of multiple-choice cloze is more reliable to the Chinese foreign language learners. Second, through the comparative study, the paper discovers that the content validity of cloze test as the measurement of reading skills is not valid. Third, after careful analysis of the students’ responding strategies by the interview of the subjects, the paper finds that, in cloze procedure tests, more word-level reading skills and immediate sentence contexts are used to fill in the blanks.

Finally, the limitations of this research should be pointed out. Students involved in this research are at the same stage of English learning, and the test samples only display a partial picture of skills used in cloze tests. In addition, there is still a lack of precise criteria for skill classifications, which may result in the overlapping of some skill categories. Therefore, the future research will manage to analyze tests and strategies with reference not only to college students but also to adults or high school students at different stages of English learning.

Bibliography

Alderson, J.C. Clapham, C and Wall, D Language Test Construction and Evaluation [M]Beijing: Foreign Language Teaching and Research Press & Cambridge University Press 2000

Arnold, J Affect in Language Learning [M] Beijing: Foreign Language Teaching and Research Press & Cambridge University Press 2000

Alderson, J. Charles. “The cloze procedure and proficiency in English as a Foreign Language”. Reprinted in Oller, J.W. Issues in Language Testing Research. Rouley Mass: Newbury House Publisher, Inc 1983

Bachman, L.F and Cohen, D Interfaces Between Second Language Acquisition and Language Testing Research[M] Beijing: Foreign Language Teaching and Research Press & Cambridge University Press2002

Brown, H.D Teaching by Principles: An Interactive Approach to Language Pedagogy [M] Beijing: Foreign Language Teaching and Research Press 2001

Brown, J.D. “ A Closer Look at Cloze: Validity and Reliability” Reprinted in Oller, J.W Issues in Language Testing Research. Rouley Mass: Newbury House Publisher, Inc 1983

Cohen, A.D Strategies in Learning and Using a Second language [M] Beijing: Foreign Language Teaching and Research Press 2000

Carroll, D. W Psychology of Language [M] Beijing: Foreign Language Teaching and Research Press 2000

Chihara, T., J. W. Oller, Jr., K. Weaver, & M. A. Chávez-Oller. 1977. Are Cloze Item Sensitive to Constraints across Sentences? Language Learning, 27(1), 63-73.

Ellis, R The Study of Second Language Acquisition [M] Shanghai: Foreign Language Education Press 2000

Garman, M. Psycholinguistics [G] Beijing: Peking University Press 2002 & Cambridge University Press 2002 Learning Asia 2000

Henning, G A Guide to Language Testing: Development, Evaluation and Research [M] Beijing: Foreign Language Teaching and Research Press & Heinle Thomson Learning Asia 2001

Hughes, A Testing for Language Teachers [C] Cambridge University Press 1989

Harmer, J How to Teach English [M] Foreign Language Teaching and Research Press 2000

Harrison, A. A Language Testing Handbook. London: Macmillan Press London 1983.

Heaton J.B, Writing English language tests [M] Beijing: Foreign Language Teaching and Research Press, 2000

Homburg, T. J. & M. C. Spaan, 1981, ESL Reading Proficiency Assessment: Testing Strategies In M. Hines& W. Rutherford (eds.), On TESOL, 81 (pp. 25-33). Washington, D C: TESOL

Ingram, E. Basic Concepts in Testing. In J.P.B Allen and A. Davies 1977

Johnson, K Introduction to Foreign Language Learning and Teaching [M] Foreign Language Teaching and Research Press 2002

Klein Braley, C. “A cloze-up on the C-test: a study in the Construct Validation of Authentic Tests”. Language Testing. 2(1): 76-104 1985.

Madsen, H. S. Techniques in Testing. London: Oxford University Press. 1983

Oller, J. W. (ed). Issues in Language Testing Research. Rouley Mass Newbury House Publisher, Inc. Pp203-205 1983

Tarone, E and Yule, G Focus on the Language Learners Oxford University Press 1989

Ur, P Course in Language Teaching Practice and Theory [M]Foreign Language Teaching and Research Press & Cambridge University Press 2000

Wood, R Assessment and Testing: A Survey of Research[M]Beijing: Foreign Language Teaching and Research Press & Cambridge University Press 2001

陈亭华,语篇分析与中学“完形填空”试题设计 《江苏外语教学研究》,1995(2):67-70

柴菊香,英语完形测试的设计 《周口师专学报》,1998, (1):62-64

陈辉,《大学英语四级考试王长喜英语预测试卷》2003,北京:学苑出版社

何国维,华维芬,完形测试中的评分问题 《现代外语》,1995,(2):31-32

胡凤明,试论完形填空的效度和信度 《唐山师范学院学报》,2002,(1):72-74

蒋静泓,完形填空的解题技巧 《全国优秀英语学术论文集》,1995,(2):865-869

刘润清,韩宝成,《语言测试与它的方法》1991,北京:外语教学与研究出版社

舒运祥,《外语测试的理论和方法》1999,上海,世界图书出版公司

徐昂,多项选择完形填空与传统完形填空信度和效度之研究 《外语与外语教学》,1998,(2):26-28

袁铮,完形填空在英语测试和教学中的作用 《上海师范大学学报》,1997(1):142-145

杨乾龙,完形填空的出题原则与应用 《上海教育学院学报》,1998,(4):85-87

杨桂英,完形填空的语篇概念 《青岛大学师范学院学报》,1999,(2):65-66

杨惠中& C.Weir,《大学英语四六级考试效度研究》,1998,上海:上海外语教育出版社

〈日〉Sandra S.Fotos著 金木编译. “完形填空能作为整体语言能力测试手段 吗?”,《国外外语教学》,1993

朱亚夫,完形测试及其设计应注意的问题 《广州师院学报》,1995,(1):88-92

邹申,《英语语言测试》--理论与操作,1998,上海:上海外语教育出版社

张彦斌,戴炜华,《理工类全国专业技术人员职称英语等级考试指定用书》2002,辽宁大学出版社

张娓莹,《八大名校大学英语考前演练试卷》2002,国际文化出版公司

朱跃,《英语与社会》1999,合肥:安徽大学出版社

Appendixes

Multiple-choice cloze tests for further study of Chapter 3

Appendix 1: Five passages are selected from Wang Changxi Model Tests of Forecast 2003 CET-4

One

Traditional marriage in Britain is currently in a disturbance. Not only id the divorce rate rising, 1 the rate at which people marry is falling. Living together is more popular than 2 before. The shape of the family is now no longer one man, one woman and their children. Instead, there are 3 numbers of families which include step-parents, half-sisters and brothers, or merely on lonely parent coping 4 her own.

Compared with other countries, Britain is still conservative in its marriage patterns. In America, the divorce rate is 5 more shocking. Two out of five marriages 6 in divorce. In Sweden living together is now more popular than marriage among couples in their early twenties and a similar 7 seems to be emerging in Denmark.

Although this is happening on a smaller 8 in Britain, it has not yet become such a marked trend. But 9 we do follow the American and Scandinavian patterns, the 10 will see many more couples living together before marriage—and even more divorce.

Interestingly enough, it is women 11 than men who get a divorce in the courts. Seven out of ten divorces are 12 to the wife. Divorce, of course, only reflects the 13 winding up of a marriage which may have 14 broken up long before. The partner who asks for divorce may not be the partner who 15 up the marriage. Women usually have more to gain from the courts in the way of money, rights 16 the home, and child maintenance. 17 there is also a certain unequal proportion in one of the grounds that the sexes choose for divorce. The grounds 18 unreasonable or cruel behaviour are overwhelming, chosen by ten 19 more women than men. Does this mean that women will 20 up with less than they used to?

1. A and

B but

C however

D yet

2. A ever

B never

C much

D none

3. A decreasing

B growing

C diminishing

D enlarging

4. A with

B on

C through

D of

5. A still

B even

C rather

D ever

6. A end

B break

C separate

D slip

7. A figure

B form

C model

D pattern

8. A range

B size

C scale

D number

9. A unless

B if

C when

D while

10. A result

B consequence

C future

D family

11. A other

B rather

C more

D better

12. A granted

B judged

C sentenced

D given

13. A lawful

B legal

C justifiable

D sociable

14. A efficiently

B effectively

C actually

D indefinitely

15. A broke

B cause

C call

D burst

16. A to

B for

C with

D on

17. A While

B Therefore

C So

D But

18. A on

B that

C of

D which

19. A ratios

B times

C rates

D volumes

20. A put

B stand

C come

D take

Two

But why should anyone want to build a computer that can recognize a person by 21 ? One obvious use of such a device would be in security systems, 22 the computer 23 whether a person really is who he 24 to be. For this task, the computer would not need to be very large, 25 it would only have to 26 with a limited number of people using perhaps one particular phrase. 27 wanting to gain 28 must say the password into the computer’s microphone, 29 if his voice is recognized he is admitted.

The computer which listens to voice measures two 30 : the frequences are. Intensity—how loud or soft the voice is, and 31 how high or low the frequences are. But these systems have not yet been perfected and are far from foolproof. One 32 developed at the Bell Laboratories in American has a typical failure rate: it rejected people incorrectly nine per cent of the time 33 mistakenly accepting 10 percent of those who should have been rejected. This was in spite of the fact that the speakers were expected to be 34 and not to drown their voices. Even 35 , certain individuals were rejected 45 percent of the time, suggesting that the pitch and 36 of their voices varied 37 one occasion to another 38 than other peoples. 39 the speakers had been allowed to 40 their voices, the results would have been even worse.

21. A sound

B tone

C voice

D noise

22. A which

B where

C what

D how

23. A identify

B intensify

C verify

D verify

24. A utters

B claims

C appears

D alleges

25. A since

B because

C for

D so

26. A handle

B settle

C interfere

D deal

27. A Everyone

B Someone

C Anyone

D None

28. A permission

B access

C admission

D acknowledgement

29. A and

B but

C or

D yet

30.Adifferences

B variables

C alternatives

D separation

31. A height

B rhythm

C pitch

D wave

32. A being

B was

C been

D is

33. A when

B which

C while

D where

34. A coordinate

B cooperative

C correspondent

D assistant

35. A though

B if

C so

D as

36. A density

B intensity

C extent

D intention

37. A with

B from

C in

D on

38. A rather

B other

C more

D better

39. A Unless

B Though

C If

D As

40. A pretend

B disgust

C discharge

D disguise

Three

If ever you are called 41 to introduce a speaker, it would be well for to bear in 42 that you have a responsibility to do 43 than simply get up, recite a few facts, and then sit down. You have at least five major responsibilities.

It is really your responsibility to 44 the tone for the speaker, to establish close relationship with his audience, and 45 the audience that the speaker is a person worth 46 to, In accomplishing this broad goal, you may also be able to ease the 47 and thus relieve your speaker of 48 feelings o