PROCEEDINGS OF THE OKLAHOMA ACADEMY OF SCIENCE Volume 63—1983

{Page 78}

WATER RESOURCE KNOWLEDGE ASSESSMENT OF COLLEGE-BOUND HIGH SCHOOL GRADUATES

Ted Mills

Department of Curriculum and Instruction, Oklahoma State University, Stillwater, Oklahoma 74078

A Water Resource Knowledge Assessment test was developed to determine the quantity and kind of knowledge possessed by recent high school graduates in Oklahoma. Test validity and reliability was determined with assistance from thirty-one test reviewers.

Student responses indicated the level of knowledge to be low, particularly in the areas of contemporary issues, resource management, and historical influence. Mean score comparison of male/female, urban/rural, and science/nonscience major subgroups—though not significant—favored males, graduates of urban high schools, and nonscience majors. However, item analysis showed that for all test items where significant differences existed, four test items favored females and nine test items favored nonscience majors.

INTRODUCTION

Water is an essential component determining the quality of our lives. Growth of our state's and nation's population and standard of living is placing an increasing demand on this limited and often variable resource, while its effective management is hampered by a lack of public understanding. An important part of that which influences wise public decision making is the collective factual background information possessed by the public.

The public schools make a major contribution to the public's basic water knowledge. The extent to which the schools prepare the voting public for management of resources, water in particular, is critical to wise resource management. The analysis of water resource education outcomes in the public schools is the first step in building a sound public awareness program.

The primary purpose of this study was to: (a) develop a water knowledge assessment instrument to test recent high school graduates, and (b) determine the validity and reliability of the instrument. The secondary objectives were to: (a) identify the quantity and kind of factual water information the population possesses, and (b) identify significant differences in test scores within various subgroups of the population under study, i.e., (i) male/female, (ii) urban/rural, and (iii) science major/nonscience major.

Initial activity in the development of the Water Resource Knowledge Assessment Instrument (WRKA) included a thorough ERIC (Educational Resources Information Center) and library search for an existing instrument measuring water-related knowledge which would be suitable for high school students in Oklahoma or the whole Southwest. Such a test could not be found.

To develop the water assessment instrument, it was necessary to research science texts, government documents, theses, journals, and general environmental science texts to identify potentially useful test items. In some cases it was necessary to modify test items obtained from these sources; in other cases, items were created.

The content balance of the test was in part determined by using "A Conceptual Scheme for Studies of Aquatic Environments" (1) and "A Conceptual Framework for Water Education" (2). These two outlines of the structures of knowledge unique to water provided guidance in selecting representative test items from each of numerous concept categories.

Validity

The validity of a test is commonly defined as the degree to which a test measures what it is designed to measure within a given population. Content validity is based on what qualified professionals can determine by examining the test itself. The 62

{Page 79}

initial items were sent to 44 water resource experts in Oklahoma for their evaluation of each item and the content balance of the instrument. The 31 experts who responded can be categorized by occupation as follows: university personnel involved in water research (17 persons), including personnel from Oklahoma State University, Northeastern Oklahoma State University, Southwestern Oklahoma State University, Cameron University, East Central Oklahoma State University; employees of state and federal agencies (6 persons), including Oklahoma Water Inc., U.S. Army Corps of Engineers, U.S. Geological Survey, and Oklahoma State Department of Education, Oklahoma Geological Survey, and Oklahoma State Department of Health; and high school science teachers (8 persons) from the following communities: Altus, Tishomingo, Lawton, Tulsa, Kingfisher, Moore, Cleveland, and Oologah.

To facilitate reviewer input, specific criteria for test evaluation were specified in a cover letter. For each test item, the expert was to indicate if the item was (a) acceptable, (b) acceptable but with revision necessary, or (c) not acceptable. If revision was deemed appropriate, a suggested revision was requested. The evaluators were advised that the test was to be suitable for measuring the knowledge of recent high school graduates.

Responses from 31 experts resulted in the revision and deletion of items as well as the inclusion of additional items. The resulting 53 items were categorized using the outline of goals and conceptual areas specified in "A Conceptual Framework for Water Education" (2). The distribution of test items across the conceptual framework categories is as follows.

Of the 53 test items identified by consensus of the panel of experts, 46 dealt with facts and concepts and seven with personal opinion.

To further refine the initial test items, they were administered to eleven senior senior high school students just prior to their graduation in May, 1980. Informal feedback from this group was used to further clarify the meaning of selected items.

Reliability

Test reliability, the extent to which the test is consistent in measuring what it purports to measure, includes test dependability, stability, and relative freedom from errors in measurement. To determine the reliability of the 46 factual test items (seven personal opinion items were necessarily deleted), they were administered to 273 college students during the first week of freshman biology. One hundred fifty-nine members of this group were recent high school graduates. Students were instructed not to guess, course grades were not involved, and no names were to be put on response sheets. A reliability coefficient of 0.77 (see Table 1) for the 46 items used was obtained with the population of 159 recent high school graduates. The value 0.77 is above the value required to estab-

{Page 80}

lish test reliability. (Copies of the WKRA Test are available from the author upon request.)

The following analysis concerns the response of 159 recent high school graduates enrolled in their first week of university coursework.

The highest possible score on the WRKA test is 46; the lowest is 0. The mean score of 18.60 reported in Table 1 indicates that the sample population found the test difficult. The mean difficulty, an index representing the percent of the group answering test items correctly, also suggests that the water-related questions were difficult for this population to answer.

Table II shows the distribution of scores, from a high of 32 to a low of 5, out of a total possible score of 46. Approximately 50% of the population scored 19 or less.

Type of Knowledge by Conceptual Categories

Each of the test items had previously been assigned to a category within a conceptual framework for water (2). To determine the population's strength in each conceptual category, the percent of correct responses for all items in a category were calculated. The percent of correct responses by category indicates the relative knowledge possessed by students in each area. Table III shows the relationship existing between percent correct and responses by category.

Differences in Test Scores Between Subpopulations

The 159 subjects were identified by sex, size of high school graduating class, and major fields. To determine if significantly different scores existed within the above subpopulations, x² and t-test values were determined.

Female/Male, Chi square comparison of female-male responses by quartile resulted in a significant difference at the 0.01 level. There was a tendency for females to score either high (Q₁) or low (Q₄), while a majority of male scores fell in Q₂ and Q₃. Table IV shows the percent of correct responses by quartile.

{Page 81}

There was no significant difference between female and male mean scores. However, t-test analysis by item showed that a significant difference in mean scores existed on four test items. On all four items the mean response of females was significantly higher than that of males. For males, however, the total mean score was slightly higher than for females.

Urban/Rural. Urban and rural categories were defined by size of high school graduating class, with more than 100 defined as urban and less than 100 defined as rural. No significant differences were identified using x² and t-test analyses. Although not significant, the mean score of the urban population (N = 112) was higher than that of the rural population (N = 33).

Major Field. No significant difference in response patterns x² values) or overall mean scores (t-test) existed between science majors and nonscience majors. The average score for non-science majors was higher than that for science majors. By item, t-test analysis showed significant differences in nine items. The mean response of non-science majors was significantly higher for all items where significant differences existed.

The purpose of this study was to develop a valid, reliable Water Resource Knowledge Assessment Test for recent high school graduates. Additional objectives included a preliminary survey of high school students to determine the kind and amount of their water knowledge, and a comparison of female/male, urban/rural, and science/non-science major responses.

The WRKA test has potential value in the gathering of baseline data which can be used to (a) determine the current general level of water knowledge and (b) evaluates the results of state-wide curricular efforts in water education.

The 159 recent high school graduates taking the assessment scored low. The mean score of 18.60, with a range of 5 - 32, out of a possible 46 indicates that these students had not been well prepared in the area of water resources.

The categories in which students scored highest, as evidenced by the percent of correct responses for the relevant test items, were hydrologic cycle, chemical and physical properties, and influence on the physical world. The fact that these topics are often included in the high school curriculum may explain the higher scores. The three categories where scores were the lowest were contemporary issues, water resource management, and the historical influence of water. These topics are evidently not emphasized in the public school curricula.

Female scores tended to cluster in the upper and lower quartiles, while male scores clustered in the second and third quartiles. Females scored significantly higher than males on test item numbers: 11—the direction in which water moves across Oklahoma; 38—phytoplankton in an aquatic food chain; 41—the cause of acid rain; and 43—the relative oxygen requirements

{Page 82}

of fish. The overall mean score for males was slightly higher than that for females.

Urban students performed slightly better on the test than did rural students. T-test comparison by item showed a significant difference on items 16 and 36. Urban students tended to know the amount of water required to flush a toilet (item 16), whereas rural students scored significantly higher on item 36, which dealt with using the coliform index as a measure of water purity.

The mean score of nonscience majors was higher than that of science majors, but not significantly. T-test analysis by item showed that the nonscience majors scored significantly higher on 19% of the test items. These nine test items fell in the following conceptual categories.

The test performance of the nonscience majors offers no clear information that might explain why this group performed better than the science majors.

Recommendations

The small size of the population and the sampling procedures employed do not allow generalization beyond the population studied. However, "this initial study indicates that a void may exist in the water education of college-bound public school youngsters. It points out a lack of knowledge in the areas of current water issues, water resource management, and the influence of water in shaping our past. These concepts embody information required by those who would assume leadership roles in the state and nation. The fact that the college-bound science major knows less than the non-science major supports the need for further study and analysis of the water-related knowledge possessed by those completing high school.

It is recommended that:

1. the WRKA test be further refined, using non-college-bound as well as college-bound populations;
2. the refined WRKA test be administered to a random sample of a large segment of recent high school graduates;
3. the experimental study be designed and conducted to verify and determine the possible cause for significantly different scores of females/ males, urban/rural students, and science/nonscience majors in some conceptual areas;
4. the public school curricula be analyzed to determine the extent to which water-related concepts are infused in the curricula; and
5. instructional materials be developed and distributed for use with elementary, middle, and senior high school students. These materials should emphasize current issues, water resource management, and the historical influence of water in Oklahoma.

The author gratefully acknowledges the assistance given by the 31 reviewers of the WRKA Test and Ms. Jonell Roland. Support for this study was provided by the Oklahoma Water Research Institute, OSU.

REFERENCES

1.   The Need for Marine and Aquatic Education: To Inform Americans About the World of Water, Sea Grant. University of Delaware, Newark, 1978, pp. 20-21.

2.   A Conceptual Framework for Water Education: An Educator's Guide to Goals, Concepts, and General Objectives for Curriculum Development. Water and Man Inc., Salt Lake City, 1979, p. 4.