EXPLORING AUTHENTIC INQUIRY: FACTORS THAT INFLUENCE STUDENTS’ LEARNING OF THE PROCESS OF SCIENCE

(1) INTRODUCTION

    This proposal is a five-year plan of research and instructional development investigating undergraduate biology students’ learning of the process of science through an inquiry-based lab exercise.
    Understanding the process by which scientific knowledge is generated is an essential part of many science curriculum standards and frameworks (AAAS, 1993). An understanding of this process has also been found to be an important component of both learning science content (Edmondson, 1989; Songer & Linn, 1991; Edmondson & Novak, 1993) and teaching science content (Brickhouse, 1990). Unfortunately, as Ryder et al., (1999) put it, "For students, the world of professional science is largely outside their everyday experience." (p.201) As a result, most students have a poor understanding of science as a process (Cotham & Smith, 1981; Kuhn, 1989; Grosslight et al., 1991; Lederman, 1992; Roth & Lucas, 1997). Many researchers have identified a core group of conceptions and misconceptions about the NOS (Nature of Science). These are listed below:
Issue Common Misconceptions Reference(s)
The scientific method involves creativity and originality. 1. the scientific method is a rigid form
2. creativity and originality are not involved in science		 Abd-el-Khalik et al. (1998)
Meichtry (1993)
Pomeroy (1993)
Edmondson (1989)

Scientific knowledge is tentative, revisionary, paradigm-bound, and parsimonius		 3. theories can only be replaced, not revised
4. correctly done experiments yield unchangeable, absolute, objective facts
5. old theories do not change; new theories only build on old
6. science involves ruling out all alternative hypotheses
7. scientific theories are limited to observable phenomena		 Lederman & Druger (1985); Lederman (1986); Lederman & O'Malley (1990); Lederman (1992); Lederman (1999)
Tamir (1994)
Abd-el-Khalik et al. (1998)
Cotham & Smith (1981)
Meichtry (1993)
Palmquist & Finley (1997)
Pomeroy (1993)
Edmondson (1989)
Ryder et al. (1999)
Scientific consensus is determined socially 8. consensus determined by conclusive evidence alone Ryan & Aikenhead (1992)
Roth & Lucas (1997)
Ryder et al. (1999)
Scientific models are constructed by scientists  9. scientific models are copies of reality
10. scientific models are discovered by scientists		 Ryan & Aikenhead (1992)
Palmquist & Finley (1997)
Grosslight et al. (1991)
Ryder et al. (1999)
Edmondson (1989)
Theories are explanations with a high degree of confidence;
Laws are descriptions with a high degree of confidence;
Hypotheses are tentative descriptions or explanations;		 11. progressive hierarchy:
12. accepted hypotheses become theories
13. theories eventually become laws		 Ryan & Aikenhead (1992)
Abd-el-Khalik et al. (1998)
Brickhouse (1990)
The role of experiment and observation in science 14. anomalous results should lead to theory revision*
15. science is more about making accurate measurements than observing and experimenting		 Tamir (1994)
The distinction between observation and inference. 16. confusion and or intermingling of observation and inference Abd-el-Khalik et al. (1998)
Kuhn (1989)

* The authors of this study (Tamir, 1994) viewed this as a correct conception; I and many others (for example Chinn & Brewer (1993) view it is only one of many possible responses to anomalous data.

    To remedy this, most standards and frameworks for science education emphasize that students need to engage in scientific inquiry themselves in order to understand the process of science. In order to apply these standards in the classroom, it is necessary to have answers to the following questions:

  1. Process: Can undergraduate students perform scientific inquiry within the spatial and temporal confines of the usual teaching laboratory setting? Research is a full-time enterprise requiring a wide array of techniques, equipment, and expertise; in contrast, most teaching labs have limited equipment and meet only once a week for three hours. Teaching labs can therefore only approximate the experience of scientific research. These studies will examine how closely the students’ experience in a teaching lab can match that of scientific researchers.
  2. Outcome: To what extent does performing a specific scientific inquiry exercise help students to understand the process of science more generally? Professional scientists come to understand the process of science by practicing scientific inquiry over a period of years; we will examine the parts of this practice that can be effectively acquired in a single lab exercise.
  3. The TA’s Role: How do TA teaching behaviors influence process and outcome? The Teaching Assistants (TAs) who teach the lab sections are active and dynamic participants in the process that the students experience. Their understandings, beliefs and behaviors will play a crucial role in the lab.
  4. Relevant Factors: How do variations in the authenticity of the activity affect the students’ learning of the process of science? Scientific research is unpredictable and time-consuming; teaching labs must make effective use of limited time. We will examine the features of a lab exercise that are crucial to its success.
    This project will build on previous work to explore answers to these questions in the context of a technically simple but educationally rich lab exercise, the Red and White Yeast Lab. The findings of the studies presented in this proposal will contribute to the understanding of scientific reasoning in undergraduate students as well as to an understanding of the design and evaluation of inquiry-based laboratory exercises. As an exemplar of these principles and practices, this project will produce a thoroughly-tested laboratory exercise and accompanying teacher training materials designed to increase students’ understanding of the process of science.
     This proposal begins with a description of the context of these studies: UMass Boston, the students, and the course. I will then describe the Red and White Yeast lab exercise and what makes it a uniquely useful model system for investigating inquiry-based labs. Finally, I will describe the studies I will undertake as well as their significance and expected outcomes.

(1.1) Institutional context: The University of Massachusetts Boston and General Biology I (Bio 111)
     The University of Massachusetts Boston (UMB) is the only public university in the Boston area. UMB has a strong commitment to educating students who are poorly served by the other colleges and universities in the Boston area. Its student body is quite diverse; of the 13,000 students, 8,900 are undergraduates, 53% of whom are female and 31.5% are minorities; the average age of the undergraduates is 28. UMB therefore provides a rich environment to explore differences in student learning. The university also has an undergraduate and a graduate teacher education program as well as a graduate program in Critical and Creative Thinking, both of which will be sources of research assistants for this project.
     The Biology Department is part of the College of Arts and Sciences and has 23 full-time faculty members. All Biology faculty have active research programs and teaching responsibilities. The undergraduate program has a strong emphasis on laboratory experiences; 27 of the 51 undergraduate courses offered have a laboratory component. The department offers BS and MS degrees in Biology and offers Ph.D. degrees through the program in Environmental Biology
     General Biology I (Bio 111) is a one-semester course with three 50-minute lectures and one 3-hour lab per week; it serves roughly 250 students each fall. The students are biology, nursing, and human performance and fitness majors as well as non-degree students who are typically completing their pre-medical requirements. I give the lectures, write the exams, and design the lab exercises. Each lab section involves 15-30 students and is taught by graduate teaching assistant (TA). I train and supervise the TAs and have written a TA manual with specific lesson plans for each lab session. Overall, the course emphasizes problem solving over memorization, thus I cover a limited number of topics and several lab sessions are devoted to problem-solving exercises. The syllabus includes genetics, cell biology, biochemistry, and molecular biology. I also place particular emphasis on the history and process of science.

(1.2) Authentic Experimentation and the Red and White Yeast Lab
 In order to prepare the students for discussions of the process of science later in the course, I have designed the first lab of the semester to be a practical experience of the process by which scientific knowledge is generated. This lab, the Red and White Yeast Lab, has many of the features of authentic experimentation, as described by Chinn & Malhotra, (in preparation).
    Chinn and Malhotra make a distinction between authentic experimentation, experimentation as practiced by scientists doing research, and simulated experimentation, experimentation as usually practiced in teaching and educational research laboratories. Several of these differences relevant to this project are summarized in the table below (adapted from Chinn & Malhotra, in preparation).
 
 
Authentic Experimentation Simulated experimentation
Variables many, of unknown nature, constructed by experimenters few, known in advance
Controlling Variables not obvious what is to be controlled, multiple controls needed straightforward, single controls
Planning experiments creative, requires inventing a model and choosing an appropriate analogy straightforward, no analogies involved
Flaws in experiments much concern about experimental errors little concern
Responses to anomalous data data are regularly and rationally discounted data cannot be rationally discounted
Role of multiple studies many types of studies with conflicting results one type of study with consistent results
Role of others’ research careful reading of peer-reviewed articles is essential little or no role for this reading

    In addition to Chinn, I would add that, while revision and refinement of methods has little or no role in simulated experimentation, it is an essential part of authentic scientific experimentation (Kuhn, 1962; Collins & Pinch, 1993). In addition, while the complex nature of peer persuasion ? the degree to which different investigators find different experiments convincing ? is not typically a part of simulated experimentation (Peterson & Jungck, 1988), it is an essential part of authentic experimentation (Kuhn, 1962; Collins & Pinch, 1993). An important component of peer persuasion is the intellectual context, the set of inter-related theories, accepted methods, and ‘plausible’ theories that surrounds any experimental result. This context is usually incomplete in simulated experiments. The studies in this proposal will examine the roles of several of these differences in the process and outcome of this lab exercise.
    During the Red and White Yeast lab, the students develop their own hypotheses and models, design their own experiments based on variables that they define, and interpret their own data. They argue about data and the interpretation of data and use these arguments to refine their experiments. In this experimental environment, the students construct scientific knowledge about a particular phenomenon. I have been using and developing this lab for the past 5 years with introductory-level biology students at MIT and UMB, as well as with students in the MIT teacher education program. A description of this lab will be published this fall (White, in press) and this lab will be disseminated as part of the NSF-funded ResearchLink 2000 project.

(1.4) Description of the Red and White Yeast Lab
     The Red and White Yeast Lab spans three 90-minute lab meetings over three weeks. It begins with a biological phenomenon: a patch of an engineered strain of Bakers’ Yeast (Saccharomyces cerevisiae) growing on solid medium in a petri dish. This patch has a white edge and a red center (see below; the image is approximately 50% actual size). More information about this system, including color pictures, can be found at the ResearchLink 2000 www site: http://intro.bio.umb.edu/RL2000/R_W_Yeast/
    Working in groups of three, the students design, perform, and interpret two rounds of experiments using fresh plates of sterile medium and sterile toothpicks. Their objective is to account for why the center is red and the edge is white. The emphasis in not on necessarily making a complete explanation but on confirming or disconfirming a part or parts of their explanation. The purpose is to have the students experience the process by which people can increase or decrease confidence in hypotheses.
     The first lab meeting begins with a very brief description of the biology of yeast and a description of how the patch was prepared. The students are told that yeast are microscopic cells that reproduce asexually. Each yeast cell divides to give two more-or-less identical daughter cells approximately every 2 hours. They are also told that all the cells on the patch are descended from a single yeast cell and that some of these cells were spread on the plate in a thin layer which has grown for a week to form the patch they see now. The students work in groups of three to come up with hypotheses to explain the phenomenon. The class then discusses the various hypotheses and the TA records them on the board. At this stage, with only the patch to look at, their hypotheses are often poorly-defined and frequently are expressed as questions rather than explicit hypotheses. In a typical semester, there were at least 17 different hypotheses expressed by the 250 students. Some typical initial responses are listed below:

     The students are given sterile toothpicks and fresh plates with nutrient medium, and they discuss possible experiments in their groups. They then discuss these experiments as a class. Although the TAs are told to encourage explicit linkage between hypotheses and experiments, at this stage the two are often not clearly linked. In a typical semester, there were at least 13 different experiments designed by the 250 students. Typical Round 1 experiments include:     Virtually any experiment the students devise will give results that will bear on the discussion that follows.
     After the students’ experimental plates have grown for a week (the same time that the original patches were grown) they are returned to each lab group at the beginning of the second meeting. Students discuss their results in their lab groups and then the TA leads a discussion with the class as a whole. The TAs are encouraged to guide the students to confront hypotheses with experimental results and to have the students exchange data among groups. At this point, there are frequently disagreements over experimental results and interpretation of data. These disagreements are used to inform the second round of experiments which the students design in groups, discuss as a class, and perform. These second round experiments tend to be more carefully controlled replicates of Round 1 experiments that the class has found compelling in addition to experiments of novel design.
     At the final meeting, one week later, the students look at the plates from their second round of experiments and the class discussion focuses on reaching consensus about what they’ve found. The students then write a lab report where they describe a situation where they experienced the interaction between hypothesis and experiment; a situation where their hypothesis changed because of an experimental result, or where a result motivated the design of a new experiment.
    Although the path of the discussion differs from lab section to lab section, most sections discover one or more of the following:     As an example, the following series of events are one of the many threads of the discussion that occurred in one lab section of Bio 111 at UMB.
At the initial hypothesis stage, many students favored ‘red is starving’. During the discussion of the students’ initial models, the students refined this to, "There are lower nutrient levels in the crowded interior of the patch and yeast cells turn from white to red when they are deprived of nutrients." In the first round of experiments, several groups tested the effect of complete starvation by placing samples of red and white on the lid of the petri dish, where there are no nutrients. Other groups attempted to re-create the patch by spreading large samples of either red or white on fresh medium.
At the second meeting, the students observed that, on the lid, the red had remained red and the white had remained white. One group was immediately convinced that this refuted the ‘red is starving’ hypothesis, since the white sample, which was clearly starving, failed to turn red. Others found this less convincing and argued that the dramatic starvation on the lid was somehow different than slow nutrient deprivation on the medium. This disagreement was partially resolved by the results of the re-creation experiments. In these cases, the students observed that large samples of red had grown into mostly white patches with a little red in the center, while even large samples of white had grown into patches with an entirely white center. Some students argued that these results provided further refutation of ‘red is starving’ since the large white patches had no red center. While many were convinced, the remaining dissenters cited differences between the ways in which the large white patch and the original patch were produced. They focussed on the size and thickness of the patches and on how the cells were spread on the medium. During this discussion, one group showed a patch which they had prepared from a large sample of white ? in contrast to the others, their patch had pinkish tinges in the center. This led to further discussion, with some saying that this result showed that large patches of white could have a red center, supporting ‘red is starving’, while others argued that the pink patch had to be due to accidental contamination of the white sample by red cells. As a result, many groups designed experiments for round two that involved multiple carefully-prepared samples of pure white.
    At the third meeting, the repeated samples of white had grown to produce patches of only white, and many were convinced by this that ‘red is starving’ had been refuted. The dissenters, reduced in number by the results of these and other experiments, continued to argue that the re-constructed patches were somehow different from the original patch and thus would not be expected to turn red at the center. At this point, the students could not move forward without a more complete understanding of the intellectual context of their hypotheses and results. In addition to a greater knowledge of yeast biology, they needed to know the limits of their techniques and the range of plausible hypotheses. Lacking these, the discussion went as far as it could and ended with a general consensus and an ‘agreement to disagree’.
    This sequence of events shows many of the features of authentic experimentation not found in other studies of scientific reasoning. Other studies have been based on, for example, finding patterns in number triples (Gorman, 1989), determining the rules of behavior of a simple robot (Klahr et al., 1993), and discovering the mechanism of gene regulation (Dunbar, 1993). In each case, variables and experimental treatments were pre-defined, the analysis of results was straightforward, and the "scientific truths emerged unproblematically from the experimental data" (Ryder et al., 1999, p. 203). In contrast, the students’ experience with the Red and White Yeast Lab was much more rich and complex. For example, the variable ‘level of starvation’ was constructed by the students themselves and they were obliged to invent experiments to vary it (samples on the lid, large patches, etc.). Conflicting results led to discussions of experimental error and refinements of technique. Combining results of others’ work was an essential component of model-building and persuasion of peers was not simple and straightforward.

(2) FIVE-YEAR RESEARCH & TEACHING PLAN
     This study will explore the three questions from the first page of this proposal in the context of the Red and White Yeast Lab. In the first study, we will adapt criteria from previous work to devise and test measures of the process that the students experience as they perform this lab (Question 1) and the outcome of this experience in terms of their changing conceptions of the process of science (Question 2). The remaining studies will use these measures in controlled experiments to evaluate the importance of different components of authentic scientific inquiry on the process and outcome (Question 4).
     Finally, we will combine the results of these studies of process and outcome to examine how the two are connected. Previous studies have shown that students’ images of the process of science depend on their particular experiences with science and scientific research (Ryder et al., 1999). This study was unable to examine the coupling between experience and understanding because of the heterogeneity of student’s experiences and small sample size. In our studies, because we will be able to examine these four issues both in terms of process ? what the students actually did, and of outcome ? what they learned from what they did, we will be able to study this interaction in detail.

(2.1) Themes that apply throughout the project

(2.1.1) Analyses of Process and Outcome.
     Groups of students will be video- and audio- taped as they perform the lab exercises. The resulting tapes will then be transcribed as a record of the process that the students experienced. As a measure of outcome, we will use clinical interviews similar to those used in previous studies of students’ understandings of science (Grosslight et al., 1991; Ryder et al., 1999). Each Subject will be interviewed before the yeast lab and again after its conclusion. We will present students with several open-ended stimulus questions based on the multiple-choice questions used by Cotham & Smith, (1981) as well as versions of the interview questions used by Ryder et al., (1999) and Edmondson, (1989). Our interview questions will allow the students to choose the context and examples used in their responses; post-interviews will also contain some questions specifically referring to the Red and White Yeast Lab. The transcripts of classroom sessions and interviews will be analyzed in terms of the following four issues that have been highlighted by previous research on students’ understanding of the process of science:
  1. What is the nature of scientific ‘proof’? There is general agreement among philosophers of science and science educators that scientific knowledge is ‘tentative and revisionary’ (Cotham & Smith, 1981) and thus, theories can never be proven conclusively. However, several pencil and paper studies (Cotham & Smith, 1981; Lederman, 1992; Roth & Lucas, 1997) have shown that many university students believe that theories can be ‘proved conclusively’, given sufficient data. These findings may be confounded as a result of the confusion between lay usage of the word ‘prove’, which implies conclusive proof, and scientists’ use of the word ‘prove’, which implies a greater degree of uncertainty. This confusion was highlighted when students were interviewed (Lederman & O'Malley, 1990) and it was found that while most students stated that hypotheses could be ‘proved’, their definitions of ‘proved’ included varying degrees of certainty. Some studies have shown that students can come to understand the role of uncertainty in scientific knowledge (reviewed in Lederman, 1992), although this misconception is persistent in many students even at the college level (Edmondson, 1989). Uncertainty of results and conclusions is an integral part of the Red and White Yeast Lab; we will track students’ expressions of and responses to uncertainty during the lab. In the pre- and post interviews, we will ask students to explain their understanding of the nature of scientific proof.
  2. How do hypotheses and experimental data interact? In contrast to our current understanding of the process of science, many students believe that hypotheses arise from the data via an inductive method rather than that hypotheses are created to account for data (Cotham & Smith, 1981). This misconception can lead to fundamental misunderstanding of the role of hypotheses and models in scientific reasoning (Grosslight et al., 1991). In addition, in studies of children and adults, Kuhn, (1989) found that many subjects were unable to clearly distinguish between hypotheses and data. In these cases, the subjects’ understanding of hypothesis and data were "melded into a single representation of ‘the way things are’" (Kuhn, 1989). While this confusion was most prevalent in younger subjects, it was present in a substantial number of adults. In the Red and White Yeast Lab, students are confronted with a large mass of data and will be observed as they develop their hypotheses; we will examine the extent to which they distinguish hypotheses and data as well as the sources of and motivations for their hypotheses. Interview questions will focus on the distinctions and interactions between data and hypotheses.
  3. How do scientists respond to anomalous data? In the commonplace view of scientific inquiry, the only reasonable response to anomalous data is revision of the theory (Chinn & Malhotra, in preparation). This idea is reflected in many studies, including Schauble, (1996), who counted the number of valid inferences students drew from experimental data. A more detailed examination of students’ and scientists’ responses to anomalous data revealed seven distinct responses: ignoring the data, rejecting the data, excluding the data, holding the data in abeyance, reinterpreting the data so that it is no longer anomalous, peripheral change of the theory, and fundamental theory change (Chinn & Brewer, 1993). In the Red and White Yeast lab the inferences are neither simply valid nor invalid. For example, ruling out ‘red is starving’ is a valid inference based on either the results of the ‘starving on the lid’ or the ‘big white patch’ experiments. However, different groups found the two experimental results differently persuasive; in this case, what matters is the perceived validity of the results. Analyzing students’ decisions about the differences in perceived validity will give us insight into students’ intellectual context for their work as well as their underlying models of scientific change. Students have many opportunities to grapple with anomalous data in this exercise. We will observe and code these behaviors using categories described previously (Chinn & Brewer, 1993). Others have found that some responses to anomalous data are more productive ? more likely to lead to a concrete conclusion ? than others (Dunbar, 1993); we will likewise be able to observe which responses are more productive than others. Interview questions will ask the students to explain the range of scientifically-appropriate responses to anomalous data.
  4. How are conflicts of ideas resolved in the scientific community (Ryder et al., 1999)? Philosophers and historians of science have concluded that these conflicts cannot be resolved on an entirely objective basis, but resolution must involve subjective judgements ? what Kevles, (1996) called "intuition and imagination". These subjective judgements involve the intellectual context of the work as described previously. In contrast to this view, many students believe that conflicts can be resolved in a completely objective manner. In addition, the distinctions between error, disagreement, and fraud are an important part of the public’s response to scientific disagreements (Collins & Pinch, 1993; Kevles, 1996). The many disputes over evidence and interpretation in the Red and White Yeast Lab will be analyzed in terms of depth of argument (Resnick et al., 1993), conversational moves (Pontecorvo & Girardet, 1993), mechanistic vs. correlational arguments (Ahn et al., 1995), as well as the role of ‘intuition and imagination’. The interview questions will focus on scientific methods for settling disputes.
    Taken together, the analyses of process will form a qualitative assessment of the path the students took as they constructed an explanation of the phenomenon. The analysis of outcome will quantitatively assess the desired cognitive changes. The combination of results from these studies will provide insight into the three questions that frame this project. We will compare what the students do in the lab with the known behavior of professional scientists to examine the extent to which the students are ‘doing science’ in the experimental and control sections. Comparing the process the students experience with its outcome will provide a greater understanding of the coupling between how the students did the science and what they learned about how science is done. Comparing the process and outcome in the control and experimental sections will give us insight into the role that this crucial element of authentic experimentation plays in students’ learning. In addition, the variable techniques and outcomes of the different TAs control and experimental sections will provide us with information about successful and unsuccessful teaching strategies that will inform our TA training.

(2.1.2) Analysis of TA’s role - Background & Mehods
 The Teaching Assistants (TAs) who teach the lab sections are active and dynamic participants in the process that the students experience. Their understandings, beliefs and behaviors will play a crucial role in the lab. Thus, a simple well-defined inquiry-based lab exercise like the Red and White Yeast Lab is an excellent experimental space in which to explore the importance of teacher variables in an inquiry setting. To examine this role in detail, we will videotape the TAs as they teach the lab and as they participate in interviews before and after the lab sessions. Prior to teaching the lab, TAs will be asked to describe their lesson plans, expectations for the class, understanding of the NOS and their views of the nature of science teaching. After completing the lab sequence, interviews will examine the same issues, now in the context of actual practice. Additionally, the post-teaching interviews will focus on particular classroom sequences and the TAs will be asked to discuss what they were doing and why. These videotapes will be transcribed and coded using a method similar to that described in section 1.1.2. The coding schemes will be tuned to examine the following features of the TAs role that have been identified in previous studies of teaching in inquiry-oriented classrooms:
 


During Year 1, we will develop scoring schemes for teacher behavior that include these features as well as others that we may identify as relevant. Videotapes will then be scored and the results correlated with process and outcome as described previously.
Many of the issues listed above have not been subjected to rigorous experimental analysis. The Red and White Yeast Lab will provide a ‘test site’ for examining and evaluating the means by which a teacher can encourage students to contribute to constructing an understanding of science. By focusing on issues that are common to al inquiry-based exercises, our studies will provide a rigorous foundation for future study and curriculum development.

(2.1.3) Conceptual Justification for the Experiments
     Many educators are currently developing inquiry-based science laboratory exercises. Most experimental studies of these exercises have compared the process and outcome of these labs with non-inquiry exercises. Very few studies have compared different versions of a particular exercise in order to rigorously examine the key parameters of its design. A notable exception is the work of White & Frederiksen (1998) who examined the effect of adding a metacognitive component of their inquiry-based physics curriculum. It would be particularly valuable for educators who are developing inquiry-based exercises to have information about the key parameters, trade-offs, and consequences involved in designing effective curricular materials. One particularly effective way to understand the fundamental principles of the design of effective inquiry exercises is to choose a model lab system and examine the effect of variations of its key parameters. The Red and White Yeast lab is an excellent model system for this type of analysis. It is a short, simple lab that is conceptually rich and easily amenable to experimental manipulation. Thus the purpose of my experimental manipulations is to explore the parameters of a successful inquiry-based lab ? to find the ‘limits of the envelope. The results of these studies can be used by other lab developers to inform their design and evaluation processes.
 To make the case that these experimental variations will be valuable outside the context of the Red and White yeast lab, I will describe how the manipulations I propose address key features common to authentic inquiry-based tasks, describe some of our expected findings, and explain how these findings could be applied in other lab exercises.

These findings will provide information about how far students can be ‘cognitively stretched’ and the consequences, both positive and negative, of this stretching.

(2.1.4) Connection to teaching
    My primary teaching responsibility is the two-semester Bio 111 and 112 series. I was hired specifically to improve the teaching and learning in these crucial introductory majors courses and to be a resource for education in the Biology department. The work of this study will have many important effects on this teaching responsibility including:
(a) Our findings detailing the important pedagogical components of inquiry-based lab exercises will be used directly in the design and revision of the lab curriculum and TA training materials. These findings will be of value in developing other inquiry-based lab exercises in Bio 111 and Bio 112 as well as in helping my colleagues to develop labs in their courses. In addition, these findings will be of help in the campus-wide General Education initiative which places an emphasis on inquiry and discovery in science courses.
(b) During the lectures, I frequently describe how the material covered in the lectures was produced, "How we know what we know." I make connections between the students’ experiences with the Red and White Yeast lab and the process of science in the research world. For example, in lecture, I describe the process that led to the conclusion that DNA is the genetic material. I mention that, as they observed in their lab, any one of the experiments I described failed to convince all members of the scientific community. I add that, as in lab, every member of the scientific community must make up his or her mind independently about which hypotheses they feel are supported by the data. I also emphasize that, as in lab, debates over data and interpretation are a vital part of the scientific process. The increased knowledge of the details of the student’s experiences gained through these studies will help me to make these connections clearer and more meaningful.
(c) The pattern that the students frequently observe, that red always grows up to give red and white while white always grows up to only white, is connected to material throughout the course. Red cells contain a plasmid (an extra-chromosomal DNA molecule) carrying a gene that produces the red pigment; cells that lack the plasmid are white. In roughly 1% of the cell divisions, the plasmid is not transferred to one of the daughter cells; thus, a red mother cell can give rise to a red daughter and a white daughter (red ? red + white). Once lost, the plasmid cannot be re-gained (white ? only white). An explanation of the details of this mechanism thus includes elements of genetics, biochemistry, molecular biology, and recombinant DNA, all of which are major segments of Bio 111. As with (2) above, the increased knowledge of the details of the student’s experiences gained through these studies will help me to make the connections between lecture and lab clearer and more meaningful.

(2.1.5)Development of TA training materials.
    The vast majority of college science laboratories are taught by graduate teaching assistants (Rushin et al., 1997). These TAs are often poorly-trained in education and poorly supported (Rushin et al., 1997). There have been many efforts to improve the training of graduate TAs (including Andrews, 1985; Lumsden, 1993; Rushin et al., 1997), but the few studies of the effects of training have mostly focussed on student’s opinions (Lumsden, 1993), effects on TA’s behaviors (Rodriguez, 1985; Lawrenz et al., 1992; Nunn, 1996), or TA’s conceptions of teaching (Hammrich, 1994) rather than educational outcomes. Several studies have shown that classroom teachers behavior is a crucial factor when leading an inquiry-oriented discussion (diSessa et al., 1991; Lederman, 1992); our studies will extend this work to graduate TAs.
    An important component of this project will be the development and evaluation of TA training materials and procedures to support the TAs as they teach the Red and White Yeast Lab. As with our findings about teacher behaviors, these materials will be applicable to many teaching situations beyond the Red and White Yeast Lab.
     Our TA training curriculum will be produced via a process of development, evaluation and revision that will continue throughout the duration of this project. Formative evaluation of the TA training curriculum and materials will be accomplished with a combination of interviews and classroom videotape; this will parallel previous studies of teacher development (Abd-el-Khalik et al., 1998):

  1. Before TA training.  TAs will be interviewed about their expectations, concerns, and lesson plans. These will include ‘thinking aloud’ sessions about their intended responses to classroom situations (Hashweh, 1987).
  2. After TA training.  TAs will be interviewed as in (1) and changes in their attitudes will be noted.
  3. During the lab exercise.  TA behaviors will be observed and coded in terms described previously.
  4. After the lab exercise.  TAs will be interviewed and asked to compare their plans with their actual practice, ‘debriefed’ to explain their choices and understandings, and asked to comment on the relationship between the TA training and their actual experience. The debriefing itself will also form an important part of the TA training.
These results will be directly applicable to the Red and White yeast lab; they will also apply to training teachers to lead open-ended discussions in a student-centered way. In addition, the development process will help inform other TA training and teacher education efforts in the sciences.

Time-line of TA Training Thread
Year 1: Produce TA Training Video Version 1
     Our Year 1 analysis will yield videotape clips that we will assemble into a two-part TA training video. The first part will be a ‘video lab manual’ that shows a typical lesson plan and relevant lab techniques. The second part will be ‘video cases’ ? classroom situations that demonstrate important issues when teaching the Red and White Yeast lab. The TAs will watch the video and discuss the TA cases as part of the regular TA training workshop I lead at the beginning of the fall semester.
Subsequent Years: Evaluate and Revise TA Training Video and Curriculum
     Based on the evaluation described in section 1.2.1 and our findings in the analysis thread, we will revise the TA training video each summer as appropriate. We will disseminate the various versions of the video at professional conferences for feedback. Our final version will be designed to be applicable to open-ended discussions in general as well as specifically for the Red and White Yeast Lab. Our findings will be published and our materials distributed freely on video, DVD, or via the www.

(2.1.6) Advisory board of education researchers.
    This board will consist of the following faculty members, all of whom have agreed to participate: Jeanne Bamberger from MIT who has extensive expertise in teacher training, students’ conceptual development, and educational assessment; Carol Smith from UMass Boston, who has extensive experience using clinical interviews to assess children’s conceptions of science; and Clark Chinn from Rutgers University, who has worked on developing students’ understanding of science in studies very closely related to those in this proposal. These individuals are uniquely qualified to advise and guide this project and will be consulted on a regular basis. In addition, each summer, advisory board members will attend a one-day symposium at UMass Boston where my research assistants and I will present our data and analyses for comment, suggestions, and critique. These symposia will be designed to allow a thorough analysis of our data and conclusions and will serve as regular guidance for our studies.

(2.1.7) ResearchLink 2000
    The Red and White Yeast lab has been chosen to be one of the ten research systems disseminated nationally by the NSF-funded ResearchLink 2000 program. This program brings college science teachers from around the US to summer workshops where participants are trained in the use of the selected lab systems. The resulting ResearchLink 2000 adopter sites will provide opportunities for dissemination of the developing curriculum and training materials as well as potential test sites for additional experimental manipulations.

(2.2) Overview of five-year plan
     These five studies are based on the three questions on the first page of this proposal; all three questions will be addressed in different contexts in each of the five studies. We will begin by developing our measures of process and outcome in year 1. In the following years, studies will document the effects of manipulations of the lab exercise on students’ process and outcome. These manipulations will involve key features of authentic experimentation: choosing experimental questions and variables; designing experiments; revising techniques; and using multiple experimental methods. Thus the results of these studies will be directly applicable to the development of other inquiry-based lab exercises designed to communicate the process of science.
     In addition to significant individual differences which will be described in the relevant sections, all of the experimental manipulations have a substantial effect on the complexity of the exercise as experienced by the students. In the first three experiments, the students in the experimental sections will experience a less complex situation because of reductions in variable choice, hypothesis choice, or number of rounds of experiments. In the last experiment, students in the experimental sections will experience greater complexity due to the addition of new tools with which to explore the phenomenon. In all cases, the tension is the same: simplified experiments allow clear focus on the issues at hand, while complex experiments have a richness and variety simple experiments lack. In each individual case, it is not clear a priori what the effects of different levels of focus and richness will be on the students’ conclusions about the process of science although it is likely to depend, at least in part, on students’ prior knowledge and experience. All four studies will thus inform the design of inquiry-based lab exercises with an understanding of the relative costs and benefits of different degrees and types of complexity.

(2.2.1)Year 1: Preliminary Ethnographic Analysis
The interactions between students, materials, the NOS, and the TAs will be complex and dynamic. As a result, we will begin by using ethnographic methods (Spradley, 1980) to get a clearer picture of these interactions using the words and ideas of the participants themselves. These ethnographic observations and analyses will be based on the concepts and issues described in previous sections as well as those that arise during our analyses. Our first year’s studies will be designed to observe and categorize the widest variety of behaviors, responses, and situations in order to develop the scoring systems we will use in the controlled studies in Years 2, 3, 4, and 5. We will interview a small number of subjects in great detail, trying many different interviewing questions and methods. Our classroom videotape during this year may also involve participant observation as described by Spradley (1980) in order to have a deeper understanding of the students’ thought processes.
Specifically, the goals for the first year ethnographic analysis are to devise or produce:

These examinations will not be intended to provide a rigorous analysis, but to develop the tools we will need for the later analyses. Although this limited sample size will not allow us to observe all types of behaviors and responses, we hope that this broad survey will allow us to observe the majority.

     Subjects Twelve Bio 111 students in four groups of three; these groups will be members of two lab sections taught by the same graduate TA.
     Procedure During the summer, I will train my staff in recording and analysis of these materials using the students in the summer Biol 111 class as practice subjects. In the fall, we will record paid volunteer students in the Bio 111 course as part of their regular lab section meetings. Interviews of the same students will be conducted immediately before and immediately after the lab exercise. Students will be interviewed again much later in the semester to explore any delayed effects of the lab; if these effects are substantial, we will include delayed interviews in later studies. During the remainder of the fall of Year 1, the spring of Year 1, and part of the summer of Year 2, we will analyze the transcripts and videotapes of the lab sessions and interviews. We will develop coding schemes and test revised versions of the interview protocols as necessary.
 Analyses Our analyses will focus on finding the most effective means for analyzing the data. We will develop coding schemes for students’ utterances based on the four issues outlined in section (2.1.1). Although the sample size is too small to draw meaningful conclusions about process and outcome, this study will give us insight into the trends we can expect in the later studies.

(2.2.2) Year 2: Examination of TAs’ Influence on Process and Outcome
 In this year, we will expand our analysis to a larger sample size of students and TAs in a rigorous analysis of the interactions in the lab and their impact on the students and TAs. We will use the interview questions and coding schemes developed in Year 1 to triangluate the relationship between TA and student actions and TAs and student’s changing conceptions. We will observe roughly 30 students in 10 laboratory sections taught by 5 different TAs. In this ‘natural experiment’ we will use the variation in style between different TAs to identify the factors that lead to the most productive process and outcome in the lab. We will be able to triangluate process and outcome by combining data collected as described previously.

(2.2.3) Year 3: What are the effects of allowing students to define the experimental question and variables themselves?
     Overview
    The students’ ability to choose the experimental question and define variables is an essential component of authentic experimentation that is not present in most simulated experiments found in teaching labs (Chinn & Malhotra, in preparation). In my experience with the Red and White Yeast Lab, framing the question and choosing variables allows the students to construct explanations more closely grounded on their pre-existing knowledge. This closer connection allows them to think more deeply about their experiments and data. As a result, they are much more engaged with the phenomenon and their explorations of it that they are in other, less authentic labs I have observed. On the other hand, a simplified ‘hypothesis space’ (Klahr & Dunbar, 1988), may allow a clearer focus on experimental design and data analysis. To explore these issues in more detail, the class will be divided into experimental and control groups. Students in the control group will be allowed to choose which hypotheses to investigate and to define variables as described previously; the experimental group’s assignment will be to evaluate the ‘red is starving’ hypothesis. Thus, both the hypothesis under test and the relevant variable, starvation, will be defined for the students.
     Subjects The analyses of process and outcome will focus on 36 Bio 111 students in 12 groups of three; 6 control groups and 6 experimental groups. Subjects will be chosen so that their understandings of the process of science (as measured by a simplified multiple-choice test based on the COST developed by Cotham & Smith, 1981) and knowledge of yeast biology (as measured by a simple multiple-choice test) will be approximately matched in control and experimental groups. There are typically six TAs in Bio 111 and each TA teaches two sections. To control for inter-TA variability, each TA will teach one experimental and one control section.
 Procedure
    In the summer, we will complete our analysis of the recordings from Year 1. In the fall, the subjects will be videotaped as they complete the lab with their classmates and interviewed before and after the lab. From the fall through the following summer, we will analyze the transcripts and videotapes for process and outcome.
 Analysis
    Based on my observations of students working with the lab in the past, it is likely that the students in the experimental section will experience a much more focussed exploration of the phenomenon. Possible effects, in terms of the four issues include:

    We will also compare students’ attitude towards the lab in the control and experimental sections. Because student-centered labs are rare in science classes, many students find the uncertainty and open-ended exploration of the Red and White Yeast Lab confusing and frustrating. De-emphasizing the answer is particularly uncomfortable for students trained that science classes are about learning the right answer. Assigning students to evaluate a particular hypotheses places more emphasis on the answer and may make these students more comfortable. We will compare the behavior of students in the control and experimental sections for differences in ‘comfort level’ and examine its effect on process and outcome.
 
(2.2.4) Year 4What are the effects of allowing the students to perform a second round of experiments?
 Overview
    The ability to revise and refine experimental methods is an essential part of authentic scientific inquiry. While many simulated experiments allow students to repeat experimental trials, very few allow revision of methods. Students working with the Red and White Yeast Lab frequently do very different experiments based on very different motivations in the first and second rounds of the experiment. Many of the disputes could not be settled without a second chance to revisit their hypotheses and data. On the other hand, some students’ correct understandings are de-stabilized by the second round discussions while still others find the second round redundant. Especially considering the time limitations of many teaching labs, it will be important to determine the pedagogical value of the second round of experiments. In this study, the students in the control lab section will carry out two rounds of experiments as described previously. In the experimental lab sections, the lab will terminate after the second meeting where the results of experiments from round 1 are analyzed. The experimental lab sections will not meet during the class periods when the second rounds of experiments are performed and discussed.
 Subjects & Procedure will be identical to Year 2.
 Analysis
    Because the students will not have an opportunity to revisit their hypotheses and experimental methods, they will not have an opportunity to resolve many disagreements experimentally. They may then be forced to resolve these disagreements through argument as opposed to experiment. Likely differences with respect to the four issues include: (2.2.5) Year 5 What are the effects of allowing the students access to additional experimental methods?
 Overview
    Authentic scientific inquiry into any scientific question involves multiple research groups using a wide variety of experimental approaches. In contrast, simulated laboratory exercises typically use a single experimental method. In my experience with the Red and White Yeast lab, many students have asked for other tools to help them to resolve their disputes. The addition of other techniques could help the students resolve disagreements or it could make the discussion so complex that, within the time allowed in lab, disagreements will remain unresolved. The students in the control lab section will have access to fresh plates of nutrient medium as described previously; this medium is called YPD. In the experimental lab sections, the students will have access to fresh plates with another type of medium, CSM-leu in addition to  plates with YPD medium. On CSM-leu medium, red yeast will grow to give patches that are entirely red; white yeast will not grow at all.
 Subjects & Procedure will be identical to Year 2.
 Analysis
    The effects of this manipulation are likely to be profound. Students who put samples of red onto CSM-leu plates will get red patches without any white edge. Samples of white will not grow at all, but will not turn red. These are inconsistent with ‘red is starving’, ‘white protects red’, and several other typical initial hypotheses. One might therefore expect that the students would reach the typical conclusions sooner than in the control sections. However, the finding that red gives only red on the CSM-leu plates is in seeming contradiction to the observed pattern that red gives red and white on the YPD plates. Thus this added tool may help some students to resolve disagreements more effectively, while the added complexity may destabilize an emerging consensus in others. At some point, the students may become overloaded with hypotheses, data, and experiments. The results of this study may give us information as to the limits of complexity that students can handle and the consequences of over-reaching them.
     We will also explore how the students integrate this new tool into their experiments. Do they simply repeat the same experiments on YPD and CSM-leu or do they use the different media for different experiments? To what extent are they able to integrate information from the two different types of media? What will they conclude about the difference between the two types of plates? To what extent do these depend on their prior knowledge of biology? The answers to these questions will give us a deeper understanding of how students approach inquiry-oriented tasks and how they construct explanations based on data.

(3) OUTCOMES

     This project will extend previous studies of students’ scientific reasoning to an authentic inquiry lab exercise and explore the factors which influence students’ learning of the process of science. We will examine the connection between the inquiry conducted by the students and their changing conceptions of the process of scientific inquiry. We will explore how alterations of the authenticity of the lab exercise influence students’ inquiry and understanding of science. These studies will continue the development and refinement of methods for understanding students’ cognitive change. We will produce a set of parameters that can be used to design and evaluate authentic inquiry lab experiences for teaching the process of science. We will also increase the understanding of how teachers, and especially graduate teaching assistants, should conduct inquiry-oriented lab exercises to support the desired student learning outcomes.
     These studies will result in the production of extensively-tested curricula. We will carefully revise and refine the Red and White Yeast lab for dissemination through ResearchLink 2000, published reports, and the www. Successful dissemination of this lab will be supported by our many years of testing and revision with a large number of teachers. We will produce TA training materials to support teaching the Red and White Yeast lab that will be valuable in training TAs for other inquiry-based lab exercises. These materials will emphasize the practical details of conducting and promoting open-ended discussions as well as the cognitive and pedagogical issues involved.
A deeper understanding of the process that the students experience with inquiry in the lab will allow me to make more effective lecture presentations about the history and nature of science. The parameters identified and explored in these studies will inform the development of other lab exercises in Bio 111 and 112 as well as in other science courses at UMB.
    These studies will be primarily conducted by research assistants from the graduate school of education and the program in Critical and Creative Thinking at UMB. The RAs in my research group typically have experience in middle- or high school science teaching or in scientific research before joining my lab. I have found that this mixture of people with experience in research and experience with students has been very productive. After graduation, students in these programs typically take positions as middle- or high school biology or chemistry teachers. As classroom teachers, they can apply their experiences in my lab with inquiry-based labs, analysis of classroom videotape, and assessment instruments in their own teaching.

(4) SUMMARY OF PRIOR RESEARCH AND EDUCATION ACCOMPLISHMENTS

     At the beginning of my career, my formal research training was in the domain of molecular biology. My research projects involved cell biology, genetics, molecular biology, and developmental biology. I designed measures, developed tools, and carried out large-scale analyses. These projects resulted in publication in peer-reviewed journals (Hynes et al., 1987; White & Yanosfsky, 1993; Madi et al., 1994; White & Woodward, 1995). This training included first-hand experience with the process of scientific inquiry: experimental design, data collection, data analysis, and data presentation. As a result of these studies, I have acquired knowledge of the subject matter as well as experience with materials, techniques, and organisms that can be employed in designing effective educational laboratory experiences. Through this, I have also become acquainted with biological researchers in many diverse fields who can serve as resources in the development of educational materials.
    Following completion of my Ph.D. in 1992, I changed the focus of my efforts from basic biology research to biology education. From 1992 to 1997, I worked as an instructor at MIT. There, I developed and evaluated curricula (White, 1998), supervised teams of graduate teaching assistants, and developed a video- and case-discussion-based TA training program for the Biology department. I developed and led intensive teacher-training workshops for graduate students and faculty. I also developed an open-ended software simulation of metabolic pathways that is part of the BioQUEST CD-ROM. As one of two co-directors of the MIT-Wellesley Teacher Education Program (TEP), I made extensive use of classroom videotape and clinical interviews to analyze students’ conceptual processes. The other co-director and I used videotapes of our own classes in evaluation and course-development. Much of my detailed experience with the Red and White yeast lab is based on experiences in one of the teacher education courses. There, the lab served as an example of the complexities of teaching using inquiry and of the parallels between scientific conceptual change and educational conceptual change. The MIT TEP was partially supported by the NSF-funded TEAMS-BC (Teacher Education Addressing Math and Science ? Boston and Cambridge) project. Through this collaborative, I helped to develop and evaluate curricula with an emphasis on inquiry and discovery. Other collaborative efforts included analysis of classroom videotape and presentations of our findings. This post-doctoral experience also introduced me to the science education community and literature.
     In 1997, I was hired by UMB to take over and extensively re-work the majors General Biology courses (Bio 111 & 112) and to be an education resource for the Biology department. I have completely revised the lecture and lab of both courses with an emphasis on problem-solving and inquiry. I adapted parts of the curriculum I developed at MIT to the labs and developed new labs using computer simulations of genetics and molecular visualization. I have continued to develop teacher training programs and seminars for faculty and graduate TAs. I have also begun a research program that focuses on the teaching and learning in my courses; this program utilizes graduate research assistants from the School of Education. Our first research project, on the effectiveness of allowing students to use note sheets during exams (White et al., in prep.) is due to be submitted this summer. In this project, we correlated the form and content of the single page of notes each student was allowed to bring to each exam with differences in students’ exam scores. Interestingly, for many students, the form and content of their notes were not correlated with performance. We are currently investigating students’ misconceptions and cognitive changes in the domain of protein structure and function. In this project, we are examining students’ responses to open-ended written surveys before and after students have been exposed to software that allows the students to manipulate 2-dimensional representations of 3-dimensional protein molecules. I am currently supervising four undergraduate independent study students as they develop and evaluate educational multimedia for use in Bio 111 and 112. This experience in conceptual analysis, videotape analysis, curriculum development, and supervising large creative teams has prepared me to lead the major research project described in this proposal. Carrying out the studies proposed here will help me to build on a strong background in molecular biology and biological research as I apply these skills to biology education research.

References