Responding to a Request for Evaluation Proposal

.

Paper details 

Request for Proposal

DESIGN WORK FOR AN IMPACT EVALUATION OF STEM INSTRUCTION FOR ENGLISH LANGUAGE LEARNERS

U.S. Department of Education

I. INTRODUCTION

This contract with the U.S. Department of Education (ED) is for feasibility and design work for a rigorous impact evaluation in the area of science, technology, engineering and math (STEM) instruction for English language learners (ELLs). The contractor shall design an impact study of curricula or instructional practices in STEM for ELLs.

A. Legislative Authority

This design work is authorized in Title III, Part A of the Elementary and Secondary Education Act, as re-authorized by the No Child Left Behind Act of 2001.

B. Background

Between 1979 and 2008, the number of school-age children (children ages 5-17) who spoke a language other than English at home increased from 3.8 to 10.9 million or from 9 to 21 percent of the population in this age range. While Spanish is the native language of the majority of ELLs, ELLs speak more than 150 languages. This presents challenges in the classroom as students of various language backgrounds and abilities are faced with the need to master content without the benefit of proficiency in English. It also presents challenges for teachers, many of whom are not adequately prepared to provide instruction appropriate to and effective for ELLs.

Effective STEM instruction for ELLs is of particular interest because these areas have been identified as those key to the United States’ future in the global marketplace. Without knowledge and skills in these areas, the U.S. will not be able to maintain leadership and effectively compete with other nations around the world.

International comparisons indicate that U.S. student performance in the STEM areas lags behind that of other countries. A 2009 assessment of 15-year-olds in 65 countries found that students in 23 countries had average scores in math that were higher than the average score of U.S. students. Students in 18 countries had higher average scores in science as compared to the average score of students in the U.S. National assessments also highlight the need for improvements in student achievement in the STEM areas. For instance, the 2009 National Assessment of Education Progress in Science revealed that only 72% of fourth-graders, 63% of eighth-graders, and 60% of twelfth-graders performed at or above the basic level.

For ELLs, academic performance in the STEM areas may be further hampered by students’ lack of proficiency in English. Curricula and instructional strategies that are designed for native English speakers may not be effective in teaching content to ELLs. Furthermore, STEM teachers—including those who are highly effective in educating native English speakers—may not be equipped with the pedagogical knowledge necessary to tailor instruction to meet the needs of ELLs.

ED’s Institute of Education Sciences, through its comprehensive centers, network of Regional Education Laboratories (RELs), and research grantees, is currently sponsoring a number of projects to broaden understanding of how to more effectively educate ELLs in the STEM areas. This includes research examining enhancements to instructional practices and materials in middle school science and math and elementary science and professional development for middle school science teachers of ELLs. It also includes the development of an intervention intended to increase ELLs’ knowledge of academic language for math.

ED seeks to expand its portfolio of research on STEM instruction for ELLs by conducting a rigorous national impact evaluation in this area. If feasible, the study will be an evaluation of either a professional development program designed to improve teachers’ instruction for ELLs in one or more STEM areas, a curricular or instructional strategies intervention for ELLs in one or more STEM areas, or of an intervention that includes both components (professional development and classroom intervention). ED has a preference for a study of an intervention in science at the middle or high school levels. However, interventions in all STEM areas for the elementary through secondary grades will be considered.

C. Key Research Questions

The design work will be conducted to address the following key question:

1. What are design options for an impact study of professional development for STEM teachers of ELLs or curricula/instructional practices for ELLs in the STEM fields, or a package that includes both of these components?

a. What are appropriate outcome measures to study? Are these available from extant data, such as state/district administered tests?
b. What should professional development and/or instruction look like in the control group?
c. What kinds of districts and schools would be most appropriate for the impact study? Do a sufficient number of these kinds of districts/schools exist to support an impact study?
d. What strategies are likely to be effective in convincing districts and schools to participate in the evaluation?
e. What implementation challenges would an impact evaluation need to address?

II. SCOPE OF WORK

This section specifies the tasks and subtasks that the contractor shall perform. All deliverables shall be submitted to ED’s Contracting Officer’s Representative (COR) and ED’s contracting specialist (CS) electronically unless otherwise noted.

The COR reserves the right to inspect the contractor’s work while in process. The contractor shall submit to the COR any progress report or other written documentation of work to date that the COR requests to facilitate that inspection, according to contract clause 52.246-4 (fixed price) and 52.246-5 (cost-reimbursement).

Task 1—Meetings with ED

1.1 Kick Off Meeting. The contractor’s project director and up to two additional key project staff shall meet with the COR and other appropriate ED staff within 10 work days after the effective date of the contract award to discuss details regarding the tasks outlined in the proposal, scheduling activities, and other issues related to the contract. This meeting shall be at ED’s offices in Washington, DC. The contractor shall provide an agenda for the meeting at least two work days in advance of the meeting. The contractor shall come prepared to identify any areas of concern and to suggest ways of responding to these concerns. Within one week following the meeting, the contractor shall prepare a memo summarizing the key issues and concerns raised at the meeting, and how each shall be addressed.

1.2 Periodic Meetings. The project director and up to one additional key staff member shall meet with the COR and other appropriate ED staff in Washington, DC on an as needed basis, not to exceed 2 times over the duration of the contract. The purpose of the meetings is to brief ED on the progress of the work and to discuss issues as they arise. Within one week following the meeting, the contractor shall prepare a memo summarizing the key issues and concerns raised at the meeting, and how each shall be addressed.

Deliverables: Kick off meeting agenda
Kick off meeting memo
Periodic meeting memo

Task 2—Technical Working Group for the Evaluation
2.1 Establish Technical Working Group (TWG). The contractor shall establish a TWG for the purpose of advising the contractor on research questions, possible interventions, outcome measures, and design options. The TWG shall be comprised of up to 10 individuals and should include experts in English language acquisition, STEM (science and math, at a minimum) instruction and teacher professional development, as well as experts in evaluation methodology.
The contractor shall submit a list of proposed panelists no later than one week after the Kick Off Meeting. When preparing the list, the contractor shall evaluate the potential for conflicts of interest by assessing whether each potential TWG nominee has a bias or financial interest in the outcome of the study. After IES endorses the proposed TWG members, the contractor shall contact each proposed member and, within four weeks of IES endorsement, submit letters of commitment for the panelists to the COR and CS for approval by the CO.

Deliverables: List of proposed TWG members
Letters of commitment
Draft meeting materials
Revised meeting materials
Summary of TWG meeting

Task 3 – Identifying Possible Interventions

Through literature reviews, web searches and discussions with experts in the field, the contractor shall identify potential interventions for a possible future impact study. Interventions may include professional development models or programs for teachers who are instructing ELLs in the STEM content areas, as well as classroom practices or curricula utilized by teachers when instructing ELLs in the STEM content areas. The interventions may be applicable to any grade(s) from kindergarten through twelve and to one or more STEM subjects.

The contractor shall submit a draft intervention memo, and the memo shall include, but need not be limited to, those interventions on the preliminary list. The memo shall fully describe each intervention, including any empirical evidence of effectiveness and/or theoretical support for the intervention, and discuss the feasibility of including each intervention in a rigorous impact study. In assessing the feasibility of evaluating each intervention, the contractor shall take into account factors including but not limited to: the transportability of the intervention to multiple sites, the extent to which implementation support is available to ensure that the intervention is well-implemented, the extent to which districts would likely be willing to implement the intervention on a pilot basis, and the extent to which appropriate student outcome assessments exist. To the extent possible, the memo shall provide information on how widely implemented the interventions currently are and the approximate costs of implementation.

Deliverables: Preliminary List of Interventions
Draft Intervention Memo
Final Intervention Memo

Task 4 – Design Options for Future Study

Based upon ED input, and under TWG advice, the contractor shall develop design options for evaluating one or more of the interventions identified through Task 3. The contractor shall submit a draft design report that describes how the selected intervention(s) could be rigorously evaluated through a national evaluation. The draft report shall address the design questions listed in Section C of this Performance Work Statement.

Deliverables: Draft Design Report
Final Design Report

Task 5 – Investigating Possible Study Sites

The contractor shall recommend possible sites for the future impact evaluation. The contractor shall not be responsible for recruiting the sites to participate in the evaluation, but rather for identifying sites that have characteristics that make them suitable candidates for future study recruitment efforts.

The contractor shall investigate the suitability of sites for the impact study based upon publicly available information, including websites and extant data sources. ED does not anticipate that the contractor will contact sites. Recommended sites shall be those with sizeable ELL populations in the STEM classes of interest (based upon work conducted under Tasks 3 and 4) that are not already implementing the selected intervention(s) on a widespread basis. The contractor shall take other factors, such as the availability of extant data at each site, into account as well.

Deliverables: List of possible sites

Task 6 — Reporting Requirements

The contractor shall submit one copy of the following performance reports each month to the contracting officer (CO) and one copy to the COR simultaneously with the monthly invoice.

Monthly Progress Report/Exception Reports. The reports shall summarize the major activities and accomplishments for the reporting period. In addition, they shall provide information for each project task regarding significant findings and events, problems encountered, and staff use. The reports shall also specify the extent to which the project is on schedule, briefly describe the activities planned for the next month, identify and discuss significant deviations from the substantive and time factors in the management plan, and identify and discuss any decisions which may be needed from ED. If there are no exceptions, the reports shall state that there are no exceptions. If there are exceptions to the management plan, the contractor shall describe the plan for resolving the problems.

Monthly Manpower/Expenditure Reports. These reports, prepared and signed by the project director, shall summarize the actual personnel assignments for the month just completed, showing for each staff member the hours charged by task. The report shall project similar assignment information for the upcoming month. The reports shall also exhibit expenditures, segregating project costs by individual and by task, and specifying for all travel the locations, duration, and personnel for each trip.

Deliverables: Monthly progress and expenditure reports

.


Order This Paper Now