Unit Assessment System (UAS)
A Plan for Integrating and Managing the
Assessment of Candidates’ Performance in the
System Development
1. Stakeholder Involvement
The UAS demonstrates the unit’s
belief that it must be reflective of the educational environment in which and
for whom the plan is implemented. This
process for the review of candidate preparation at
·
School of
Education: Dean, Director of Teacher
Education Student Services, Department Head of Curriculum & Instruction,
Certification Officer, ATU Liaison to Cohort Schools, and Curriculum &
Instruction faculty;
·
Arts &
Sciences: faculty representatives from
Math, Science, English, Social Studies, Music and Art;
·
Associate Vice
President for Academic Affairs responsible for University Assessment Plan,
·
P-12 faculty and
administration: representatives from each Cohort site at both Elementary and
Secondary levels,
·
Candidates, and
·
Teacher Education
Alumni
The Unit Assessment Committee (UAC) was established in
August 1999, to lead development of the UAS.
The committee was composed of the Dean, the Director of Teacher
Education Student Services, one faculty member from each of the candidate
programs in the department of Curriculum & Instruction, and the ATU Liaison
to Cohort Schools. The
The continued involvement of
the task group of stakeholders is visualized through key elements to be noted
in the life of the UAS:
·
The UAC will
continue to maintain leadership in implementing the UAS.
·
The ATU Liaison
will meet monthly with Clinical Practice Instructors (CPIs) at each of the
Cohort Schools to maintain communication and focus in the implementation and
evaluation integral pieces of performance evidence.
·
Field-based
practitioners will provide on-the-job assessment of candidates’ application of
the knowledge, performances, and dispositions acquired in the Unit’s programs
that will be communicated throughout the system.
·
The Unit
Portfolio Committee will continue to implement and evaluate the use of the Exit
Portfolio annually. Unit faculty, field
supervisors, and candidates will be trained and involved in its continuing
development, evaluation and refinement.
Decisions about the use of
data collected are made collaboratively through the unit’s governance system:
System
Components
2. Evidence of
The UAS has been designed to
ensure that the preparation of candidates is anchored in credible and
educationally vital evidence of the desired understandings. The unit’s core values extend into a conceptual
framework, Professionals for the Future,
providing the skeletal support for delivery and assessment of the preparation
programs within the Unit. Professionals for the Future was
developed with the involvement of the stakeholders. It is the result of the review of credible
sources and incorporates what teachers should know and be able to do as
described in:
University
and Unit mission statements
Relevant
national and state standards for teachers including:
National Board for Professional Teaching Standards
(NBPTS)
Interstate New Teacher Assessment and Support
Consortium (INTASC)
NCATE Program Standards
International Society for Technology in Education
Standards (ISTE)
Content Cluster Group Standards
Arkansas P-12 content and developmental standards
Professionals
for the Future was developed with
thoughtful consideration of all of the aforementioned sources. Core values are organized into four foundations
and six unifying factors that drive curriculum design. Course objectives are derived from the
conceptual framework and aligned with the standards. Performance tasks and assessment of
performance are, then, directly related to standards.
The unit’s goal is to produce candidates who are Professionals for the Future and will
internalize, initiate, and sustain a professional commitment to impact learners
in diverse learning communities. The
Peer Review Process samples the validity and reliability of the system ensuring
the integrity of the design. The NCATE review process is also used by the unit
to ensure the integrity of the conceptual framework and that appropriate
standards are included in all programs.
3. A Student
Assessment System
Through introductory courses candidates
will recognize the need for standards for their professional area. Standards are published in the introductory
experience materials such as handbooks and the unit’s Policy and Procedures Manual. Candidates receive/purchase copies of both
for reference throughout the program of choice.
Professional standards are also published to appropriate unit web
sites. Subsequent courses will require
the demonstration of a good understanding of those standards. The use of Taskstream links candidates to the
multitude of standards currently available in all programs. Instructors will demonstrate their
understanding of the relationship of instruction and tasks to the standards and
to anticipated candidate learning. The
candidate’s work will reveal their control of knowledge, performances, and
dispositions that will enable them to perform at a minimum level of experience
and effectiveness to impact learners in diverse learning communities. They will know what it takes to be Professionals for the Future.
Understanding of the
standards develops slowly and reveals itself as a progression along a continuum
for any single standard. Use of
recurring tasks and relevant rubrics will help the unit and the candidate to
see their ability to make increasing sense and use of knowledge, performances,
and dispositions. Course and Program
matrices have been developed to articulate the progression. Appropriate standards-based rubrics within
courses and at benchmarks have, also, been developed. The matrices and rubrics demonstrate that
assessments are related to standards, are embedded in instruction, provide
standards-based feedback, are conducted on a continuing basis, and are
integrated with learning experiences throughout the candidates’ development.
Reflection on practice, as a unifying factor extending from core values through the four foundations extended to all programs, is included throughout the curriculum and in the assessments. Shulman’s (1987) definition of reflection as “a process that involves reviewing, reconstructing, reenacting, and critically analyzing one’s own and the class’ performance” (p.15) has been adopted. The employment of this definition throughout programs facilitates candidates’ focus on the achievement of standards. Specifically, it is believed that by applying critical thinking skills and analyzing their experiences, candidates can become more knowledgeable about themselves and their performance as professionals. Reflective exercises orient candidates to the purposes and obligations that are most relevant to final assessments of their knowledge, performances and dispositions.
Field experience is offered
frequently and throughout the programs.
Expectations of performance communicate standards clearly. These experiences are the candidate’s opportunity
to demonstrate the ability to translate knowledge and understanding into action
that will affect student learning. Field
evaluations, visit summaries, and adapted PATHWISE observations are used to
assess the ability to translate knowledge and understanding into action with an
emphasis on the standards in relevant problem contexts. Together, instructors
and candidates begin to anticipate the candidate’s positive effect on student
learning and achievement of the standards.
Candidates are also directed
to the standards through the advising process.
Preregistration each semester allows candidates and faculty advisors to
plan each coming semester with care for the candidates’ professional
development. Applications for admission
to professional studies and internship guide candidates and advisors in
reviewing the evidence of their performance in the standards.
The UAS provides more than
circumstantial evidence to recommend a candidate for initial or additional licensure. It will provide evidence that will allow the Unit
to draw accurate inferences about specific candidate understandings reflecting professionals
who internalize, initiate, and sustain a professional commitment to impact learners
in diverse learning communities. Two major types of assessment activities
include those that address candidate performance and learning, and those that
deal with unit policy level issues.
The system demonstrates the unit’s
concern for whether a candidate’s understanding has been seen in different
contexts, at different times, and on different types of assessments, before a
confident judgment is rendered. The system is grounded in authentic performance
application. The system has been
designed to be feasible and candidate-friendly.
Rubrics that are standards-based and PATHWISE-aligned have been
developed for each and every task throughout the curriculum in each program.
The overall evidence for
achievement of the standards must be “sufficient.” Therefore, multiple measures
of candidates’ knowledge, performances, and dispositions will be made. Multiple data sources will include student
entrance and exit exam performance, student retention throughout the program,
grade point average, observations, the use of various instructional strategies
and technologies, faculty recommendations, demonstrated competence in academic
and professional work such as working and exit portfolios, performance
evaluations, research and concept papers, student work samples from field
experiences, recommendations from the appropriate professionals in schools, and
themes and results from follow-up surveys.
At the same time, consideration will be given to the
economy of assessment, gathering only as many samples of performance as
decisions require, rating in terms of the relevant criteria, training raters to
be efficient, using rubrics, checklists, and work samples when appropriate, and
utilizing candidate self-evaluation when appropriate. Candidates are trained to be reflective
raters so that they develop a clear sense of the key dimensions of sound
performance—a vision of poor and outstanding performance.
The unit’s plan for assessment extends from the university
plan for assessment of student academic achievement at Arkansas Tech
University, which responds to two principal imperatives: 1) the mandate of the Arkansas General
Assembly (ACT 874 of 1993) for the assessment of general education at Arkansas’
institutions of higher education; and 2) the mandate of the North Central
Association of Colleges and Schools that institutions develop, submit and
implement a plan to assess student academic achievement. The university assessment plan serves two
general purposes: 1) to provide the
University, its students and other constituencies with evidence that the
institution is achieving its objectives; and 2) to provide the University and
its various subdivisions with information that will guide efforts to strengthen
and improve academic programs.
Information is available to initially describe the candidate
prior to admission, or upon admission, to teacher education’s professional
development phase/stage. Also available for
initial licensure programs through the university plan are results from the
Arkansas Assessment of General Education Test (AAGET), better known as the
Rising Junior Exam, which consists of three parts: 1) The Collegiate Assessment
of Academic Proficiency (CAAP), designed to assess foundational academic skills
in the areas of writing, reading, mathematics, science reasoning, and critical
thinking; 2) The Academic Profile, yields scores and subscores for humanities,
social sciences, natural sciences, reading, writing, critical thinking,
mathematics, and a total score; and 3) The Writing Sample, an essay scored
under the supervision of the Department of English and Foreign Languages. Students must complete the AAGET no earlier
than accumulating 45 college-level credits and no later than completing 60
college-level credits. The
In initial licensure
programs, proficiency results from completion of general education
requirements, monitoring of cumulative and major GPA, and the completion of the
Praxis I, Praxis II, and Praxis III tests are summarized through averages, spread
of scores, and distributions by program in an attempt to describe the candidate’s
knowledge of subject and pedagogical content.
Faculty is integral in the
accomplishment of the unit’s mission, and must, then, be an integral part of
the assessment process. The faculty is
involved in identifying and defining student outcomes, selecting or building
the measures of candidate success, reviewing assessment results, and using
these results to develop effective instructional strategies to strengthen candidate
performance; that is, modeling best practices, being committed to continuous
learning and purposeful reflection, working collaboratively with internal and
external constituencies. Professional
observations and carefully considered judgments made in the day-to-day
management of instruction play a key role in assessment in all programs,
especially in the assessment of oral and written communication. The unit emphasizes the continuing
development of dependable performance assessment in these contexts. The purpose of assessment has shifted from
just assigning a grade in a course to documenting learning and the achievement
of standards. Each course syllabus
includes a matrix relating objectives, standards, and assessment. These matrices have been developed for all
programs.
The progress of candidates at
different stages within programs is monitored through authentic
performance-based assessments using systematic procedures and timelines. Specific benchmarks and assessments are
organized into at least three summative decision points for all programs:
·
Admission to
professional studies
·
Retention such as
admission to internship
·
Exit readiness
The specific performance based assessments and related standards proposed at each point in development are program relevant and specific. The goal is to describe how the standards are revealed in increasingly sophisticated ways.
Our goal is to be sure that
selected performance measures and demonstrations reflect professionals who will
internalize, initiate, and sustain a professional commitment to impact learners
in diverse learning communities, Professionals
for the Future. The candidate will
be called upon to demonstrate specific knowledge, performances, and
dispositions, and to apply those they have mastered. Performance assessment will play a key role
in determining candidates meet respective program standards and can impact
learners in diverse learning communities.
The unit will know that a candidate has met the standards when there is
sufficient evidence for: professional and pedagogical knowledge, skills, and dispositions;
strong and developing knowledge of the school culture; growing expertise
concerning systemic and developmentally appropriate practices; and a strong and
developing liberal arts background.
Dispositions refer to values,
conceptions and commitments. Professional
and pedagogical knowledge, skills, and dispositions, knowledge of the school
culture, systemic and developmentally appropriate practices, and a strong
liberal arts background, as they relate to aspects of diversity, are essential
in the preparation of Professionals for
the Future. The unit will determine that the candidate has
met the standards when there is evidence: 1) of a candidate’s knowledge and
understanding of the unit’s conceptual framework; 2) that a candidate adopts
conceptual goals for professional performance in the field; 3) that a candidate
makes use of the knowledge of others and justifies his/her performance with
emphasis on issues of diversity; and 4) that a candidate is aware of professional
practices that are defined according to what students learned rather than
according to how professionals behave.
This evidence will come from faculty’s evaluation of performance in
course work and evaluation from the field.
Matrices developed in each program delineate sources of evidence.
Professionals for the Future are developing leaders within the learning
community. The understanding of and
participation in leadership with and among other stakeholders in learning
organizations will powerfully influence the goals of learning organizations and
ultimately student learning. The
preparation of candidates includes consideration for increasing levels of leadership
as it relates to: professional and
pedagogical knowledge, skills, and dispositions; strong and developing
knowledge of the school culture; growing expertise concerning systemic and
developmentally appropriate practices; and a strong and developing liberal arts
background.
Effective leadership requires
mastery over relevant content matter. The
unit not only collects indicators of candidates’ content knowledge, but uses
the information to make decisions about a candidate’s progression through the
program. Critical indicators include candidates’
performance on designated tests of professional knowledge such as the Praxis I
Academic Skills Assessments in reading, mathematics, and writing; performance
in their major as well as their cumulative performance in all courses, the
Praxis II subject area assessment; the AAGET, the Praxis II tests of
pedagogical knowledge, and the Praxis III.
Embedded in the assessment
structure, from core values to exit interviews, is the belief that to be Professionals for the Future, one must
be able to impact student learning. Programs
provide a variety of opportunities for candidates to see and understand the
role of leadership at various levels of the school culture. Candidates must know the knowledge or content
skills that children/youth should learn, how these are determined, and who
determines them. They must know how to
deliver them in ways that lead to learning.
As professionals, they need to appreciate the sociopolitical context in
which they work as well as the environments that shape their students’
lives. The most visible resulting evidence
that the candidate’s performance will have a positive effect on students’
learning and achievement will be presented in exiting activities such as the portfolios
and action research. The candidate will
be expected to demonstrate that he/she can find ways to understand the level of
accomplishments of schools and students, to use that knowledge as a basis for
design of standards- and performance-based action, to identify and apply
suitable measures of effects on student learning as a result, and finally, to
reflect on the whole sequence and hypothesize how improvements might be
accomplished.
Candidates will not only
“know” content, but will be able to understand it and relate it to ideas,
information, and knowledge previously learned as demonstrated in systemic and
developmentally appropriate practices.
Observations will verify that candidates can perform using a variety of
methods, can communicate with a variety of stakeholders in the learning
community, can adapt their performance to the culture and context of student
learning, and can effectively use technology .
Candidates who meet these performance competencies will have a positive
effect on student learning and are Professionals
for the Future.
Purposeful reflection is a unifying factor in the conception of Professionals for the Future. Program-specific curriculum provide for
the increasing sophistication of reflective thinking about: professional and pedagogical knowledge,
skills, and dispositions necessary to impact learners in diverse learning
communities; the collegial and collaborative roles of professionals in school
cultures; make and discuss professional judgments within a larger educational
system where a variety of developmentally appropriate practices may be applied;
and the content and underlying larger rationales for learning new content
and/or skills. Activities such as
learning units, working portfolios, research awareness projects, internship,
exit portfolio and action research are prime examples of how candidate learning
and reflection are embedded in curriculum and assessment. The tasks, evaluation rubrics based on core
values, and rater training have been provided and will continue to be provided
to appropriate stakeholders. The process
involved in these activities as well as the results of candidates’ performance are
analyzed. Recommendations for changes and training of appropriate stakeholders are
accomplished through unit professional development activities.
The Exit Portfolio will
provide critical performance-based evidence that candidates in initial
licensure programs are sufficiently able.
That is, a candidate for initial licensure is able to perform well with
knowledge and skill in at least a few key contexts with a beginning repertoire;
demonstrates flexibility in, or adaptability to, diverse contexts; needs
limited coaching; and, demonstrates acceptable use of personal judgment and
responsiveness to the specifics of a feedback/situation. The Exit Portfolio is a reasonably critical
and comprehensive look at standards.
There will be clear evidence that the candidate has the knowledge of the
content described in each standard, has the proficiencies to apply that
knowledge to the teaching situation, has enthusiasm and attitudes appropriate
to successful teaching, and can have a positive affect on students’ learning
with respect to the content specified in the standards. Until there is insufficient evidence in the
Exit Portfolio, a candidate will not be recommended for licensure.
In summary, rubrics mapped
across professional curriculum are designed to check attributes present, rate
proficiency demonstrated, and describe performance via standards evidencing
leadership. Evidence for leadership may
include modeling, advocating, communication of high expectations, and the
expression of a commitment to student learning and personal learning.
Finally, with the
availability of results of performance on the Praxis III, there is additional
verifying evidence for the preparation of candidates for initial licensure.
The unit will use the
collective presentation of candidate assessments and related data to document
and define the quality of programs preparing candidates to meet program-specific
standards. Quality will be determined on
the basis of whether candidates are acquiring learning outcomes and whether the
programs and unit are achieving their respective goals. That is, whether the knowledge, performances
and dispositions are best nurtured by the experiences that lie ahead or by
seeking out additional experiences to ensure the accomplishment of benchmarks
and achievement of readiness to teach.
Evaluation and assessment
within the unit will be a process for determining the extent to which the
goals, and outcomes as conceptually developed and organized, are actually
producing the desired results. The UAS
will enable the unit to identify the strengths and weaknesses of its conceptual
plans, and to determine the areas in which the curriculum is effective and in
need of improvement. The UAS will enable
the unit to determine whether the curriculum as designed, developed, and
implemented is producing the desired results.
Data will be collected and analyzed
at summative decision points and between in all programs. Results will be provided to appropriate
stakeholders for review and input which is expected to lead to continuous
programmatic and unit improvement. The unit
will host a stakeholders’ meeting annually.
Assessment and evaluation information will be compared to available
external sources, i.e., scores and sub-scale results on state licensure
tests. The collective data on candidate
and graduate performance will be summarized in the Dean’s Report to the unit
and stakeholders, the State Report Card, the NCATE review for accreditation,
and learned society program reviews.
The Peer Review Process, the
Exit Portfolio, Exit Evaluations, and induction follow-up studies will be key
components in documenting the shift from program processes, what candidates do
in their preparation programs, to the competencies of candidates, what candidates
can do with and for P-12 learners. Yearly
review of all data collected will follow the Peer Review Process, submission of
Exit Portfolios, and Exit Evaluation in the spring semester. This review will prompt proposals for change
in an appropriate sequence for stakeholder input prior to the unit and university
governance structure for revision of programs and/or assessment system.
Program assessment will be a
formative process and an ongoing part of unit activity. The assessment process is the systematic
determination of what knowledge, performances, and dispositions are best
nurtured by which experiences provided, or what experiences may be added to
ensure the accomplishment of benchmarks and achievement of the vision, Professionals for the Future who will
internalize, initiate, and sustain a professional commitment to impact learners
in diverse learning communities. The unit
and stakeholders will review program assessment results annually in preschool
in-service. Aggregated assessments from
individual candidates and other sources will be used to refine and revise the
conceptual framework and program opportunities.
Questions leading examination of the data will include:
·
Do we have
sufficient and revealing evidence of knowledge, performances, and dispositions?
·
How can we build
more effective tasks around and from the need for evidence of critical
knowledge, performances, and dispositions?
·
Are performance
tasks effectively anchoring the courses and focusing the instructional work?
·
Are we able to
distinguish between those who really have a grasp of the standards and those
who don’t?
·
What
misunderstandings and misconceptions are occurring? How do we check for those? How do we assist the candidate in resolving
them?
Answers to these questions
will be considered through the Peer Review Process and in preschool in-service
to analyze the sources of performance evidence and provide for on-going
development.
The unit will, in response to
the annual review, reset goals and focus faculty development in support of
their accomplishment. Support includes,
but is not limited to PATHWISE training; unit in-service and work days; and
professional development opportunities in the process of aligning standards,
curriculum, instruction, and assessment.
The goal is that, as a unit, there is clear purpose of assessment, and
that the assessment is effectively communicated to candidates with performance
criteria conveyed in understandable ways, such as standards-based rubrics,
prior to the assessment and throughout challenging performance exercises.
Licensure tests, technology
assessments, portfolio evaluations, field evaluations, and the state report
card will, particularly, document the preparation of teachers such that
individual diagnosis and group needs assessment are available to improve the
opportunities for candidate development.
Identifying the strengths and weaknesses in the performance of
individuals, and diagnosing the strengths and weaknesses of groups of candidates
such as by tasks completed, by courses completed, or by program completed, can
be used to provide for educational needs of the group, to improve the opportunity
for learning by task or course, or to refine the program. These data will assist the unit in
identifying those candidates ahead of, or behind, the expected level of
performance for decisions about continuing or providing remedial opportunities.
The unit conducts an annual
Peer Review Process. Each fall semester candidates
are asked to rate faculty on their effectiveness through the use of a
university-designed survey. Candidates
also rate how well they have mastered each of the course objectives. The faculty reviews these ratings and prepare
teaching portfolios which include overviews of courses taught, examples of
performance tasks and instructional strategies, sample(s) of candidate work,
and evaluation rubrics. Through the Peer
Review Process faculty discuss the results of the evaluations and any changes
they wish to make to better support candidate’s achievement of the
standards. Recommendations for change
are then submitted to respective department heads and the dean of the unit. Proposals for change are taken to the Teacher
Education Council, which includes the variety of stakeholders in its
membership. The Peer Review Process and
preschool in-service will provide clear evidence that data collected on
candidate performances and evaluative assessments gathered from candidates are
used to make decisions on curriculum and program practices.
The unit will ensure that its
assessment system is continuously managed.
Coordination of the offices of the Dean and the Director of Teacher
Education Student Services and their respective staff began in the fall of 2001and
has designated a full-time Document Examiner, one 75% time secretary, and one
graduate assistant to conduct and maintain the UAS. A networked, PC-based File Pro database
system has been developed for collection, recording, storage, and retrieval of
data. The system became operational in
the fall of 2000. The secretary and the Document Examiner now monitor the
database and ensure that appropriate data are entered or received from
designated sources for recording. Data
are secured and confidentially maintained. The Dean, the Director of Teacher
Education Student Services, the secretary and the Document Examiner have full
access by password. Faculty have access
in read-only status by password. The
secretary and the Document Examiner provide data to the Dean, the Director of
Teacher Education Student Services, and the unit’s department heads all of
whom, in turn, distribute information to appropriate stakeholders throughout
the year and annually in summary. These
data summaries, then, provide the basis for reviewing and revising the
conceptual framework, program curricula, and the assessment system.
The UAS is an element of the
institutional assessment plan and is, therefore, linked to all other units. For example, relevant data received from the
institutional assessment include admissions’ demographics. Relevant data provided by the unit to general
education include Praxis I and II assessments.
Data is made available to departments of Arts & Sciences and to the
Vice Presidents of Academic Affairs and Institutional Research.
7. Reviewing and Revising the Assessment System
The unit will provide for
annual review and revision of the UAS.
Obtaining valid and reliable results will be vital to the success of the
assessment plan. It is the unit’s goal
to make the judgment-based evaluation process determining candidates’
professional readiness, systematic and objective as it can be while focusing on
the most important attributes of performance relevant to the standards specific
to each program.
The validity of the
assessment is maximized by being sure about the purpose, defining the candidate
characteristics to be evaluated, specifying levels of performance along
appropriate continuums and through articulate rubrics, using exercises that
sample the range of performance contexts, and comparing ratings with other achievement
data when possible. The reliability of
the assessment is maximized by using clear criteria, continuously training
raters, carefully attending to planning, and implementing appropriate scoring
procedures. These activities are
strengthened by stakeholder collaboration, especially faculty collaboration
across content area departments.
Data collected, and the
implementation of the UAS, will be reviewed annually, at the close of the
spring semester, by the UAC. Summaries
will be prepared and distributed to appropriate stakeholders. Stakeholders will be gathered in the
pre-school in-service meeting in August.
The UAS will be a standing agenda item.
Central questions to be asked are:
·
How adequately is
performance described at benchmarks and between?
·
How adequately is
data recorded/reported?
·
How adequately,
and to whom, are results communicated, i.e., candidates, Director Teacher
Education Student Services, Faculty, Committees, Dean, institution and
departments, and other appropriate stakeholders?
·
What are the most
critical pieces of performance data for candidates in each program?
·
What is the
relationship of the data to the Unit’s conceptual framework?
·
Does the
assessment system assess the alignment between standards-based performances and
assessment?
·
Is the assessment
system clear and understandable?
·
Are assessment
results consistent across sources and across time?
·
Is the assessment
system feasible from the perspective of all stakeholders?
·
Does the
assessment system use multiple and diverse assessment strategies to fairly
assess candidates’ performance?
·
Does the
assessment system tell us what we want and need to know when we want and need
to know it?
·
To what extent
does the assessment evidence provide: 1) a valid and reliable measure of the
standards? 2) sufficient information to support inferences about each candidate’s
readiness to teach? 3) meaningful opportunities for candidates to exhibit the
knowledge, performances, and dispositions through authentic performance tasks?
UAS
Summative
Decision Point: Admission to Professional Studies in Initial Licensure
Programs