 |
|
|
Program Educational Objective Evaluation Process Establishing Program Educational Objectives
The department
has a long tradition of external evaluation dating to 1970. Periodic surveys of both alumni and their
employers were routinely performed and acted on. The department was the source of the current
campus student opinion surveys starting in 1971. The department was also the point of
initiation for Industrial Advisory Boards (now more commonly called the
Advisory Boards) beginning in the mid 1970s.
The design of the
continuous improvement system began in 2000 and was followed by a staged
collection of materials beginning in the 2001-2 academic year. During the subsequent two years, the system
was continually refined and brought to full implementation. Although informal reviews and system
refinements were occurring on a weekly basis throughout 2001-2003, the first
comprehensive objective review involved all data collected up to the end of
2003. This initial “closing of the
loops” occurred during the Spring Semester of 2004. With the substantial faculty retirements
(Stone, Han, and Marquis) from 2005-2007, subsequent biannual Advisory Board
reviews were renewed in 2007 with the newly contracted faculty (Medlin, West,
and Cross). During the period
from 2001 to 2004 the entire department faculty has met once or twice a week
during the academic year to create the continuous improvement system now in
place. Dr. Howard attended several conferences on ABET methodology during the
2001-3 period. Dr. Howard trained as an
ABET evaluator in the period 1999-2000 period.
Dr. Kellar has also attended ABET training sessions for chairs. Program faculty members have attended
numerous campus sessions on continuous improvement methodologies. Since the 2004
ABET review, three of the five full-time tenured track faculty members have
retired and been replaced: Drs. Medlin
and West were unfamiliar with the department’s continuous improvement system
while Dr. Cross being an internal candidate from the department research
program had some familiarity with the system.
Extensive training was provided these new faculty members. As of 2010, all program faculty members are
well versed and directly involved in supporting and managing the continuous
improvement system. All teaching faculty
members in the metallurgical engineering program are actively engaged in
periodic reviews of the program educational objectives. The
program faculty members proposed initial ABET-conforming program objectives in
2001. During the subsequent year, program constituents were asked to review the
objectives. The first review was
conducted by the 2002 Advisory Board, followed by reviews in 2004 by
comprehensive and often overlapping assemblages of program constituents grouped
as follows: Primary ·
Constituent focus
groups ·
Alumni surveys ·
Employer surveys ·
Graduate student
surveys ·
Recent outstanding
graduate awardees surveys ·
Advisory Board
reviews Secondary ·
SDSM&T constituent
departments surveys ·
SDSM&T undergraduate
student opinion surveys ·
SDSM&T
Student Satisfaction-Importance (SSI) Survey After this pre-2005 exhaustive
review of program educational objectives for use in our ABET-conforming
continuous improvement process, a more streamlined review has been employed
that relies on the Advisory Board, alumni
surveys, and constituent focus groups. These
groups overlap but have unique functions and input methodologies both of which
preclude consolidating the program educational objective review under the
purview of any one group. For example, the
Advisory Board may engage in the review of departmental personnel matters,
which would be inappropriate to consider in a body including students. There are many such matters requiring the use
of several groups to complete a review of program educational objectives. The secondary sources above are not
specifically employed since the departments are represented in the current
constituent focus groups, opinion surveys are confidential, and the
Satisfaction-Importance Survey is more germane to assessment functions. Since 2004 the program objectives
have been reviewed by the ·
Advisory Board ·
Constituent focus
group ·
Alumni survey. The composition
of the 2007-2012 Advisory Board is as follows:
The department
holds biannual meetings with its Advisory Board to conduct a review of Program
Objectives and the department’s success in achieving them. The review also includes a re-examination of
the objectives to assure they are current and significant. Materials presented to the board include the
results of alumni and employer surveys, which are designed to gauge the extent
to which program graduates are achieving the program objectives. The board members are selected to represent
as many of the program’s constituents as possible. The Advisory Board is provided the most recent
survey information from alumni, current students, constituents, etc. making
their review the most comprehensive whereas the alumni surveys and the
constituent focus groups are generally asked to offer input on specific topics
such as the currency of the program education objectives. However, every group is encouraged to offer
any constructive comments they wish. Alumni surveys query all
alumni who graduated since the previous alumni survey. This means all alumni are asked for their
input. Owing to the variation in numbers
of program graduates, alumni surveys are grouped in four-year groups to
preserve anonymity and ensure large enough numbers for statistical
meaningfulness. A total of 54 alumni
were surveyed using Survey Monkey in 2008 with 51 responses. This unusually high response is in itself an
indication of the strong, favorable relationship between faculty and program
graduates. The three responses not
received were caused by incorrect email addresses. As shown in Table 2-1 the Advisory
Board reviews are scheduled every two years; alumni surveys every four years;
and constituent focus group meetings every four years. The two surveys are staggered by two years
since some of those surveyed are the same individuals. This staggering precludes pestering some
alumni for the same information in the same year yet, assures important
constituent input for objective input. The schedule was revised in 2006 to
restart the cycle in 2007 because of the 60 percent replacement in program
faculty that occurred between 2005 and early 2007. Restarting the cycle in fall of 2007 provided
for all the new faculty members to participate in a common evaluation review
cycle early in their tenure. The program faculty
members review all program evaluation input every two years after
the results of the Advisory Board are available.
The review culminates with action statements. Figure 2-1 shows
a schematic of the continuous improvement process used by the metallurgical
engineering program to determine progress towards program objectives. Figure 2-2 shows this process interfaced with
the process to determine progress in meeting program outcomes.
Achievement of Program Educational Objectives
The following process is used
to determine the extent to which program educational objectives are being
achieved: ·
All program
alumni are specifically surveyed on the achievement of the program
objectives. The survey asks for their
self-assessment. It also asks for input
related to the achievement of the objectives, such as the extent of community
and professional service. The survey results
are used by the program faculty to evaluate the level of attainment of the
objectives. ·
Program
constituent focus groups are convened to review the program’s direction and
level of achievement. The focus group
includes employers, alumni, and other departments who engage our graduates
(most often as graduate students). The
focus group is led by a non-program campus professional to assure free exchange
and anonymous input who writes a summary report to be used by the program
faculty in their evaluation. ·
The Advisory
Board has all pertinent surveys and reports available to them when they conduct
their review. They are specifically
charged with reviewing the program educational objectives and progress in
achieving them. They submit a report of
their findings and recommendations. ·
The program
faculty members meet several times a month on program and departmental
matters. Many of these meetings are
specifically to review ongoing continuous improvement matters. Over a period of several meetings consisting
of an initial review, discussions, and a final review with written documents is
generated. Each biannual review begins
with reviewing the previous objective actions followed by a review of the
alumni survey, focus group report, and the Advisory Board reports. A summary is written as to the how the action
items were addressed and the result of the actions taken. Needed new action items for the coming period
are then formulated and documented. As described above, the
program faculty members review all program evaluation input every two years
after the results of the Advisory Board are available. The review process always consists of two action
categories: 1. Improvement in the evaluation process and 2. Improvement in the achievement of the program educational
objectives. The action statements
summarize actions taken within the evaluation process and the
curriculum. The departmental faculty meets several times
each month to discuss program operations and progress in the implementation
of needed program objective improvements.
The CIS Officer is monitors, tracks, and documents assessment
information and evaluation work. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||