CareerTech Testing Center

The CareerTech Testing Center (CTTC), a division of the Oklahoma Department Of Career and Technology Education (ODCTE), develops and maintains skills standards and competency assessments for Oklahoma’s CareerTech system. The division works closely with instructors, program administrators, industry representatives, and credentialing entities to ensure skills standards and assessments reflect national standards and local industry needs. Organized in 1980, the CareerTech Testing Center currently offers skills standards and competency assessments for more than 100 career majors. The CTTC’s online competency assessment system delivers approximately 75,000 assessments per year.

By the Industry for the Industry

In competency-based education, industry professionals and certification standards identify the knowledge and abilities needed to master an occupation. CareerTech uses industry input to develop instructional materials that help prepare the comprehensively trained, highly skilled employees demanded by our workplace partners.

Tools for Success

CareerTech programs rely on three basic instructional components to deliver and evaluate the effectiveness of competency-based instruction:
Skills standards provide the foundation for competency-based instruction in Oklahoma’s CareerTech system. The skills standards outline the knowledge, skills, and abilities needed to perform related jobs within an industry.
Curriculum materials contain information and activities that teach students the knowledge and skills outlined in the skills standards.
Competency Assessments determine occupational readiness by testing the student over material outlined in the skills standards and taught using the curriculum materials.
Each of these components satisfies a unique purpose in competency-based education; however, they work together to reinforce the knowledge and skills students need to gain employment and succeed on the job.

Skills Standards

Skills standards list occupations in an industry and identify the knowledge, skills, and abilities employers expect of workers in those occupations. Skills standards developed by the CareerTech Testing Center include recognized national standards and workplace skills.

Uses of skills standards:
Skill Standards provide direction for students, instructors and program administrators in the journey towards workplace readiness.
Specifically, skills standards help CareerTech instructors:
􀀳 improve program planning,
􀀳 track individualized instruction,
􀀳 facilitate discussion with local advisory committees,
􀀳 perform self-evaluations, and
􀀳 evaluate curriculum and instruction.
Skills standards also help CareerTech program administrators:
􀀳 identify current job descriptions,
􀀳 document industry standards,
􀀳 guide program evaluations, and
􀀳 anticipate program growth and future challenges.
Components of Skills Standards:
Skills standards include several basic components that provide important information about jobs within an industry:
1. Occupation - Skills standards are divided into occupations. An occupation represents a combination of skills for which education, training, and employment occurs.
2. Duty - Each occupation is further separated into duties. A duty defines a major area of job responsibility.
3. Task - Duties contain one or more related tasks. A task is a measurable and/or observable unit of work and requires the application of one or more skills.
4. Additional components - Other Skills Standards components describe individual tasks and provide direct linkages to curriculum resources and competency tests.
a. Clarifying information denotes general topics and subject matter that are specifically associated with an individual task.
b. Task codes identify individual tasks within a duty area and are used to encode all test items referencing that task on the competency assessment for an occupation.
c. Frequency Ratings represent how often the task is performed on the job; Criticality Ratings denote the importance of performing the task correctly. Although rating scales may vary among occupations, CTTC typically uses a rating scale from 1 to 3. A frequency rating of “3” indicates the task will likely be executed at least once daily, while a criticality rating of “3” signifies that it is extremely important that the task be performed correctly. Conversely, a “1” rating indicates the task will be performed less than once weekly or that it is only slightly important that it be performed correctly.
d. Curriculum crosswalks list commonly used curriculum resources and reference tasks to specific curriculum units within those resources. Crosswalks also provide a concise way to evaluate how the skills standards align to national skills standards and/or prepare the student for industry and/or professional certifications.
How Are Skills Standards Developed?
Once the need for instructional materials for an occupational area has been determined, the CareerTech Testing Center researches existing industry materials and develops trial skills standards. A panel of subject matter experts convenes and reviews the trial skills standards. The trial skills standards serve only as a starting point; the subject matter experts may adopt national standards or opt to change the document to more accurately reflect the knowledge and skills currently demanded in the workplace.
During the validation process, the panel reviews and refines every aspect of the skills standards. Individual tasks, duty areas, and entire occupations can be added, deleted, rephrased, or reorganized to reflect actual occupational expectations. Whenever possible, the committee incorporates and/or aligns the skills standards to industry-recognized standards or certification requirements.
Once the panel validates the organization and content of the skills standards, it assigns Frequency and Criticality Ratings to the individual tasks. This step solidifies industry’s “mark” on the skills standards, since these ratings drive competency assessment development. Each of the duty areas should also be “weighted” by the panel of subject matter experts. This is “weighting” should be done by percentage and according to the importance of the overall duty area (the combined total of all duty areas should equal 100%).
Finally, the CareerTech Testing Center makes the new skills standards available to educators and helps instructors use them effectively their programs.
Benefits of Skills Standards
Skills standards offer the direct benefits of providing a roadmap for training CareerTech students and verifying that instruction addresses industry needs. However, encouraging programs to “train to the skills standards” benefits all partners in the training process.
Students Receive Training That Enhances Their Education
Skills standards developed by the CTTC combine national standards with input from industry. Therefore, training to these skills standards enables students to pursue credentials that are recognized across an industry and prepares them for workplace success.
Consistency of Training Provides Students Portability
The consistency of programs on a statewide basis increases portability of students between programs and enhances their ability to secure college credit for training in a career and technology education program. Industry can access a statewide pool of highly qualified, occupationally competent applicants trained to its specifications. Businesses can hire completers of career and technology programs with confidence in their technical knowledge and skills, regardless of where the student received training.
Local Programs Become the Catalysts for CareerTech Success
Local programs can prepare students for a national job market without sacrificing the flexibility to customize training that satisfies the needs of the businesses in the communities they serve. Training to skills standards enables programs to prepare students for technical careers and provide highly skilled employees to industry.
Keeping It Real
In order to ensure that CareerTech programs provide training that meets the demands of our industry partners, skills standards must be accurately reflect workplace expectations. Generally, skills standards developed by the CTTC undergo revision every three years. Skills standards for occupations in industries that evolve quickly may be revised more frequently; those for static occupations and/or occupations tied to certification or licensure requirements may not be revised as often.
Obtaining Skills Standards
Skills standards are available at no charge and may be downloaded from the CareerTech Testing Center’s website, 1. Information about alignment to industry standards and endorsement of CTTC skills standards by professional organizations can also be accessed through this site.

Curriculum

Curriculum materials provide the means to CareerTech’s end: training students for career success. Curriculum materials facilitate instruction of skills and knowledge necessary to master an occupation to industry standards. In addition to complementing classroom instruction, curriculum resources provide supplemental activities to enhance learning and provide hands-on training experiences.
In occupational areas, the Skills Standards serve as a guide for identifying curriculum materials and are used by CareerTech instructors and specialists to select or develop materials for use in the classroom. The Curriculum and Instructional Materials Center (CIMC) serves as the instructional materials resource for Oklahoma’s CareerTech system and, with the help of program specialists and administrators, determines the need for new development based upon the availability of existing products.
Development of instructional materials begins with the Skills Standards. Business and industry professionals, subject matter experts, and instructors create an instructional analysis that facilitates the development of learning objectives, which drive the content included in the curriculum. Curriculum materials also undergo a validation process by business and industry to verify content and instructional flow. Finished curriculum products are inserviced and available for purchase through CIMC and the Customer Service division.
While most products are print-based, CIMC delivers curriculum products using a variety of media. A comprehensive collection of videotape titles compliments CIMC’s printed materials. Products utilizing computer-based and web-based training are also developed for use in CareerTech programs. For a comprehensive listing of products available or to order CIMC products, visit CIMC’s online catalog2.

Competency Assessments

Competency assessments bring the process of competency-based education full circle. Skills Standards define the skills required for occupational success; competency assessments measure how well the student has mastered these skills. Competency assessment measures occupational readiness in two portions: performance evaluations and written competency assessments.
Performance Evaluations
Performance evaluations simulate the workplace and evaluate specific criteria of related tasks in an occupational area. These evaluations require students to demonstrate skill by completing an actual segment of work using tools, materials, and equipment characteristic to the occupation being tested. Performance evaluations encompass all of the duty areas related to an occupation.
Instructors drive the performance evaluation portion of competency assessment. Performance evaluations typically come from curriculum resources that include LAPs or job sheets; however instructors may also develop performance evaluation instruments.
Written Competency Assessments
Written competency assessments provide an effective means of measuring factual and theoretical knowledge related to an occupation. Written competency assessments are summative assessments that should be administered after the student has received training and passed all performance evaluations. Written competency assessments include objective-based, multiple choice test items and usually require no more than one hour to complete.
Development of Written Competency Assessments
Skills standards determine the length, structure, and content of CTTC written competency assessments. Frequency ratings, criticality ratings, and the number of clarifying information items provide an objective, mathematical way to develop tests that assess mastery of the knowledge and skills identified in the skills standards.

Using values and information in the skills standards, the CareerTech Testing Center determines the test specifications and contracts with subject matter experts to develop test items. When writing test items, subject matter experts typically reference materials identified in the curriculum crosswalk that is included in the skills standard, which reinforces the connection between standards, instruction, and assessment.
A committee of subject matter experts reviews the test and carefully scrutinizes individual test items. Specifically, the committee validates the structure and content of each question and verifies the question has been keyed correctly.
Once the development process is complete, the CTTC prepares the written competency assessment for online delivery.
Written Competency Assessment Revision
Written competency assessments undergo minor revisions annually. Minor revisions involve reviewing test items for timeliness, revising test items that performed poorly, and making other corrections that improve item presentation. Major revisions typically coincide with skills standards revisions and involve determining new test specifications, adding and deleting items to the item bank, and recoding existing items to reflect task changes. Subject matter experts are involved in every aspect of the written competency assessment revision process.
Test Delivery
Written competency assessments can be administered by any comprehensive high school and technology center that offers career and technology education programs. Each school or campus designates a Testing Liaison to serve as that site’s contact person for all competency assessment matters. Testing Liaisons administer competency assessments to students in a web-based format and are responsible for maintaining the security of tests and the online testing system. In addition to administering competency assessments to students, Testing Liaisons also communicate assessment results to instructors.
The CareerTech Testing Center provides training to Testing Liaisons and supplies website announcements concerning testing policies and procedures, new testing products, and other testing issues.
Written Competency Assessment Results
Students receive a results report (coaching report) upon completion of a written competency assessment. Results reports communicate competency assessment scores and provide a breakdown of assessment results by duty area. The results breakdown shows how well the student has mastered knowledge and skills needed to perform major job functions and identifies areas of job responsibility that may require additional instruction and/or training.
Online testing enables results reports to be provided immediately to students and instructors. Students who score 70% or greater on a written competency assessment receive a certificate noting competency attainment for the occupation tested. This certificate not only identifies the occupation in which competency was attained, but also lists industry and supplemental certifications earned by the student. Students who do not attain competency receive additional training in areas of deficiency and can retest once remediation is complete.

Testing Roles

It is critical that the CTTC’s competency testing process is secure, valid, and affords each test taker equal opportunity to demonstrate knowledge and skill in an occupational area. Therefore, all parties must know their role and demonstrate integrity when performing testing responsibilities.
Instructors
􀀳 Identify skills standards that are appropriate for their program. Skills standards serve as a guide for curriculum development and/or selection; therefore this step provides the framework instruction.
􀀳 Determine the appropriate assessment(s) for their program. The test(s) administered must assess knowledge related to an occupation after training is completed.
􀀳 Coordinate written competency test times with the testing liaison. Written competency tests should be administered upon successful completion of related training and performance evaluations. Remember: Written competency assessments alone do not adequately indicate competence and must be used with performance evaluations to determine occupational readiness.
Testing Liaisons
􀀳 Provide a secure and safe environment for testing. The testing environment should be such that test takers can concentrate on their assessments with minimal distractions.

Considerations regarding the testing environment include:
􀂃 Consistent/adequate lighting levels
􀂃 Temperature at a comfortable level with proper ventilation
􀂃 Space is quiet with minimal distractions
􀂃 Test takers should be asked to behave consistently (no eating, getting up and moving about)
􀂃 Avoid/delay the test administration when a participant appears hurried, troubled, or ill
Responsibilities of the Test Proctor include:
􀂃 Participant authentication: a picture ID should always be shown and login should be handled quickly and quietly by the proctor
􀂃 Protection of the security of the online testing system. Username AND password should NEVER be revealed
􀂃 Prohibiting the use of all communication devices (photos of test items and text messaging are common problems)
􀂃 Monitoring computer usage: Prevent access the internet or other programs
􀂃 Coordinating a secure process for accessing and administering written competency assessments through the online testing system
􀂃 Vigilance in the observance of the testing environment: Note passing, hand gestures, etc.
􀂃 Monitoring the use of reference materials, texts, notes, etc. References may not be used unless specified in the testing instructions
􀂃 Providing accommodations for students with an IEP as specified in an IEP, IRP, 504, LEP, and ELL
􀂃 Serving as a information resource for local instructors on testing-related issues
􀂃 Establishing a secure storage system for test results
􀂃 Distribution of individual results and group test analyses to instructors

Analysis of Competency Testing Data

The CareerTech Testing Center uses competency testing data to provide performance feedback on test items, tests, students, programs, and occupational areas. The “Policy on Distribution of Test Results” outlines the guidelines used by the CareerTech Testing Center to conduct data analyses and distribute testing-related reports.
Data on test items and assessments assists assessment specialists and review committees in the test revision process. Students and instructors use individual results reports to document competency and identify areas that require additional instruction.
Group analysis of student results provides feedback to instructors seeking to improve the effectiveness of career and technology training. Performance patterns in individual duties indicate opportunities to evaluate training methods and customize instruction. Instructors and schools may also use group analysis to satisfy other reporting requirements.
Local administrators and program administrators at the ODCTE use statewide analyses of written competency data to monitor program effectiveness and anticipate growth and future challenges in CareerTech programs.

Alternate Assessments and Advanced Certifications

Alternate assessments are assessments that may be taken in lieu of written competency assessments developed by the CTTC. The CareerTech Testing Center works with program administrators to identify alternative assessments for each program. Alternative assessments must align to the skills standards in and typically include industry certifications, licensure examinations and assessments offered by other entities.
In some instances, students completing a CareerTech program must meet additional requirements before attempting certification or licensure. Advanced certifications are credentials that students may seek after satisfying experience and/or higher education prerequisites.

External links

• CareerTech Testing Center homepage
• CareerTech Testing Center Blog
• Oklahoma Department of Career and Technology Education
• Curriculum and Instructional Materials Center
• Creative Services
• Curriculum and Instructional Materials Center
• Multistate Academic and Vocational Curriculum Consortium
• Printing Services

See also

• List of school districts in Oklahoma
• List of private schools in Oklahoma
• List of colleges and universities in Oklahoma