Test Development Coordinator

About NCARB

The National Council of Architectural Registrations Boards (NCARB) is a nonprofit organization made up of the architectural licensing boards of 55 states and territories. While each jurisdiction is responsible for regulating the practice of architecture within its borders, NCARB develops and administers national programs for licensure candidates and architects to ensure they have the mobility to go wherever their career takes them. Learn more at www.ncarb.org.

About the Team

The Examination team manages the day-to-day development and delivery of the ARE and works on other Council-wide initiatives. Our team works with hundreds of volunteer subject matter experts (SMEs) on the creation and maintenance of the ARE. We oversee ongoing improvements to the examination, and we support Council outreach efforts to inform licensure candidates and architects on all NCARB programs and services.

About the Position

The National Council of Architectural Registration Board’s (NCARB) Examination Department is searching for a collaborative Test Development Coordinator to join our team and work together on developing and delivering the Architect Registration Examination® (ARE®).

This position reports to the Assistant Vice President, Examination and requires significant and frequent collaboration with others across the department and with our volunteers. We take great pride in maintaining a high stakes examination that ensures proper assessment of architectural licensure candidates. If you are a motivated person who thrives in a collaborative work environment, can lead and empower others to reach goals and objectives, and can learn and speak confidently about new information and ideas, then we want to hear from you.

The Test Development Coordinator is responsible for working with Examination team members and NCARB volunteers on the development and maintenance of a fair, valid, reliable and legally defensible licensure exam. This position works collaboratively, both in person and virtually, with other members of the Examination team and with NCARB’s SMEs and testing partners in support of NCARB and Examination team goals. Examples of tasks include examination item editing and reviewing, item referencing, item bank maintenance, test form assembly, and assisting volunteer item writers by facilitating item review sessions, preparing training materials and homework assignments, and supporting test development committee efforts.

Work Location 

During the current pandemic, this position is 100% remote which requires a dedicated workspace. This is subject to change.

Duties

As part of your duties, you will:

• Provide both in-person and virtual support to SMEs throughout their item development work, including training on item authoring and form review, managing homework assignments, and providing oversight of item development workflow.
• Coordinate with Exam Managers on the overall workplan for each committee responsibility.
• Develop and review SME training materials covering all aspects of item development work.
• Facilitate item writing meetings with SMEs.
• Monitor examination item banks, including identification of areas with significant gaps.
• Edit newly authored items on a regular basis, ensuring that form, structure, and style are consistent with psychometric and NCARB standards and content fits and fulfills approved test specifications.
• Confirm that complete and accurate information about test items has been recorded in the item bank.
• Confirm that examination graphics are clear, accurate, appropriately annotated, and approved for use on the exam.
• Coordinate the identification, approval, and listing of reference and resource materials.
• Interpret basic item statistics and use the concepts of reliability and validity to facilitate SME item review.
• Conduct item-level statistical review discussions with SMEs to identify reasons and possible solutions for aberrant test item performance.
• Assist with the coordination of test launch schedules and activities, including quality assurance of test forms prior to launch.
• Support Examination team projects and initiatives related but not limited to general test development efforts, committee meeting preparation, ARE communications, and candidate support.
• Maintain the security of confidential test, candidate, and vendor information.
• Ability to travel up to 25%, including weekends.

Qualifications

At a minimum, you have:

• Undergraduate degree in industrial/organizational psychology, English, education, mathematics, or psychometrics or related fields.
• Excellent understanding and command of English grammar, style, and the principles of effective written expression in order to edit and evaluate volunteer item writing.
• Excellent oral communication skills.
• Effective presentation ability in order to communicate test development concepts, progress, and goals to volunteers, and the ability to self-assess and improve presentation methods when needed.
• Strong organizational skills to work effectively in a team setting on concurrent processes and projects to meet established goals, deadlines, and quality standards.
• Strong interpersonal and facilitation skills to work effectively with staff, volunteers, and stakeholders to accomplish NCARB’s goals and objectives.
• Ability to interpret and convey basic test development-related data to staff and volunteers.
• Excellent attention to detail and accuracy.
• Solid computer software skills, including Office, PDF software, and item authoring and banking applications.
• Ability to prioritize and manage multiple projects simultaneously to achieve required deadlines.
• Experience successfully coordinating volunteer activities.
• Two or more years of experience working in exam development ideal, preference for experience with licensure and/or certification organizations.

Benefits

NCARB's employees enjoy a comprehensive set of benefits that include strong support for health and wellness, some nice perks, and professional development. We also offer a great work environment that is thoughtful and collaborative. It is filled with lots of light and awesome colleagues who believe in our mission and embrace our values of being open, responsive, and restless.

Some highlights include:

• Competitive pay
• Medical, vision, and dental Benefits
• Paid parental leave
• Life and disability insurance
• 401k
• Tuition reimbursement
• All-staff outings and retreats

If you are qualified and feel excited about this position, please apply directly using the link below. We would love to hear from you!

https://ncarb.bamboohr.com/jobs/view.php?id=22

NCARB is an EEO employer committed to recruiting a diverse workforce.