Transcript Presentation on NMLS Testing
SAFE Mortgage Loan Originator Test Webinar September 10 th , 2009
NMLS Testing & Education
State Regulatory Registry LLC a CSBS company
Your Hosts
• Anne Altman – Manager, Mortgage Testing Programs – aaltman@csbs.org
202.728.5750
• Pete Marks – Programs – pmarks@csbs.org
VP, National Mortgage Testing & Education 202.728.5723
• Rich Madison – Director, Mortgage Education Programs – rmadison@csbs.org
202.728.5737
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Conference Call Logistics
• *6 to mute and un-mute your call • Please do not put your phone on hold • Please identify yourself and your organization when asking a question
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Agenda
• • • • •
SAFE MLO Test:
Background Info & Test Specifications
Pieces of the Test Development Puzzle
– 1. Subject Matter Experts – 2. Content Outlines – 3. Test Questions – 4. Review Committees – 5. Test Forms
Test Administration
– Preparing to Take the Test – The Test Center – Computer Test Operations and Functionality
Other Testing Information Q & A Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
SAFE MLO Test: Background and Overview
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Background Information
• SAFE Act requires NMLS to develop tests for MLO’s • NMLS contracted FINRA and its subcontractor, Pearson VUE, to develop the SAFE MLO tests.
• NMLS contracted FINRA to administer the SAFE MLO tests using its current dual-vendor channels - Pearson VUE and Prometric’s high-stakes testing facilities.
• SAFE Mortgage Loan Originator Test – National Component – State-Specific Test Component
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
National Test Component Basics
• All mortgage loan originators must take and pass the national test component • Test questions = 100 – 90 scored – 10 unscored/pretest • Passing score = 75% • National Test Component launched July 30 th
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
State Test Components Basics
• Test questions = 55-65
(varies; determined by state)
– 45-55 scored – 10 unscored/pretest • Passing score = 75% • 10 state tests currently available – AZ, IA, ID, LA, NH, NC, PA, RI, VT, WA – More state tests to be released shortly
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
SAFE Deadlines for Testing
Both National and State Components
– All new licensees: July 31, 2010 – All existing licensees: December 31, 2010 – Several states have elected to have earlier deadlines * Please check with each particular state’s regulatory authority
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Pieces of the Test Development Puzzle
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Part 1. Subject Matter Experts (SME’s)
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
SME’s are:
• selected by each states’ regulators and approved by NMLS • well versed in laws and regulations • a mix of state regulators and industry professionals • able to devote their time, effort, and expertise to the development of the SAFE test
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
“What do SME’s really do?”
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
SME’s:
serve on one or both of the following test development committees: – National Mortgage Review Committee – State Test Development Committee
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
SAFE National and State Test Committees
• Reviewed and provided input on national or specific state test component content outlines • Examined, revised, and approved all test questions on national and state test components • Will participate in future content outline and test question maintenance cycles
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Part 2. Content Outlines
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
“What are Content Outlines?”
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Here’s what they look like…
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Here’s what they look like…
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Content outlines are:
• The basic structure or blue print for the test • The roadmap for constructing test questions • A tool to help candidates prepare for the test
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
“Tell me more about content outlines …”
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Content Outlines
• Each test component has its own outline – national component – all state specific components • Outlines map out: • specific test content •
and
question distribution (weightings)
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
“How were the content outlines created?”
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Test Development
•
Fast Track
– national and first 11 states (AZ, IA, ID, LA, MA, NH, NC, PA, RI, VT, WA) •
Normal Track
– all remaining states
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Fast Track Content Outlines
National Component
• Drafted by regulator work group • Reviewed and approved by national committee
State Components
• Template established by regulator work group • Approved by national committee • Customized for each state by state specific test committee
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Normal Track Content Outlines
Job Analysis
• In April 2009, a job analysis survey was conducted • Individuals from the 55 states/jurisdictions were invited to participate • Invitations were sent to regulators and industry professionals • Over 10,000 people responded
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Normal Track Content Outlines
The job analysis survey results:
– produced only minor revisions to state outline template weightings – validated the original national outline
Moving forward:
– all future states will use new content outline template – during annual test maintenance cycle, FT states will incorporate new outline template
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
“Can I access these content outlines?”
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Yes, Absolutely!
Outlines are on the NMLS Resource Center Professional Requirements> Testing> Test Availability Schedule and Content Outlines Currently posted: – National outline – State outlines • AZ, IA, ID, LA, MA, MI, NH, NC, PA, RI, VT, WA • DC, GA, KY, MD, NJ, VA, WY (just posted)
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Content Outlines
Nationwide Mortgage Licensing System content outlines
State Regulatory Registry LLC a CSBS company
And we should also mention…
• Content outlines are an overview of what will be on the test. • They can be utilized as a study tool for test takers.
No additional study guides or study materials will be developed or endorsed by NMLS. Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Coming Soon:
More Outlines
In early November, more content outlines will be posted (SD, NE, AR, NM, NY, MS) • As new states come on board, new outlines will be developed and published • Check back often
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Part 3. Test Questions
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
“How are test questions created?”
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Test questions are:
• Written by subject matter experts • Reviewed, edited, and approved by committees • Created from content outline categories and subcategories • Supported by reference materials (laws, regs, etc.)
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Test Questions Debunked
• All questions are 4-option multiple choice • No short answer, fill-in-the-blank or true/false • Only one correct answer and 3 incorrect options • No “All of the above” or “None of the above”
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Test Questions Debunked
• No trick questions – No questions with multiple correct answers – No questions without a correct answer • Good wrong answers • No proper names – “ A borrower is applying for a loan...”
not
– “John is applying for a loan…”
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
“Why are some questions not scored?”
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Un-Scored (Pretest) Questions
Un-scored questions are being “tried out.”
We only want to put onto our tests questions that meet high quality standards.
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
The Process for Un-Scored Questions
1.
2.
3.
4.
We gather statistics on them first When we have enough “hits”,
psychometricians
analyze the stats They are reviewed to determine if a question is problematic Eventually, questions that “make the cut” become scored questions
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Un-Scored Questions:
• Do not affect a candidate’s score • Are not marked in any visible way to the test-taker
(candidates can’t distinguish between a scored or an un-scored question)
• Are scattered randomly throughout the test
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
“What if a law changes that affects a test question?”
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Test and Item Bank Maintenance
• All questions are reviewed annually • If a new law affects a question , it will be addressed • During maintenance, pretest questions can become scored and old scored questions may be retired • After maintenance, new versions of the tests are released • If an outline changes , it will be updated online (with new date posted)
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Test and Item Bank Maintenance
What if a law change goes into affect prior to the scheduled maintenance cycle?
• Updates to laws and rules are consistently monitored • If a question is found that is no longer accurate, the following steps may occur: – SMEs may be called on to access the validity and edit or revise if necessary – A question may be retired and replaced with another
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Part 4. Review Committees
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Who is included in the State Review Committees?
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
State Review Committee
•
6 – 8
state industry representatives •
Experienced
in mortgage industry (time and professional associations) •
Balance
of regulators and industry professionals • Requires
time
and
dedication Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
“How much
time
and
dedication
does this take?”
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Virtual Meetings
• All meetings are
conducted by Webinar
• There are a total of
8-9 meetings
• The committee has
one meeting a week
• The committee agrees on the
same day and time
for each weekly meeting
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Why virtual meetings?
• Maximum
flexibility
in scheduling and requires only a computer and a separate phone line • Pearson VUE can work with multiple states
concurrently
• Each committee member can see test questions directly with
no security risks
of paper copies • Ability to track progress and unique issues
customize the next steps
, including which items to present, depending on the state’s
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
What do the meetings accomplish?
• The
first meeting
is focused on
developing content outline
and is
2 hours
• The next
6-8 meetings
are focused on
approving test questions
and are
4 hours
each
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Same Goal, Different Role
•
Test development
knowledge versus
content
knowledge – Pearson VUE is expert in how to test – Subject matter experts know what to test • Every
process step
is introduced by a
training
presentation of about 30 minutes
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Committee Process Steps
•
Review content outline template
and develop a
customized, state-specific test blueprint
•
Review test question root pool; customize content and approve questions for use on test
•
Develop new test questions
for unique state-specific content areas •
Classify test questions
outline to the areas of the content •
Reference test questions
and rules to state law, regulations,
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Part 5. Test Forms
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
“What are test forms?”
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Test Forms
•
Definition:
Forms are the different versions of a test •
Purpose:
The purpose of having multiple forms in a test is to enhance test security
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Test Forms
• For example, the national test component has 90 scored and 10 un-scored questions.
• The different forms (versions) of the test all have the same number of questions .
• To ensure fairness and reliability, different forms of the national test are all of equal difficulty .
• Questions present in random, computer-generated order each time for each form.
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Test Administration
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Preparing to Take the Test
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
MLO Testing Handbook
Nationwide Mortgage Licensing System
Everything you need to know State Regulatory Registry LLC about testing
Enrolling: Step-by-Step Instructions
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Test Enrollment Basics
• A candidate does not have to complete education prior to enrolling or taking a test.
• Before enrolling, a candidate must have a unique NMLS ID number
(must either already have made a filing or created a base record ).
• • A candidate must request and pay for an enrollment in NMLS before scheduling a test session
(*enrollment fees are non-refundable – listed on testing page)
.
Each enrollment window is open for 180 days the candidate must schedule to take the test.
, during which
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
“After candidates create enrollments, how do they schedule test appointments?”
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Next Step: Scheduling a Test
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Basics of Test Scheduling
• To schedule, must have an NMLS ID number • 500 test centers for candidates to choose from (Pearson and Prometric) • After selecting a state on “Find a Test Center” page, a list of all test centers in that state will appear • Scheduling – online or over the phone
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Basics of Test Scheduling – More Info
• All tests are available at all test centers
(i.e. candidate may take AZ test component in PA)
• A company can request a group appointment (see Prometric or Pearson web sites for more info) made if *Special accommodations requests may also be necessary.
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Basics of Scheduling – More Info
“Can a test-taker schedule the national test component and a state component on the same day?”
Yes, but only if a test center has two back-to-back appointments available. However, please encourage candidates not to overlap their testing appointments or they can run out of time!
(*The test time for the national component is 3 hours and each state component is 2 hours.)
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
“What if a candidate needs to reschedule a test appointment?”
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Rescheduling a Test
• •
Must contact Pearson VUE or Prometric directly to reschedule Must be rescheduled no later than 12 p.m. local time two business days prior to appointment
Appointment scheduled for: Must be cancelled or rescheduled no later than 12:00 p.m. local
Sunday, Monday Tuesday Wednesday Thursday Thursday of the preceding week Friday of the preceding week Monday of the same week Tuesday of the same week Friday Wednesday of the same week Saturday Thursday of the same week
*If a holiday occurs during the normal cancellation/reschedule period, this schedule adjusts to define "timely cancellation" as one day earlier than stated above.
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
The Test Center
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Entering the Test Center
• Arrive 30 minutes prior to test time • Must bring a current, government-issued ID with candidate’s photo and signature .
• Will be photographed ; finger or palm print may be taken • Candidate may not bring anything into testing room
(may be asked to show empty pockets)
• All belongings such as notes, cell phones, watches, wallets must be placed in locker • Basic calculator (not financial calculator) and scratch paper are provided • Must read and agree to the Rules of Conduct
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Test Security-- Policies
• All candidates must read and agree to the “Rules of Conduct” prior to taking the test. • They stress the confidentiality of all of the test questions and test materials, as they are the copyrighted, proprietary property of NMLS.
• Any attempt to modify, publish, transmit, or create derivative works of the test materials, in whole or in part, is a direct violation of copyright laws, the Agreement and the Rules of Conduct and will subject you to federal and/or state sanctions.
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Test Security--Policies
• Also, when creating a base record, NMLS users agree to use the system and its test
originator license” .
subject to other sanctions .
“
only for activities related to obtaining or maintaining a mortgage loan
• Candidates who violate or may violate the Agreement, the Rules of Conduct, or copyright laws, who own, operate, are employed by, or are otherwise affiliated with organizations that are NMLS Approved Course Providers or organizations that intend to seek NMLS Course Provider Approval status, may cause their NMLS Course Provider applications to be denied or their Approval status to be revoked, suspended, or
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Computer Test Operations and Functionality
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Test Time-- National
National Component
• 2.5 hours test time (150 minutes) • 15 minutes allotted for tutorial (before test) • 15 minutes allotted for optional survey (after test) • Total seat time = 3 hours • 100 questions in 150 min. = 1.5 min. per question
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Test Time – State Components
State Test Components
• 1.5 hours test time (90 minutes) • 15 minutes allotted for tutorial (before test) • 15 minutes allotted for survey (after test) • Total seat time = 2 hours • Approx. 60 questions in 90 min. = 1.5 min. per question *State test lengths vary from 55 to 65 total questions
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
“What is the computer test functionality? Is it simple to use? How does it work?”
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Tutorial and Test Functionality
Tutorial walks candidates step-by-step through test functionality. Some key features that candidates can utilize: – Toggle back and forth between test questions – Mark a question for review and then come back to it later on – Change an answer
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Exit Survey
• The exit survey is completely optional.
• All of the data collected in the survey is monitored and shared with us. • Candidates may also provide feedback to the test center administrator on the enrollment process, test center facilities, content of test questions, etc. All comments will be reported.
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Score Reports
At the end of the test session, candidates will: • see their test scores on their computer screens • receive a paper copy of their score report These scores do not become official until they are entered in NMLS. Within 48 hours of completing a test, candidates will receive an email informing them that their official scores are now ready to be viewed in NMLS.
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
“What happens if a candidate fails?”
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
If a candidate fails…
Score report has a diagnostic table that informs candidates of how they performed on each outline content area.
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Retake Policy
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Retake Policy
• Must wait 30 days between each test taking attempt • May create an enrollment and schedule an appointment, but can’t take the test prior to the end of prescribed wait time • After every fourth failed attempt, a 180 day wait time is instituted *After passing a test component, a test taker will not be allowed to retake that particular test component.
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Other Testing Information
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
State Test Certification
• Some states that had valid tests prior to the passage of the SAFE Act may elect to certify their state tests.
• In these cases, candidates who previously passed that state’s test may not be required to take the SAFE state test component.
• Candidates must check with their state regulators.
• All candidates, however, must take and pass the national component.
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Test Enrollment Update
National AZ IA ID LA MA NH NC PA RI VT WA Total Enrollments 1230 8 53 15 40 7 19 232 96 9 1 64 1774 Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Tests Taken Update
National AZ IA ID LA MA NH NC PA RI VT WA Total Tests Taken 318 3 17 8 21 2 7 46 17 0 1 30 470 Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Any Questions?
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company
Your Hosts
• Anne Altman – Manager, Mortgage Education Testing Programs – aaltman@csbs.org
202.728.5750
• Pete Marks – Programs – pmarks@csbs.org
VP, National Mortgage Testing & Education 202.728.5723
• Rich Madison – Director, Mortgage Education Programs – rmadison@csbs.org
202.728.5737
Nationwide Mortgage Licensing System
State Regulatory Registry LLC a CSBS company