Microsoft has developed a specific version of CHAP, called MS-CHAP (Microsoft Challenge Handshake Authentication Protocol version 1, sometimes denoted as MS-CHAP-v1), improving the overall security. Indeed, CHAP requires that passwords are transferred in plain text over the network, which is a potential vulnerability. MS-CHAP provides a hash function to store (via a hash) the password on the server. When the remote machine responds to the challenge, and it has to hash the password using the proprietary algorithm.
Unfortunately the MS-CHAP-v1 protocol suffers from security vulnerabilities related to weaknesses in the proprietary hash function.
Version 2 of MS-CHAP, MS-CHAP-called V2 was set in January 2000 (RFC 2759). This new version of the protocol defines a so-called "mutual authentication" method, allowing the authentication server and the remote machine to verify their identities. The process is as follows:
The authentication server sends a verification request (session identifier and a random string) to the remote client.
The remote client responds with:
its user name,
a hash containing arbitrary string provided by the authentication server, the session ID and password,
a random string.
The authentication server checks the response from the remote client and in turn send:
a notification of success or failure of the authentication
an encrypted response based on the random string provided by the remote client.
The remote client then in turn verifies the response and if successful, establishes the connection.