What HTTP authentication is all about

HTTP is able to use several authentication mechanisms to control access to specific websites and applications. Some of these methods use the 401 status code and the www authenticate response header.

The most commonly used HTTP authentication methods are:

Basic

The username and password are sent as an unencrypted base64 encoded text. You should always use HTTPS, as the password is not encrypted and can easily be captured and reused if you use HTTP and not HTTPS.

Digest

The credentials are passed to the server in hashed form. Although the credentials cannot be captured over HTTP, the request can be replayed using the hashed credentials.

NTLM

This method uses a secure challenge/response mechanism that does not allow  password capture or replay attacks if you use HTTP. It only work with HTTP/1.1 persistent connections. You cannot always use it with all HTTP proxies. Also you should not use this method if the connections are regularly closed by your web server.

How the HTTP authentication methods work

Basic Authentication

If you want to make your http transactions more secure, basic access authentication is a method you can use to provide a username and password when requesting a website or any other resource. When you open a website which uses HTTP authentication the server will send back a header requesting authentication for a given resource. You will be asked to provide your username and password in order to access the website. The credentials are not encrypted and that is why it is recommended not to use this method of authentication over HTTP and always use HTTPS.

Server side

The server will request the user agent to authenticate itself by sending a request for authentication using “HTTP 401 Not Authorized” code.

Client side

The user agent uses the authorization header, after which it sends the credentials back to the server and is created in the following way:

• The login credentials are combined into a string "username: password"
• The "username: password" string is encoded using the Base64 scheme

Digest Authentication

Digest access authentication is also used to negotiate credentials between the server and the user agent. A hash function is used on the password before it is sent back to the server.This authentication method is much safer than basic authentication, where the credentials are only encoded.

Steps

• Typically the user asks for a page that requires authentication but does not provide login credentials because he entered an address or followed a link to the page
• The server responds with the 401 "Unauthorized" response code, returning the authentication domain and a randomly-generated, single-use value called a nonce
• A pop-up window will open and the user will be prompted for a username and password. You can also click cancel at this point
• When you enter the credentials, the client will resend the same request but will add an authentication header
• If the username or password are not correct, the server will return the "401" code again

Advantages 

• The credentials are actually hashed (MD5 (username: realm: password))
• The nonce the server returns can contain timestamps. This information can be used to prevent replay attacks
• The server is can create a list of recently issued or used server nonce values in order to prevent reuse

Disadvantages

This method is vulnerable, and the client could not verify the server’s identity this way. Also, some passwords are stored using reversible encryption.

NTLM Authentication

NTLM is an authentication method developed by Microsoft and optimized for Windows platforms. NTLM is considered to be more secure than Digest.

NT LAN Manager (NTLM)  is a challenge-response authentication method that is a securer variation of Digest authentication. NTLM uses Windows credentials and not unencoded credentials and requires multiple interactions between the client and the server.

The NTLM authentication scheme uses much more resources than the standard Basic and Digest schemes and might influence performance. Once authenticated, the user identity is associated with that connection until it is closed. These connections cannot be re-used by users with a different user identity.

How to monitor HTTP authentication

You can use WebSitePulse performance and full-page levels of monitoring to make sure your clients are actually able to log in using HTTP authentication. All our transaction levels of monitoring support HTTP authentication.

For safety reasons, we suggest you to create a free test account with limited rights for the purpose of monitoring. Once you have the username and the password for the specific website, all you need to do is enter the details in the advanced configuration section of the specific target. Simply fill in the details in the Server Authentication section.

Sources:

• httpwatch.com    
• Wikipedia