I am using the URLConnection and HttpURLConnection classes to screen-scrape some information from a webserver, and am wondering what's the best way to determine if HTTP authentication is required to access a specified URL.
When I access the URL and examine the response headers sent back, I can see that there is a response header with a null field name that has the following value:
Out of curiosity, why is there a response header returned with a null field name?
This is the full list of response headers generated by the web server when I try to access the password-protected URL:
Is it sufficient to check the HTTP response code (401 in my case above) to determine if password authentication is required when trying to access such a URL?
Salman: Is it sufficient to check the HTTP response code (401 in my case above) to determine if password authentication is required when trying to access such a URL?
Yes and No! Yes because 401 indicates that an authorization is required. No because it may not be password authentication
This is what the RFC 2616, section 10.4.2 states:
10.4.2 401 Unauthorized
The request requires user authentication. The response MUST include a WWW-Authenticate header field (section 14.47) containing a challenge applicable to the requested resource. The client MAY repeat the request with a suitable Authorization header field (section 14.8). If the request already included Authorization credentials, then the 401 response indicates that authorization has been refused for those credentials. If the 401 response contains the same challenge as the prior response, and the user agent has already attempted authentication at least once, then the user SHOULD be presented the entity that was given in the response, since that entity might include relevant diagnostic information. HTTP access authentication is explained in "HTTP Authentication: Basic and Digest Access Authentication" .
So, checking the response code is sufficient but beware that you must not get into an infinite loop when the authorization fails!