I came across a strange scenario where user-agent is coming null and due to which our application is breaking. It is strange for me for user-agent being null.
Please clarify the below queries.
1) What are the cases when user-agent will be null for a web client?
2) Is user-agent being null according to W3C standards?
I'm not sure if this is the place this query has to be posted. Can any one of you please help me out with the clarification. It would be really greatful!!!
As far as I know if someone is running a program to crawl your web site, in such a case user-agent header may not be set.
Even some web stripping software may allow to change the user-agent and even not set it.
It could also be some one is running a malicious program to scrape your site.
It could be someone is running a diagnostic program checking your site.
It could be a search engine indexing your site (I think most search engines would include some kind of identification in the user-agent header).
The http 1.1 specification says "User agents SHOULD include this field with requests."
The link is here
There could be other reasons why such a thing could happen..
Some browsers (or browser extensions) allow people to suppress the header from being sent, or an arbitrary value to be sent. This is sometimes used when badly developed web sites exclude users based on which browser they use.