REST based services are now preferred ways of developing web services. In the past , i have seen the discussion on other forums that REST based services can eliminate the need of middle ware(broker based architecture), mainly asynchronous messaging. What is your opinion on this? REST based services(mainly http) are primarily synchronous. How we can achieve message delivery guarantee, scalability,caching which are much doable in broker based architecture etc
The short answer is that if you want message delivery guarantee, scalability and caching (typical MQ broker qualities) then you should use a message broker. Don't expect or try to make REST do MQ things. It was not designed for that. And precisely because they are quite different is why I don't see that they are competing technologies. They solve different problems and often complement each other.
For communication between microservices within the same organizational space then a message broker offers the best set of features such as guaranteed delivery, reliability, encryption, clustering, efficient wire-level protocol, async behavior, and so on. REST operates over HTTP which is not reliable and message delivery is not guaranteed. However, the REST pattern is far more expressive and is ideally designed for use by external developers who don't know (or need to know) more than just the resources the endpoint represents. So a possible architecture is that the services within your organization communicate over a message broker but communicate with the outside world via a REST interface (possible via a gateway or something similar).
There may be use cases where using REST internally is the best option and it is often used by microservice developers to allow services to communicate with each other.