Can service discovery and binding automated ?

[upul indrajith]

I have some knowledge in SOA. But I m really lacking particle sides of web services. I have some questions to ask

(1). Can the service discovery and binding of the discovered service automated in Web services. Is there any standard API support for this ?? (AXIS , JAX-RPC etc ... ) I mean when ever a client wants to consume a service , can it discover a service which is registered in a service registry (hopefully UDDI for web services ) and bind or invoke the service and get the desired out puts he wants ?

As far as I know , this service discovery should be done by some human , If its the case

(2). Do we need to have a service end point , every time to invoke or bind a service to a client ???

(3) Can we consume a web service with out a exact service end point and or a exact method signature as it described in WSDL if it a RPC style web service ???


[ November 18, 2008: Message edited by: upul indrajith ]

[Peer Reynders]

Originally posted by upul indrajith:
Is there any standard API support for this??

The standard API for accessing UDDI and ebXML registries is JAXR which is available under J2EE 1.4 and Java EE 5. Total automation of web service consumption by eliminating human intervention was even back in 2004 called "The Self-Organizing Myth". Service Level Agreement (SLAs) are required between organizations, sometimes even between departments before a client can rely on a web service. This means that human intervention cannot be eliminated. So registries are primarily in use in private environments where SLAs are already in place and where it is practical to impose quality controls, uniform interfaces and taxonomies. Furthermore automatic adaptation to evolving web service contracts (WSDLs) would be extremely difficult if not impossible.

Do we need to have a service end point , every time to invoke or bind a service to a client ???

You are not being clear here. A service never binds to a client.

A SOAP web service exposes its service to the "public" through its endpoint at a particular URL

The contract of that endpoint is usually expressed in a WSDL file. The WSDL file also contains the endpoint URL.

The client developer uses that contract (WSDL) to build or generate a client that complies with that contract.

The client software at runtime sends its requests to (and receives its responses from) the endpoint URL. So the service better be up and running, listening to its endpoint URL for requests when the client makes a request - otherwise the client will get an exception.

JAX-WS lets specify a WSDL (URL) when you create a service at runtime. That way you have to simply change the location of the WSDL and/or update the endpoint URL in the WSDL when the web service moves.

Can we consume a web service with out a exact service end point and or a exact method signature as it described in WSDL if it a RPC style web service

No

You have to know where something is before you can interact with it. Even if you are using a registry you have to describe to the registry what you are looking for. Then once the registry finds a contract that "vaguely matches", you still have to build the requests and understand the responses as outlined by the contract to use the service. Not something that is easily automated. Hence the realization by experts that the specialized interfaces that are defined in WSDL actually promote tight coupling.

With RESTful HTTP web services you can use XPath or even JQuery to extract the bits from the representation in the response that you are interested in. However you still have to have a vague idea of the representation's structure. And you still have to know how to formulate compliant requests.

This post might be of some interest.
[ November 18, 2008: Message edited by: Peer Reynders ]

[upul indrajith]

Hence the realization by experts that the specialized interfaces that are defined in WSDL actually promote tight coupling.

This article introduce a way to over come the tight coupling between a service provider and the service consumer at the run time.
[ November 21, 2008: Message edited by: upul indrajith ]

[Peer Reynders]

Unlike a tightly coupled approach, using the adapter approach presented in this paper leaves you with Web services invocation code that is highly reusable and configurable.

Basically the only thing that has changed is that the "coupling" has moved from stub "generation/compilation" to adapter "configuration" - the adapter now deals with the coupling that results from the specialized interfaces defined in the WSDL.

2007; Vol. 11, No. 1: REST Eye for the SOA Guy (PDF)

A significant advantage of the uniform interface constraint lies in the area of scalability. For a client to correctly interact with a SOA service, it must understand the specifics of both that service�s interface contract and data contract. But for a client to invoke a REST service, it must understand only that service�s specific data contract: the interface contract is uniform for all services. I can�t overstate this difference�s impact on large-scale systems. Imagine, for example, that the Web comprised millions of Web sites and that each defined its own special interface. To use your Web browser to interact with a particular site, you�d likely need to download or write a new browser plug-in that understood that site�s interface.

plug-ins ~ adapters

See also 2008; Vol. 12, No. 2: Demystifying RESTful Data Coupling (PDF)

[Jimmy Clark]

Tight coupling between a service provider and consumer at run-time is not a problem, in my opinion. The two applications need to connect at some point. At a syntax level, there can be zero coupling, and this is the benefit that a web service delivers. Any other type of "coupling metric" is not really significant enough to jump thru five or six more hoops, just to say "we have loose coupling in regards to xyz."
[ November 21, 2008: Message edited by: James Clark ]

[Peer Reynders]

Originally posted by James Clark:
Tight coupling between a service provider and consumer at run-time is not a problem, in my opinion. The two applications need to connect at some point. At a syntax level, there can be zero coupling, and this is the benefit that a web service delivers."

Tight coupling between two entities residing in the same process space, on the same machine isn't a problem because they are under the control of the same owner. Once you start distributing, the need for loose coupling increases, especially if the entities are under the control of different owners.

Any other type of "coupling metric" is not really significant enough to jump thru five or six more hoops, just to say "we have loose coupling in regards to xyz."

SOA in Practice: The Art of Distributed System Design p.37.

If there is such a fine list of aspects of loose coupling, and minimizing dependencies is good, you might be wondering why you don't simply use all these forms of loose coupling in each SOA. The answer is that there is a price to pay for loose coupling, in that it makes systems more complex. That means more development and/or maintenance effort.

p.36

The aim of loose coupling is to minimize dependencies. When there are fewer dependencies, modifications to or faults in one system will have fewer consequences on other systems.

So while "loose coupling comes at a price" opportunities shouldn't be squandered either - if you accept tight coupling you should be consciously aware of it and if you need loose coupling, then you need to be aware of the cost.

Sun TR-94-29 November 1994: A Note on Distributed Computing 6. Lessons from NFS

Hard mounts mean that the application hangs until the server comes back up. This generally prevents a client program from seeing partial failure, but it leads to a malady familiar to users of workstation networks: one server crashes, and many workstations�even those apparently having nothing to do with that server�freeze. Figuring out the chain of causality is very difficult, and even when the cause of the failure can be determined, the individual user can rarely do anything about it but wait. This kind of brittleness can be reduced only with strong policies and network administration aimed at reducing interdependencies. Nonetheless, hard mounts are now almost universal.

The other problem is that the static stubs (and adapters) introduce dependencies on the parts of the service interface that the client may not need. Even in object-oriented design this goes agains the Interface Segregation Principle (ISP (PDF))
[ November 21, 2008: Message edited by: Peer Reynders ]

[upul indrajith]

Well, I do have some thoughts. I just want to clarify them before I proceed.

Ok. We know dynamic bindings are impotent. But what is the point of having dynamic bindings with out having a proper discovery part ? . And how do you select the service to be invoked out of set of services ? Obviously the selection of the service should be based on some dynamic factors (availability performance etc...) .That is why we need to have dynamical binding of web services.

For example lets say there are 3 web services called A , B and C. All services are giving the same business functionality. And there can be any number of web services in the environment which serve the same functionality as A , B, and C.

According to my understanding the real dynamic binding mean invoking a service at run time which you are totally unaware of. To do that we need two thing first

(1). Finding of all the available services which meets consumers requirement - a service Discovery part

(2). Selection of the service which is based on some dynamic factors. - a service selection part

So when a service consumer may invoke either A , B , or C or any other available service in the environment. So the service discovery part can not be removed out from the scenario. (But in the article i provided above doesn't contain such a part.)

This is just a thought i have. What do you think about this.
:roll: . Am I correct about this argument or am I missed something here ??
[ November 22, 2008: Message edited by: upul indrajith ]

[Peer Reynders]

Originally posted by upul indrajith:
For example lets say there are 3 web services called A , B and C. All services are giving the same business functionality. And there can be any number of web services in the environment which serve the same functionality as A , B, and C.

The problem is that you cannot simply consume just any web service if every web service is allowed to make up its own service interface and data contract - and if they do, you are forced in advance to "adapt" to the existing services, so new services that use a different interface are still out of your reach. The problem with the "loose" versus "tight" coupling discussion is that the are many forms of loose coupling. Adopting loose coupling in one form may force you to accept tight coupling somewhere else. When you pursue "loose coupling" in one area you have to make sure that the gains outweigh the effort and that you can tolerate the necessary tight coupling in other areas to make it work.

Service discovery increases loose coupling because your physical connection is established via a mediator (the registry); a single fixed WSDL allows a simple point-to-point connection. However to make this form of loose-coupling work you have to standardize the service interface and data contracts in some way - this means that those services have to satisfy exactly the same standardized service interface and data contract (harmonization is tight coupling) or you have to pre-build adapters for a variety of existing interfaces that you can dynamically load based on some interface identification scheme ("looser" than the previous option but still not "loose"). In the first case a new web service can be easily used as long as it exposes itself through the standardized contract (WSDL); in the second case only new services exposing themselves through one of the previously identified interfaces with the appropriate identification scheme can be used.

The problem in a public environment is that by definition standardized contracts tend to commoditize the business services. So if the web service A, B, and C all expose themselves through a standardized WSDL, their options to compete are usually limited to data quality, service reliability and availability etc. However there is an incredible pressure to offer "value-added services (operations)" to the business services in order to de-commoditize them and gain an advantage over the competitors. Once the client starts to use those value-added services he is tightly coupled to that provider.

In a privately owned environment (single organization or industrial consortium) you can apply pressure to standardize the contracts so that web service A, B, and C expose themselves through exactly the same contract - now client's in the same space can use the "loose coupling" of "discovery" and the "tight coupling" of the standardized interface to improve service availability of the "virtual service" that emerges by having three equivalent concrete services. For example, if service A goes down for maintenance simply go to the registry and retrieve either B or C and use it instead. Notice that the same can be accomplished by simply knowing fallback services ahead of time - however the registry based approach is "looser" because it allows the actual services to move to different endpoint URLs. However the client has to know whether the additional complexity of using a registry is worth the achieved gain.

So "discovery" is a form of loose coupling but there is no point in accepting the complexity of discovery if you do not derive any benefit from it (though one shouldn't dismiss the potential benefits that swiftly either). However in SOA a business process can and will stretch "over a large and heterogeneous landscape of existing and new systems that are under the control of different owners" (SOA in Practice). So in that case you have to convince the different owners of the services that you consume that standardization and registration is to their benefit - that isn't an easy task.

So often one seeks solutions that are under one's own control. So it may be simpler to build a Proxy/Fa�ade (not to be confused with the Service Facade) around the static client stubs of the various non-harmonized (but equivalent) known web services. The static stubs ("tighter") can be traded for dynamically bound adapters ("looser") if the life-cycle of the Proxy/Fa�ade is dynamic enough(i.e. provided that the increased complexity generates a real benefit). Notice that this solution doesn't involve "discovery" at all. However addition of dynamically bound adapters does make it more configurable.



Your other concern seems to be orchestration. You have to realize that there is another option: choreography.

SOA in Practice p.298

Orchestration A way of aggregating services to business processes. In contrast to choreography, orchestration composes services to a new service that has a central control over the whole process.
For Web Services, BPEL is a standard for orchestration, for which development tools and engines are available.

SOA in Practice p.295

Choreography A way of aggregating services to business processes. In contrast to orchestration, choreography does not compose services to a new service that has central control over the whole process. Instead, it defines rules and policies that enable different services to collaborate to form a business process. Each service involved in the process sees and contributes only a part of it.

Interestingly when it comes to "control of process logic", "Central Control" is considered tightly coupled while "Distributed Control" is considered loosely coupled. So "orchestration" is more tightly coupled than "choreography". SOA itself doesn't really favor one over the other because they represent different trade offs - however it is usually easier to tool "Central Control" in SOA software foundation products. This in part can lead to the emergence of "hub-and-spoke" systems where the hub can become a bottleneck.

SOA in Practice p.297

Hub-and-spoke An abstract software pattern used to transfer data between multiple systems. In contrast to the bus pattern, it uses a central component that coordinates all communication between senders and receivers.

SOA in Practice p.294

Bus An abstract software pattern used to transfer data between multiple systems. In contrast to the hub-and-spoke pattern, it uses a federation of components that all follow a common policy or protocol to send, route, and receive messages.

Because of the perceived disadvantages of "hub-and-spoke" systems some have dismissed common SOA software foundation products and are looking into Event-Driven Architecture (EDA).

SOA in Practice p.296

Event-Driven Architecture (EDA) A software architecture pattern promoting the production, detection, consumption of, and reaction to events. Some consider EDA an extension or complement to SOA; others consider EDA to be part of the SOA approach (a special message exchange pattern where the service provider sends a message to multiple consumers)

[ November 23, 2008: Message edited by: Peer Reynders ]

[upul indrajith]

so new services that use a different interface are still out of your reach

In current SOA service consumers are depending upon the service implementation regardless of the business itself. (Since they bind the service contact of the service provider). There is no clear separation between the service consumer's business requirements and the implementation of these business requirements / functionality as various web services. This much of implementation details are really unimportant to the consumer. I mean why should the service consumer bother about the service provider's interface ? Will it be more logical if the service consumer bother about the business functionality or the service requirement regardless of the interface which is exposed as a WSDL. This part is missing in the current SOA standard model , so the tight coupling arise.

However to make this form of loose-coupling work you have to standardize the service interface and data contracts in some way

What if we build some ontology on top of the registry to define various business functions / requirements and let the consumer to inquire services as a business functionality request. If we keep a mapping between those business functions with various web service interfaces , we can easily find the services who serve the relevant service with various contracts. Then the service discovery will happen based on a particular business function rather than a service interface.
[ November 26, 2008: Message edited by: upul indrajith ]

[Peer Reynders]

Originally posted by upul indrajith:
In current SOA service consumers are depending upon the service implementation regardless of the business itself.

If the are, they are not following the best practices of SOA.

There is no clear separation between the service consumer's business requirements and the implementation of these business requirements / functionality as various web services.

If that is the case then SOA best practices are being ignored. The web service contract is about the "WHAT", it's implementation about the "HOW". The hard part about service design is the identification of fairly stable transition point boundaries between business processes, their sub processes, activities, or steps. Then you establish the service boundary through a set of these stable transition points so that the service gains autonomous control over the contained processes, sub processes, activities and steps. So anything that is contained inside the service becomes implementation - change to any of these aspects happens inside the service without affecting the contract and any other client services that depend on the contract. Now a service like that may be decomposed into smaller services - but that shouldn't affect the client services either.

Thomas Erl's SOA Principles of Service Design has a breakdown of the types of coupling that can occur between a consumer and provider in Chapter 7. Service Coupling (Intra-Service and Consumer Dependencies) p.194

Logic-to-Contract [Acceptable Coupling]. Tight Coupling of the service logic to the contract is acceptable and supported by the Standardized Service Contract Principle.

Contract-to-Logic [Negative Coupling]. This form of coupling is not recommended and can be avoided through the use of "contract first" design approaches.

Contract-to-Technology [Negative Coupling]. The service contract is ideally decoupled from vendor technology, as supported by the use of open XML and Web Services Standards.

Contract-to-Functional [Negative Coupling]. This negative coupling type is avoidable through the application of the Service Reusability Principle but may be required for certain types of services.

Contract-to-Implementation [Negative Coupling] This form of coupling is not recommended, especially in relation to external and shared implementation resources.

Consumer-to-Implementation [Negative Coupling]. The Contract Centralization Pattern is used to avoid this coupling type.

Consumer-to-Contract [Acceptable Coupling]. This is a positive form of coupling, but its benefit is related to the extent to which negative service contract coupling levels have been avoided.

So the only coupling that should generally be tolerated is the consumer's coupling to the contract and the coupling of the service logic to the contract.

I mean why should the service consumer bother about the service provider's interface?

Because the service provider's interface is the contract - to only thing where coupling is acceptable as long as all the other sources of negative coupling are kept in check.

Will it be more logical if the service consumer bother about the business functionality or the service requirement regardless of the interface which is exposed as a WSDL. This part is missing in the current SOA standard model , so the tight coupling arise.

It's "missing" because what you are describing isn't SOA.

What if we build some ontology on top of the registry to define various business functions / requirements and let the consumer to inquire services as a business functionality request.

Attempts have been made to include this in UDDI 4. However UDDI 3 seems to have already taken a different route. See also Loosely Coupled weblog; Friday, November 12, 2004: Trust, contracts and UDDI

Then the service discovery will happen based on a particular business function rather than a service interface.

And how to you intend to automatically connect to the service and take advantage of it if doesn't expose a standardized interface/contract?

To take advantage of the loose coupling offered by dynamic service discovery, the services in the registry have to expose themselves through standardized interfaces.

Developing standards takes time.

Business pressures on public services will tend to lead to the emergence of non-standard, value-added operations.

New types of services cannot expose themselves through a standardized interface - they are new, they are doing something for the first time, so standardization cannot have taken place.

One core idea behind dynamic, agile businesses is that they should be able to quickly take advantages of new emerging services and provide new services themselves.

The loose coupling of dynamic service discovery cannot provide this type of agility because contract standardization has to take place before "loose coupling" can come into effect.

In conclusion many organizations do not perceive a favorable cost benefit ratio when it comes to UDDI - it is simply not worth the hassle. So UDDI has been relegated to the status of a registry/directory for companies with large web service asset inventories. In such an environment you can impose service contract standardization; there is an opportunity to make your business processes more robust by providing redundant service providers on different parts of your hardware infrastructure if the clients know how to use the registries.
[ November 26, 2008: Message edited by: Peer Reynders ]