aspose file tools*
The moose likes Web Services and the fly likes Incompatibility side effects of refactoring Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Web Services
Bookmark "Incompatibility side effects of refactoring" Watch "Incompatibility side effects of refactoring" New topic
Author

Incompatibility side effects of refactoring

jannetta steyn
Greenhorn

Joined: Nov 30, 2008
Posts: 4
Hi Folks

I'm researching the incompatibility side effects of refactoring on web services. I'm a correspondence student an thus quite isolated. At the moment I'm at a bit of a loss. I would appreciate any thoughts or comments that might get the brain juices flowing again.

So focussing on web services. Firstly, what would you guys classify as a web service. SOAP is one. RMI and CORBA are services but not web services because they don't use web protocols. What other web services are there?

Secondly. Have any of you done some refactoring on web services that might have or have affected the compatibility of the service. So, in short, have any of your refactorings, intentionally or unintentionally, affected the clients of the service. This includes functional and non-functional requirements. A refactoring could have caused you to remove a parameter in a message and therefore the client's program broke, or, because of your refactoring, your service performance suffered and clients were affected by that. How were they affected?

Any thoughts on these questions will be GREATLY appreciated.

Kind Regards
Jannetta
[ November 30, 2008: Message edited by: jannetta steyn ]
Jimmy Clark
Ranch Hand

Joined: Apr 16, 2008
Posts: 2187
So focussing on web services. Firstly, what would you guys classify as a web service. SOAP is one. RMI and CORBA are services but not web services because they don't use web protocols. What other web services are there?


SOAP is not a web service. It is an XML-based language that is used to implement a web service. It is only a part, there are other parts besides SOAP instances.

RMI and CORBA are not "services." They are older protocols which are used for distributed processing.
[ November 30, 2008: Message edited by: James Clark ]
Peer Reynders
Bartender

Joined: Aug 19, 2005
Posts: 2922
    
    5
Originally posted by jannetta steyn:
I'm researching the incompatibility side effects of refactoring on web services.


By definition "refactoring" of the service logic should have no negative side effect on the client as refactoring doesn't change any features - therefore the service contract shouldn't change. The only time this could happen: if you develop a service "contract-last" as opposed to contract first, as "contract-last" development increases Contract-to-Logic coupling.

Now when you refactor service collaborations/aggregations then contracts may have to change. But that is a complex topic. The book "Service Oriented Modeling: Service Analysis, Design and Architecture" discusses this to some extent.

Firstly, what would you guys classify as a web service.

Common types of web services
  • SOAP(/WSDL-based) web services
  • RESTful HTTP web services
  • POX-over-HTTP web services (Plain Old Xml)

  • Have a look over the web services FAQ
    This includes functional and non-functional requirements.

    "Refactoring" does not change any features. Therefore changes in requirements cannot be addressed with "refactoring" - they require "re-working" which is often preceded by "refactoring".
    [ November 30, 2008: Message edited by: Peer Reynders ]
    jannetta steyn
    Greenhorn

    Joined: Nov 30, 2008
    Posts: 4
    Hi James

    Thank you very much for your response.

    Originally posted by James Clark:


    SOAP is not a web service. It is an XML-based language that is used to implement a web service. It is only a part, there are other parts besides SOAP instances.


    This is true. I need to get my terminology correct.


    RMI and CORBA are not "services." They are older protocols which are used for distributed processing.


    Is there any reason why you can't create a service on the Internet using these protocols? I would think that you can. It wouldn't be a web service, but I think anything you can offer with a web service you could offer with these too?

    Please correct me if you think I'm wrong.

    Kind Regards
    Jannetta
    Jimmy Clark
    Ranch Hand

    Joined: Apr 16, 2008
    Posts: 2187
    Enterprise Javabeans is based on the RMI protocol. You can implement services with Session EJBs.

    The "web service" paradigm is an evolution of Enterprise Javabeans (in the context of distributed processing).

    For calling a Session EJB, you need to use JNDI and have client API's and application server libraries on the client application machines.

    For calling a web service, all you need to do is send a SOAP message via HTTP URL and get the response sent from a SOAP Engine.
    [ December 01, 2008: Message edited by: James Clark ]
    Peer Reynders
    Bartender

    Joined: Aug 19, 2005
    Posts: 2922
        
        5
    Originally posted by jannetta steyn:
    Is there any reason why you can't create a service on the Internet using these protocols?


    Firewalls. Binary protocols are considered to be too dangerous to be allowed to pass through. Text (e.g. HTML, XML) and a select number of IANA mediatypes are considered "safe-enough" and essential.
    [ December 01, 2008: Message edited by: Peer Reynders ]
    Peer Reynders
    Bartender

    Joined: Aug 19, 2005
    Posts: 2922
        
        5
    Originally posted by James Clark:
    The "web service" paradigm is an evolution of Enterprise Javabeans (in the context of distributed processing).



    Where do you see the open standards of XML, XML Schema, SOAP, and WSDL for SOAP web services being influenced by the proprietary specifications for Enterprise Javabeans?

    In terms of "distributed processing" Enterprise Javabeans are at a disadvantage because they lead to high Contract-to-Technology coupling (just like MS Enterprise Service/COM+/DCOM components) - SOAP web services are based on Open Standards (platform independent) reducing the Contract-to-Technology coupling. SOAP web service technology has more in common with CORBA (which could by used by non-Java clients to access EJBs because of the Internet Inter-ORB protocol(IIOP)). But even CORBA was about distributed objects (which violates Fowler's first law of Distributed Object Design) while SOAP web services are about services.
    [ December 01, 2008: Message edited by: Peer Reynders ]
    jannetta steyn
    Greenhorn

    Joined: Nov 30, 2008
    Posts: 4
    Hi Guys

    Thanks for the points raised. I would say though, that the things you mentioned do not mean you *can't* create a service with other protocols, but rather that it would be advisable not to. The reasons being firewalls, coupling etc.

    My focus though is the effect of refactoring on a service. Peer is correct in saying:


    By definition "refactoring" of the service logic should have no negative side effect on the client as refactoring doesn't change any features - therefore the service contract shouldn't change.


    However, as Lippert says, "Unfortunately, modifactions of subsystems cannot always be limited to internal implementations. occasionally, an API must be adapted as well in the course of a refactoring." He then goes on to discuss refactorings and to which extent they will effect the compatibility of the API.

    SOAP services would be much less effected by some of these refactorings than for instance a service created RMI because of the fact that SOAP does not require libraries and interfaces used on the service (as James mentioned). So I'm trying to determine which refactorings will effect a web service, to what extent, and how can I test it's effect.

    A simple API change, I would say, is relatively easy to check for. But what if the refactoring effects performance rather than functionality. In the article by Fowler that Peer mentioned (thanks for this reference Peers), Fowler discusses for instance why objects shouldn't be distributed and why. Let's say we refactor an existing system and reduce the distribution of objects. We would then expect an increase in performance. So again, I want to determine what types of refactorings would effect what - performance, latency, reliability etc. And again the question - how can I test this?

    One of my first thoughts when reading your comments was that a SOAP web service could still be considered to be a distributed system. This seems to be confirmed in the Fowler article in the section Interfaces for Distribution.

    Great chatting to you guys, please keep it going if you have the time


    Kind Regards
    Jannetta

    [ December 02, 2008: Message edited by: jannetta steyn ]

    [ December 02, 2008: Message edited by: jannetta steyn ]
    [ December 02, 2008: Message edited by: jannetta steyn ]
    Jimmy Clark
    Ranch Hand

    Joined: Apr 16, 2008
    Posts: 2187
    Where do you see the open standards of XML, XML Schema, SOAP, and WSDL for SOAP web services being influenced by the proprietary specifications for Enterprise Javabeans?


    These are low-level technical details that are a few levels below the concept of "distributed processing."

    The disadvantages of the Enterprise Javabeans paradigm were addressed by the SOAP-based web service paradigm. This is why I stated that "web service" is an "evolution" of Entrprise Javabeans.

    Enterprise Javabeans is an evolution of CORBA.

    There is a timeline for advancements in distributed processing.

    CORBA ---> RMI ---> RMI-IIOP ---> EJB ---> WEB SERVICES
    [ December 02, 2008: Message edited by: James Clark ]
    Jimmy Clark
    Ranch Hand

    Joined: Apr 16, 2008
    Posts: 2187
    Thanks for the points raised. I would say though, that the things you mentioned do not mean you *can't* create a service with other protocols, but rather that it would be advisable not to. The reasons being firewalls, coupling etc.


    Yes. The concept of a "service" is generic. You can create services with Bash or KornShell. In fact, you can develop an entire service-based application in shell scripts. Note, these would not be "web services" however. These would be just services in the context of whatever application or organization that creates them.

    The concept of a "web service" has a defined meaning and is generally understood.
    Peer Reynders
    Bartender

    Joined: Aug 19, 2005
    Posts: 2922
        
        5
    Originally posted by jannetta steyn:
    However, as Lippert says, "Unfortunately, modifactions of subsystems cannot always be limited to internal implementations. occasionally, an API must be adapted as well in the course of a refactoring." He then goes on to discuss refactorings and to which extent they will affect the compatibility of the API.


    With services this situation can arise when you are "refactoring" a service collaboration which can be either implemented as an "orchestration" or "choreography" (the former has a central orchestrating service, the latter does not). However in most cases modification of the provider's service contract will require modification of the consumer. The only exception is when the provider adds a new optional element to one of the XML documents that it uses. Existing consumers that don't use the optional element don't care and new consumers can use the new element - existing consumers that want access to the optional element will have to update. This compatibility is only achievable if the provider's service contract drives the generation of the service skeleton - if the contract is derived from the service implementation even the most insignificant change can lead to incompatible changes in the derived contract.


    SOAP services would be much less affected by some of these refactorings than for instance a service created RMI because of the fact that SOAP does not require libraries and interfaces used on the service.
    Can't really agree here. Even if RMI was universally supported changing the interface of the provider component will tend to break it's existing clients. The same is true for SOAP web services. The WSDL strictly defines the format of the SOAP requests that are accepted and the format of the SOAP responses that result.
  • Add a new mandatory element to the SOAP request - existing consumer breaks.
  • Remove an existing element from a SOAP request - requests from existing consumer using that element will be rejected.
  • Remove an existing element from the SOAP response - existing consumers relying on that element will break.
  • Add a new element to the SOAP response - some existing consumers may break because they always unmarshal the entire response even when only part of it is needed - the new element is "unrecognized".


  • SOAP is at its core still very RPC and therefore shares most of the drawbacks of the RPC mechanisms that preceded it.

    I finally get REST. Wow.
    The tools make the technology and the tools (and the examples and the advice you get) point at RPC.

    The other interesting point is that some of the "fragilities" are the result of how SOAP tools are implemented. XML Schema and WSDL are fairly strict so it is fairly easy to build code generators that use them. However that strictness also leads to brittleness. Changing the schema or WSDL invalidates the previous version often breaking even consumers that were not even interested in the changed portion of the schema/WSDL because the code generated by the SOAP tool insists on unmarshalling everything, not just the bits that the consumer is interested in.
    Rule of thumb regarding data sharing on the internet: "Be lenient in what you accept and strict in what you produce."
    "lenient acceptors" are not produced by code generators. "lenient acceptors" would make interactions more resilient to change but they require purposeful (and time-consuming) development.

    One of my first thoughts when reading your comments was that a SOAP web service could still be considered to be a distributed system.

    A SOAP sender and SOAP receiver are intended to connect across machine boundaries - there is no point to using SOAP inside the same machine boundary. Why use web services at all if you don't have to distribute? More importantly a remote invocation (call to a web service method) has very different characteristics with respect to latency, memory access, concurrency, and partial failure compared to a local invocation (method call on a local object) and therefore the difference between remote and local invocations should never be abstracted away by the programming environment.

    IEEE Internet Computing July/August 2008; Vol. 12, No. 4: Convenience Over Correctness (PDF)
    Sun TR-94-29 November 1994: A Note on Distributed Computing
    [ December 02, 2008: Message edited by: Peer Reynders ]
    Peer Reynders
    Bartender

    Joined: Aug 19, 2005
    Posts: 2922
        
        5
    Originally posted by James Clark:
    This is why I stated that "web service" is an "evolution" of Entrprise Javabeans.


    SOAP Web Services and Enterprise JavaBeans may have common ancestry but in no way did SOAP Web Services evolve from Enterprise JavaBeans.
    Java Enterprise Edition may have adopted SOAP Web Services after Enterprise JavaBeans but that is in no way indicative of an evolution.
    Peer Reynders
    Bartender

    Joined: Aug 19, 2005
    Posts: 2922
        
        5
    Originally posted by jannetta steyn:
    I would say though, that the things you mentioned do not mean you *can't* create a service with other protocols, but rather that it would be advisable not to.


    Service-orientation doesn't actually dictate any particular protocol. SOAP/WSDL/WS-* are merely seen as enabling technologies in a heterogeneous distributed environment where collaborating services are potentially under the control of different owners. It's only stated that proprietary or platform specific protocols like JRMP exhibit high Contract-To-Technology Coupling (and through indirect coupling high Consumer-To-Technology Coupling), while open protocols like SOAP exert lower Contract-To-Technology and Consumer-To-Technology coupling. The Service Loose Coupling Principle prefers lower Contract-To-Technology and Consumer-To-Technology coupling.
    [ December 02, 2008: Message edited by: Peer Reynders ]
    Jimmy Clark
    Ranch Hand

    Joined: Apr 16, 2008
    Posts: 2187
    SOAP Web Services and Enterprise JavaBeans may have common ancestry but in no way did SOAP Web Services evolve from Enterprise JavaBeans.


    I imagine that there are different interpretations of the definition of the term "evolution" and the different ways it can be used in a sentence. You and I most likely are not thinking along the same lines and are reading my statement in different ways. In the context of my sentence, "common ancestry" and "evolution" are comparable.
    [ December 02, 2008: Message edited by: James Clark ]
    Peer Reynders
    Bartender

    Joined: Aug 19, 2005
    Posts: 2922
        
        5
    Originally posted by James Clark:
    In the context of my sentence, "common ancestry" and "evolution" are comparable.

  • Humans and chimpanzees share common ancestry.
  • Humans evolved from chimpanzees.


  • Evolutionists would agree with (1) and get really upset about (2). Hardly comparable.

    For EJBs to be on the "evolutionary path" to SOAP web services, they would need to be on the direct development path towards SOAP web services. Considering the primarily "Microsoft" pedigree of SOAP's origin, you could make a case for them wanting to avoid all the weaknesses that they perceived in EJB.

    Yes, both EJBs and SOAP web services are technologies for "remoting support" but that is where the commonalities end. The fact that you can expose a SLSB as a SOAP web service is a gimmick (an ill advised one at that).
    [ December 02, 2008: Message edited by: Peer Reynders ]
    Jimmy Clark
    Ranch Hand

    Joined: Apr 16, 2008
    Posts: 2187
    For EJBs to be on the "evolutionary path" to SOAP web services, they would need to be on the direct development path towards SOAP web services.


    This is "your" perception of what the term "evolution" means when applied to computing technology. It is a very short-sighted perspective, in my opinion. Whole-brain thinkers might disagree, maybe not. Either way, JavaRanch enables us to share our ideas and opinions. This is a great thing.

    Hail to the Big Moose!
    [ December 02, 2008: Message edited by: James Clark ]
    jannetta steyn
    Greenhorn

    Joined: Nov 30, 2008
    Posts: 4
    Hi Guys

    I would like to bring the discussion back to the points that I raised in the beginning if you don't mind. I need to focus on my research for the moment.

    I have been building a very simple SOAP service for illustrative purposes, using PHP. Originally my intention was to use Java, but PHP just turned out to be a much quicker solution to prove some points. However, I might have to switch back to Java, or at least write some parts in Java because I think, because PHP is not strictly typed, one can get away with quite a bit more than you would be in Java. PHP also allows optional parameters to functions/methods, while in Java one needs to use overloading.

    "lenient acceptors" are not produced by code generators. "lenient acceptors" would make interactions more resilient to change but they require purposeful (and time-consuming) development.


    Would you say that a well written client should be much more tolerant to certain changes? I am not interested at this point in code generators.

    Regards
    Jannetta
    Peer Reynders
    Bartender

    Joined: Aug 19, 2005
    Posts: 2922
        
        5
    Originally posted by jannetta steyn:
    I have been building a very simple SOAP service for illustrative purposes, using PHP. Originally my intention was to use Java, but PHP just turned out to be a much quicker solution to prove some points. However, I might have to switch back to Java, or at least write some parts in Java because I think, because PHP is not strictly typed, one can get away with quite a bit more than you would be in Java. PHP also allows optional parameters to functions/methods, while in Java one needs to use overloading.


    And how does this relate to service refactoring?

    Right now you are getting caught up in the challenges of "implementation language <-> service message mapping", i.e. how different langauges marshal to the service messages and unmarshal from the service messages. While an interesting topic, it has little to do with the "refactoring of services". If anything, you would have to ensure (and show) that the choice of your demonstration implementation language does not lead your examination to highlight problems that are inherent to the implementation language and its message processing libraries rather than focusing on problems with the refactoring of services in general.

    I'm researching the incompatibility side effects of refactoring on web services.


    All web serivces have one thing in common - they use the HTTP protocol. Applications can use plain TCP over the internet - but if they don't use HTTP they aren't using the web. An HTTP client can send an HTTP request to the HTTP server which then responds with an HTTP response. That's it - no other message exchange patterns are supported. An HTTP request consists of a request line (method, path, HTTP version), headers, and an optional body. An HTTP response consist of a status line (http version, status code, textual phrase), headers, and an optional body.

    So a (technical) web service contract consist of a definition of all the requests that the service will accept and of the responses that it will return. So scenarios that need to explored are, for example:

    Is the web service client adversely affected if:
  • A request/response message is removed from the service contract?
  • A request/response message is added from the service contract?
  • The structure/contents of a request is changed (adding/removing/moving something)? What changes can't be tolerated? What changes can be tolerated (given what circumstances/assumptions)?
  • The structure/contents of a response is changed? What changes can't be tolerated (adding/removing/moving something)? What changes can be tolerated (given what circumstances/assumptions)?


  • Now in the case of SOAP web services the HTTP request/response always carries a SOAP envelope (or a MIME envelope that contains the SOAP envelope and any associated attachments) in the HTTP Body. The SOAP envelope can carry SOAP headers and a SOAP Body. A SOAP web service defines its technical contract through a WSDL which essentially defines what SOAP request and response messages can be exchanged (in the binding it mentions that SOAP envelopes will be transported over HTTP).

    Other web service types when compared to SOAP web services may use different and more numerous IANA media types as the content in the HTTP message body and they may use HTTP headers more heavily. RESTful HTTP web services may use WADL to describe their technical contract - however non-SOAP web services often use non-formal documents to describe the service contract.


    The point is that the service contract is the "API" of a web service (not the PHP or Java consumer artifact that developers work with). The service contract defines the messages that are legal. If "an API must be adapted in the course of a refactoring" then messages defined in the service contract are added, removed or changed. So an examination of the "incompatibility side effects of refactoring" has to focus on the web service consumer's capacity for tolerating changes in the service request and response messages, and what measures, if any, can improve or impair such tolerance. Some actions on the service provider side may also improve or impair the consumer's tolerance.

    Would you say that a well written client should be much more tolerant to certain changes?


    "Well written consumer" is too vague. It's a matter of deliberately developing code that forms only the most essential dependencies on the messages that are exchanged.

    I am not interested at this point in code generators.


    I don't see how you can ignore them without losing relevance. A lot of the hype around SOAP web services was based on the "inherent robustness" of text-based semi-structured data (i.e. in well formed XML). XML Schema was added for definition and validation (which really only is an issue when you cross a trust boundary) but it potentially reduced robustness. Most people use code generators (you probably used one with your PHP examples) to generate the consumer side artifacts which they then use to access the web service.

    Lets say there is a web service that gives me the business entity "PurchaseOrder" (PO) as an XML document. All the developer wants is the "Total" on the PO. A code generator will use the defining XML Schema to generate the consumer side artifacts forming dependencies on the entire PO document. Depending on the quality of the code generator the consumer side unmarshalling could easily fail if it encounters an element that it doesn't recognize (it may have been added at a later time). On the other hand the developer could simply use a carefully crafted XPath expression to extract the total from the "PurchaseOrder" XML document. That handwritten code would likely tolerate many more change scenarios than any generated consumer artifact would, simply because the handwritten verison focused on only the most essential dependencies in the document. So ultimately code generation can make service consumers more brittle in a "refactoring situation".
    [ December 05, 2008: Message edited by: Peer Reynders ]
     
    Don't get me started about those stupid light bulbs.
     
    subject: Incompatibility side effects of refactoring