John M Brown

Hi Suma,
I agree that you can find some other links that will build up your knowledge of REST 1st, because there are a lot of links that assume the basics are known or at such a high level that certain assumptions cannot be made from them...

A useful article from Zapthink gives a good explanation of how state works in a RESTful application. The last 2 letter of the acronym for REST is State Transfer, so this gives you an idea of how state is managed, but this article talks about application state vs. resource state. Ignore the fact it talks about Cloud computing as the explanation stands on its own: http://www.zapthink.com/2011/11/22/the-secret-to-a-restful-cloud/

As far as the WSDL comment, if you return hypermedia links in your representations to drive your app, you do not need a service specification (and very little documentation beyond the home URL or docRoot entrypoint to your app. If you are using XML as the format for your REST representations, then something like WADL can be used. If you are using JSON, then there is no current formal specification for JSON although there are groups working on that. A large part of the JSON community fears that using a specification in JSON will render it inflexible during design as WSDL has done with SOAP. I don't necessarily agree with that notion. Here's a few links to people looking at JSON specifications:

http://json-schema.org/

http://www.wrml.org/

Also I'm copying a private blog post I've made at work on REST to help guide you to some good starters for REST design as I was researching REST. Can't attach the powerpoint here, so you will have to ignore that comment in the blog post. Hope this helps your journey


__________________________________________________________________________________________________________________
Moving from CRUDyRest to HyperRest

So I started to do some more research via books like Rest in Practice, Internet blogs and such. I came across quite a debate of what was considered a RESTful architecture. I lot of it stems from the work of Leonard Richardson dubbed the Richardson Maturity Model. The blog by Martin Fowler linked in the previous sentence discusses the model as steps toward the glory or REST. But Roy T. Fielding who created the REST dissertation made it pretty clear in his blog that hypermedia was a prerequisiteto a RESTful architecture.


As I found more information and started exploring this concept, I realized that the 4 levels (level 0 to 3) of the RMM are not steps to build a RESTful architecure. These are learning steps, or evolutionary steps for an existing HTTP service implementation. It doesn't make sense to start at one step and build to the others unless your architecture is already at a step.


My first inclination in designing a RESTful system was to start at defining the resources and their relationships as most people do. This technique naturally steers you toward building an RMM level 2 CRUDy REST API. Sure, you can enhance your system into a level 3 from there, but for all those client apps that relied on the resource structures and relationships to build their client code - they are now coupled to that model and probably will need to change to take advantage of the hypermedia aspect coming down the road. Also any changes to the resource and resource relationships (which can inevitably happen when trying to get to level 3) will break the application code for those early-adopting clients.


The author of the Richardson Maturity Model, Leonard Richardson wrote an abstract on how he built a client (in Python) to take advantage of the Hypermedia As The Engine Of Application State (HATEOAS) concept over RESTful services. This was to show results on the advantages of this approach as it seems to be hard to grasp for many developers.


So I'm now attempting to identify a checklist (in no particular order) of analysis/design techniques for building RESTful architectures.


Although I have not prescribed a particular order to promote an iterative approach, all good designs and architectures start with enough requirements and analysis to design and build something useful.

I'm not going to go into a soapbox on Agile vs. Waterfall and Incremental vs Iterative here as it's not the point of this post. I will probably be expanding on this in the future. Just resign yourself to the fact that you are not going to know 100% of all the requirements to design your entire system before providing workable code.


Here's those initial analysis techniques to start off with:

Come up with a good (or cool) REST API name and concept that preferably aligns with the domain area you are tackling (for the REST in Practice book this is the "Restbucks" system)

Identify Actors/Personas/Roles for those that will be using the REST API. What are their needs and motivations?

Identify the Business Process / Sagas that will be going on in your system. This may be very basic and simple at first, but will evolve more.

Identify Use Cases / User Stories that will involve your system

Identify the common base media types that clients will want to use to communicate with the REST API (i.e. JSON, ATOM, XML, etc)

These analysis techniques can be used in any order to drive out refinement for each other. For example I can start with an initial user story and use that to identify some personas (or actors). This user story may lead identifying another user story that can then be filled out in the saga / business process.


After getting a good set of analysis data to where you have a good direction you can then start designing the API.


Here' the design techniques you can use on the analysis to build a good API.

From the use cases / user stories identify your entry points into the REST API. Then build a home resource (or URI) that returns a representation with transition links to these entry points. This is similar in concept to the home page of a website providing the links needed to get started based on a user's need.

For the entry points, design the representation that will bring back the appropriate transition links for the next possible steps in the use case / user story. The allowable transitions states will depend on where in the business process / saga the client app is (think of this as business process state). These transitions can also be constrained by the internal state of the resources.

Start building your resources and resource relationships. This is essentially designing the URI structures returned by the transition links. Remember that the structure of your resources and their relationships to other resources should be abstracted from the client via the hyperlinks returned in the representations as much as possible.

Ideally a client using a well designed REST API that follows these techniques should be able to to the following...


1. Perform a GET on the home URI

2. Extract the entry point links from the representation

3. Use one of the transition links to perform another REST call based on client decision logic

4. Extract the transition links from the next representation

5. Repeat steps 3 and 4 until business process is done


This flow is very similar to the one described in Leonard Richardson's abstract above for how someone browses the WEB...


1. Retrieve a hypermedia representation of the home page.

2. Decode the representation to determine the current resource state.

3. Based on the representation's semantic cues, decide which hypermedia link or form is likely to bring you closer to your goal.

4. Click the link or fill out the form. Your browser will make another HTTP request and the result will be another hypermedia representation.

5. Go back to step 2 and repeat until the resource state is to your liking.



Through this process the client should never have to generate or append to a URI in order to make the RESTful call. It only relies on the link name for out-of-band information.


For an animated presentation of this process, you can download and view the power point deck attached.


In a follow-up blog, I will discuss the challenges of reducing out-of-band information for the client app. For calls with no parameters or input representations this is pretty easy, but can be more involved when using query parameters, path parameters, and input representations.

The way I see the reasons is this...

1. There are only two reviews... and a 1 star and 2 star averages to 1.5
2. When viewing the one start reviewer, its the only thing he has ever reviewed on amazon.
3. The 2 start reviewer has reviewed more things and is actually fair and resonable rather than being over negative, like they were insulted by WRML.
4. Most people are skeptical over a newly introduced concept that is not proven or has a huge community around it, especially when it come to passionate subjects such as what makes a good REST design.
5. A lot of people using JSON are afraid that any level of specification used to mature the tooling will render its design inflexible, which is a knee-jerk reaction.

My honest opinion is that there are a number of developers and projects working to address the lack of specification in JSON, and this is one attempt to abstract formats and specifications to make RESTful designs more flexible to changes in both. I'm not sure what attempt will win out, but the gaps are just asking to be addressed to mature REST to another level.

A JSON specification is not an equivalent to SOAP any more than an IDL is equivalent to CORBA. As long as the REST constraints are held, then it shouldn't matter what format is used to transfer a representation, or even whether there is a specification for it. For example, JSON is still considered more human readable and easier to parse (depending on your message complexity) than XML and adding a specification to describe the structure of its contents does not change its format or use at all.

Beyond the WRML concept (which does not have to be used to apply most all of the practices in this book), there's a lot of good information and practices contained in a concise format (as the 1st amazon reviewer admits). Much more complete than I've found in any other REST book I've read and recommended (i.e. REST in practice, RESTful service cookbook). It makes a good reference when designing a REST system beyond a sample RESTbucks application.

Just wanted to add in (as I've read through the book). That most of the book's rules are good REST practices that would help the beginner and intermediate REST developers to gain some consistency. Hopefully most advanced REST programmers would already know most of these practices. There's no direct coupling or tie to WRML, though its mentioned as an option.

Hope that helps.

Not sure what is specifically wrong based on the code and request supplied, but it may be a configuration problem. I remember getting this error before when I did not have the docroot base URI configured right and the URI hierarchy was one level deeper than I expected when implementing a Restbucks sample application.

You might want to check where the base URI is configured in the project.

Hi Mark,
Welcome to JavaRanch! I've read through your book on Safari online and had a particular question involving caching and the use of query parameters...

In the book you say

The entirety of a resource's URI should be treated opaquely by basic network-based intermediaries such as HTTP caches. Caches must not vary their behavior based on the presence or absence of a query in a given URL

I understand the specific instance sited about not ignoring requests with query parameters (i.e. not caching the results), but does this mean just cache the filtered (or paged) results in the cache and then add to the cache when other query parameters are used making it a MRU cache?

This is just a clarification, because I don't think it makes sense to return back the entire collection or store to cache and filter it before returning to the client.

Thanks,

John M Brown
Application Architect

Thanks Peer,

This information was very useful although not very promising for my approach.

It sounds like if we want to take advantage of custom business exceptions on most providers and clients, then the standard Jax-WS way of doing things is to use checked exceptions. I still think this makes for tight coupling (as now the consumer's code cannot compile unless it specifically catches these exceptions when using exceptions).

It even looks like Microsofts WCF follows the same pattern.

Some of our existing architecture declares custom RuntimeExceptions which supported Design By Contract, but gave the client code the option of catching the custom exception OR allowing the exception to bubble up without having to specifically declare it in it's own method.

The other option is to customize business exceptions into the message payload without using faults, but that would require everyone (consumers and other SOA components), to understand the custom payload.

Sometimes I feel like I'm on these specs.

Hopefully someone will see the light on this issue and the standards will evolve to include custom RunTimeExceptions for Web Services.

As I'm trying to come up with an approach for using Exceptions and SOAPFaults for our web services here I'm trying to answer a basic question. I eventually will test it out, but someone here may already have the answer and that help expedite some choices..

I understand how declared excpetions on a method are converted to custom SOAP faults using a web services framework such as CXF, etc. The only examples though specify the exceptions as checked exceptions.

I would like to declare specific business exceptions on the methods, but I want them to be RunTimeExceptions so the consumer can decide whether they catch them or not on the consumer side.

So my question is: If you declare a custom exception that extends RunTimeException in the throws clause of a method and you expose that method in a web service, will it marshal into the SOAPFaults defined in a WSDL? And if so, will the consumer that receives the fault consider it a RunTimeException?

I'm working on getting a test to validate this. I just wanted other peoples experience. I wanted to declare business specific exceptions in the methods and have them be faults in the WSDL (because I consider these expected type of exceptions in the contract), but I don't want the contract to require the consumer to catch them like checked exceptions.

For the other generic exceptions unexpected by the code (NullPointerException, communication exceptions, etc) I would just wrap into a generic exception with a stack trace to aid in debugging.

Thanks,
John

Thanks for the reply Paul and Dirk,
I did get an email from Dirk today saying he pinged Solarmetric.

I'll let you know if I hear anything by the end of the week.

Thanks again,

Did anyone else on this list get their JDO license? I've not even heard any word from Solarmetric or Javaranch after multiple emails. It's been almost two months. Sorry to air my laundry here. But I've had no luck via email. I'm sure it's just a minor mishap.

Thanks,
John Brown

Woot! Woot!

Ali,
I hope that I was able to explain it well enough to you. I didn't think you were complaining or arguing, but sometimes I may come on strong when I post about the things we are doing that I'm passionate about. I also like to share and investigate other opinions and strategies on these things to so I can lean more. Thanks for giving me the opportunity to share

Originally posted by Ali Pope:
I see your point (just the way I have imagined). I still belive that OCL would translate in fact into constraints and not in queries (or finder methods as you named it).
However if you found this util for your project, maybe I am wrong .

./pope

You are absolutely correct in your statement that OCL translates into constraints. All OCL should evaluate to true or false and should be free of side effects. post conditions define the constraint of what should be true after the method is executed. For queries it usually equates to the fact that the result is equal to a subset (or collection) of objects that meet the filter criteria.

We use preconditions and especially post conditions a lot in our models. This information is then passed down in code generation from our MDA tool to give the developer the information needed to know what to implement in the methods. The closer that we can leverage the OCL language though, the less coding work is needed for the methods.

I know this is probably a much more rigorous model than most projects and companies are accustomed to, but along with selecting technologies like JDO and using other patterns, it fits very well with our MDA strategy.

I'll try an example on the fly, rather than look up one in our current model (I might be able to show more later).

For example: Say we have the following classes - Reservation, Guest, and Address

(If I could attach a class diagram I would, but I will have to describe it)

A Reservation has an association with Guest such that there are 0..n guests for a reservation.

A reservation also has an association of exactly one primary guest (the navigating role to the Guest class is primaryGuest.

A Guest has an association with Address such that there are 0..n addreses for a Guest.

One such OCL business rule is that a reservation must have one and only one primaryGuest (which is enforced with the primaryGuest association), and that the primaryGuest must have at least one address.

Here's that OCL as an invariant:

Reservation:: inv: self.primaryGuest.addresses->notEmpty;

This doesn't apply to a query, but lets now define a method to find all reservations whose primaryGuest has an address in Florida and has 4 guests or less. To do this I'm going to put a method on Reservation as such:

Reservation::findByStateAndPartySize(state : State, partySize : int) : Reservation[] post: result = Reservation.allInstances()->select(res : Reservation | res.primaryGuest.addresses->exists(address : Address | address.state = state) and res.guests->size <= partySize);

Then I can pass florida (State.FL as an enumeration) and 4 to the method.

The implementation of this method in JDOQL 2.0 code is this:

{ // Reservation.allInstances() maps to Query resQuery = pm.newQuery(Reservation.class); // define parameters (ie. state : State, partySize : int) map to resQuery.declareParameters("State state, int partySize"); // define types for related elements (ie. address : Address) map to resQuery.declareVariables("Address address"); // select(...| res.primaryGuest.addresses->exists(... | address.state = state) and res.guests->size <= partySize) maps to reQuery.setFilter("primaryGuest.addresses.contains(address)&& address.state = state && count(guests) <= partySize"); // result = maps to Collection reservations = (Collection) resQuery.execute(); }

There are a number of other functions in JDOQL that map well to OCL:

<collection>.includes() -> <collection>.contains() <collection>->size -> count(<collection> ) <collection>.sum() -> sum(<collection> ) <collection>->isEmpty -> isEmpty(<collection> ) <object>.isEmpty -> isEmpty(<object> ) <object>.oclIsTypeOf(<type> ) -> <object> instanceof <type>

I'm only touching the surface of investigating this, but I did find that most of the OCL we have written for queries can easily map to JDOQL with a little syntax and semantic change.

[ November 11, 2004: Message edited by: John M Brown ]

[ November 11, 2004: Message edited by: John M Brown ]

[ November 11, 2004: Message edited by: John M Brown ]
[ November 11, 2004: Message edited by: John M Brown ]

I've also been very pleasantly surprised that a large portion of JDOQL maps well with semantics in OCL (Object Constraint Language), which we have been using in our UML specifications. I'm now in the process of building transformation patterns that map from OCL post conditions to JDOQL semantic code.

I think it might be an interesting exercise seeing if it would be worth the effort to use OCL as a standard language, being as this is a standard included in UML.

Originally posted by Ali Pope:


Can you detail the idea behind this pls?

./pope

I'm also interested in what is meant by the single-string form of JDOQL.

How is it different and what's the advantage?

Thanks,