Win a copy of Spark in Action this week in the Open Source Projects forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Bear Bibeault
  • Ron McLeod
  • Jeanne Boyarsky
  • Paul Clapham
Sheriffs:
  • Tim Cooke
  • Liutauras Vilda
  • Junilu Lacar
Saloon Keepers:
  • Tim Moores
  • Stephan van Hulst
  • Tim Holloway
  • fred rosenberger
  • salvin francis
Bartenders:
  • Piet Souris
  • Frits Walraven
  • Carey Brown

Spring Microservices in Action: Microservices, Containers and Database Design

 
Ranch Hand
Posts: 75
Mac Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Welcome - I look forward to checking out your book.  I'm interested in reading how the overall architecture of Microservices is discussed in the book - in particular how database design should change, and how containers come into play with microservices in terms of isolation.  I'm also interested in the effects that Microservices have on a Continuous Delivery pipeline.  Looking forward to a book that takes things beyond the buzzwords, and puts Microservices to practice in practical environments!

Thanks
David Sachdev
 
Author
Posts: 93
5
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi David,

Thanks for posting.  The topic of database design is really one of the hardest topics to talk about for people who come from a relational database background because we are used to designing databases around complete applications.  (I know because I started my career as an Oracle DBA and developer 20 years ago :-( ).  Anyways, the key things to keep in mind is that with a microservice, is that a microservice completely owns it own data.  That means that all data access for the domain should be down through the service.  That can be difficult where for data optimization reasons it was often very easy to to perform really complex real-time queries or  across different data domains.  

What I have found is that you usually are going to have to do more calls to a service to get data then by joining.  If you need to optimize and reduce database lookups then you can put a cache in place to hold data near to your service and use events via message bus (like Kafka, or RabbitMQ) to signal when data is not valid anymore and must be refreshed.  This is true for OLTP-based applications.  For reporting, you can propagate copies of the data owned by individual services and renormalize the data into a more performant structure by having the microservice that is making changes publish that a change has occurred.  This is usually where you see things like the "data lake" pattern emerge.

Quick summary:

1.  A service should completely own its data and the data it owns should be relatively small (4 to 5 tables).  No one but the service should be able to update the data accept through the microservice interface.  I have seen people enforce this in multiple ways, including:  convention (catch it during code review), separate access to tables behind individual service passwords (only the service has access to the tables), to the extreme of each service having their own database.

2.  For performance do not be afraid to use caching to keep data that is heavily used from one service near to another service that is consuming it.  Its more expensive, but avoid having multiple services use the same cache.  A down cache could impact multiple services.

3.  Use a message bus to publish changes to data.  This way even if a microservice does not own the data in question it can respond to changes in the state of the data and communicate back to the originating state.

4.  Don't make your microservices too fine grained turned them into distributed DAOs whose sole job is to manage data.  If you start finding a large proliferation of services that is a code smell that you have decomposed your data mode.

5.  My advice with your model is start coarse-grained.  Dont try to make too many microservices upfront.  Its easier to refactor a "fat" microservice then it is to consolidate a bunch of small services.

I need to know a little bit more about what you are looking for around Continuous Delivery.  Chapter 10 in the book deals with the subject.  If you have some specific questions.  

Please let me know.

     Thanks,
        John
 
David Sachdev
Ranch Hand
Posts: 75
Mac Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

John Carnell wrote:Hi David,

What I have found is that you usually are going to have to do more calls to a service to get data then by joining.  If you need to optimize and reduce database lookups then you can put a cache in place to hold data near to your service and use events via message bus (like Kafka, or RabbitMQ) to signal when data is not valid anymore and must be refreshed.  This is true for OLTP-based applications.  For reporting, you can propagate copies of the data owned by individual services and renormalize the data into a more performant structure by having the microservice that is making changes publish that a change has occurred.  This is usually where you see things like the "data lake" pattern emerge.



I have heard a lot about "data lakes" recently in various podcasts including the AWS TechChat, but I haven't really tried to wrap my head around the pattern and concepts yet - now that you have brought it into the context of MicroServices, I will have to go back and pay a bit more attention.  Given the changes in the way data is managed in microservices, do you see any big changes in the way that ORM is done on the horizon?  Just today I heard reference to MyiBatis, and someone pushing to change our JPA layer over, and I realized it has been a while since I've pondered if JPA is still the best path forward on the DB front.  Any thoughts?

    Bookmark Topic Watch Topic
  • New Topic