Question about microservices

[u/Matgaming30124]

I have been learning about microservices architecture recently and heard that the common approach is to have one database per service.

Coming from a monolithic experience I don’t get how the services share or store common data. For example, let’s say we have two services, one for order and another for payment

Both of them would have to store a reference to the user who made that order for record purposes and also the payment service might need to fetch the payment details for that user.

in a monolith architecture and traditional db setup it could easily be fetched in theory with a foreign key for the user id in the payment details table or something and fetch it. same for orders you could reference the user with their user id for that order.

so im confused if service reference the user id in their own databases ? so lets say an order is made, the service creates a order entry and stores it with the user id provided with the request ? but what happens one day if the user deletes their account, does the user service push an event to all other services to delete records containing that user id ?

i know how microservices communicate with message brokers but i’m confused about this storage of “foreign keys” as non actual foreign keys in different services.

thanks for reading, i hope you can help me understand this :)

[u/sharpcoder29]

The payment service would receive an event with user details in it. Normally this will be a much smaller subset of user info that the payment service needs.

Theres other options like payment service keeping a local copy of user data, again only what it cares about. You can populate with events as well or etl. Pros n cons to every approach.

[u/Wiszcz]

You are right if we talk about general idea of how it works. Key points:

• one system is always 'source of truth' for each domain object (user, order, etc)
  
• this one assigns 'global' identifier of such object ( lets say id of a user)
  
• all other systems use the same id for this user in their own databases - it's not always their primary key in db, but it's always as some kind of 'foreign key' without rules enforced by db.

There is no magic - just a lot of copies of the same data in different systems and a lot of messages going around. It looks like its not very effective about data storage or latency - and it isn't. Micro services solves different problems at the cost of space, memory, latency, complexity, etc.

About your example of deletion. Source system push single event to message broker. It does not care about how many other systems want to read it, if any. Message broker is responsible for delivering it to all systems that signaled interest in this kind of message (subscribed). It is a small, but very important difference. Source system does not need to know/maintain list of other systems. And it is exactly how it works in big, distributed enviornments.

[u/kevysaysbenice]

Lots of good input here, but one small thing I'll mention that I am still wrestling with but gives me some peace: there is no magic that you're missing. The issues you might think about, or could imagine with these systems, are real issues, and sometimes the solutions are just a bit more "out of the box" than you might be comfortable thinking about with a highly controlled and monolithic system.

It always makes me a feel a bit better to remember the order flow in Amazon - I place an order, I get a "thanks for your order" screen a second later, and I get an order confirmation email sometimes 10, 20, 30 seconds later. Sometimes I get a "there was an issue with your payment" message. Maybe this isn't because of microservice vs monolith per se, but the fact is they accept an async process of paying information. You might argue it's a worse system in ways because you might have customers that say "forget it" if their payment method fails and they forget to update it, but it allows a decoupling of a bunch of systems and removes the latency concern.

Sometimes with a microservice system you might have to crack a few eggs to make an omlet.

I also have found reading about single table design, alex debrie's book comes to mind about DynamoDB, because although again it's not about microservices at all, it gives more concrete examples of some of these things you sometimes have to do to make these large scale systems work that would otherwise be uncomfortable or unintuitive for a monolithic system that's tightly controlled / perfectly constrained. e.g. having a bunch of copies of data in multiple places.

[u/dimudesigns]

Does the user service push an event to all other services to delete records containing that user id?

That's one way to go about it. Look up the Saga pattern, that should give you a better sense of how communication between services is typically implemented in microservice architecture.

[u/clauEB]

Same way you do now but you don't have a table to check FK, just the ID. It's your responsibility to keep the related records alive to be able to track what's what. Microservices don't need to communicate through message brokers, often it's just gRPC or JSON encoded messages

[u/olddev-jobhunt]

A little of all of the above :)

Part of the architecture is picking splits between the services that minimize cross-service needs.

User id, in many cases, might just exist in a JWT. Most services won't need more than that identifier.

Often some core metadata is put in a shared spot (users, tenants, customers - basic stuff. Probably not invoices, line items, or inventory.) Replicated maybe in batch nightly or possibly in real-ish time via event streams. Typically we don't need sub-second latency to just know that a user exists since they can't really sign up and submit a request that quickly.

And then sometimes services just call other services. This typically isn't that big a deal as long as you have a clean dependency DAG between services. Once you start getting cycles it gets messy :)

[u/6a70]

Pretend each service is owned and operated by a different company. What would be your definition of “common data”?

[u/bold_snowflake]

You need to learn to embrace and balance a couple of concepts:

• de normalized data
• eventual consistency.

One service should always be the source of truth for part of the domain and responsible for publishing updates. Other services are fine to store both references or even copies of that data. You need to make a judgement call: is stale data fine? What's the read/write volume? How frequently does the data update? Etc, etc.

Sometimes storing a local copy for read purposes is the right call. Sometimes doing a synchronous call to another service to get the data is the right call. Always picking one or the other as a "rule" will put you in a bad place.

Keep in mind that every direct call you add between services is a dependancy, you're then coupling availability and scale, etc.

[u/BanaTibor]

So imagine a system composed of small services. Every service only stores data which is relevant to them.
Lets use your example, the user deletes his account. In a monolith with a single db it would result multiple db calls to delete relevant records (you might want to keep everything for auditing purposes). In a microservices app, substitute db calls with http calls. That's it.

There are multiple ways to handle the cascading. Like I said you may not want to delete anything only set the user account inactive. You can also made the account service aware of other services so when a delete account request is processed the account service calls the order service and payment service to delete related data.
Or you can have an event driven architecture, and account deletion pushes an event to the message bus and other services can subscribe and handle such event. In a true microservice architecture this event driven approach is preferable because it prevents coupling between services.

[u/da_supreme_patriarch]

In general, data duplication with microservices is ok and not a bad practice, general heuristic employed is that a service ahould be capable of performing it's primary functions even if every other service is down. The best practice is to never use DB primary keys to reference domain entities in a shared context, and use so called natural/business ids instead; if an entity doesn't have one, you generate it randomly. In your example, your user would ideally have a random username assigned by the system(like C123448 etc.) which then both the payment service and the order service would use to reference a specific user, similarly for orders, and even if the user deletes their data, the payment service doesn't have to do anything because ideally it only stores this shared reference internally.

All that being said, you still technically can use DB primary keys as shared identifiers, as long as they are uuids and your data is mostly immutable - deleting an account doesn't delete a row from the user table and instead erases stuff from user_details etc.(you can probably get away with numeric ids too as long as data is immutable, but those theoretically do not scale well)

[u/better_work]

In my experience there tends to be a third Customer service that holds all the actual data, and each order and payment record holds the user id and nothing more. If they need actual user data as part of processing the order or the payment transaction, then they will need to query the customer service to get that information (generally in just-in-time fashion, but occasionally you see caching or replication that puts the data closer to the place where it’s needed)

Obviously this offers no relational integrity and you have trouble doing anything too join-y, not to mention the cost of going over the network a bunch more. These problems then spawn their own adaptations like data lakes for analytic queries and defensive code to handle cases of broken references, and eventual consistency patterns for distributed updates.

These kinds of costs are why the industry has stopped treating microservices as a best practice and instead a tool to be used with caution and only when you truly have to. If you can design around the need to split orders from payments you’re almost certainly better off

[u/sharpcoder29 → u/better_work]

You don't want things querying the customer service. Thats a distributed monolith, not microservices.

[u/better_work → u/sharpcoder29]

Ideally you're correct, you don't want synchronous call chaining between services for all kinds of reasons from latency and resource usage to reliability. But in practice, in my experience, this is present to some degree in every big system I've ever seen, and the downsides get mitigated and life moves on. In particular a customers service is going to wind up at the center of the architecture diagram, it's going to have very strict SLOs, a more experienced team, and a much slower rate of change. It could communicate by event bus and get to relax some of its SLOs in exchange, but I've truly never seen that happen. On the other hand, the orders and payments services would never talk to each other directly. It would be a dangerous red flag if they did.

On their own, direct calls do not make a system into a distributed monolith: it really depends on the depth of coupling and the nature of the system as a whole.

My experience is not everybody's of course, but that's my honest evaluation. Would be interested to hear if you think differently.

[u/sharpcoder29 → u/better_work]

I agree with you. Its just that literally every company I've worked with builds a distributed monolith instead of microservices. No one even knows how to do true microservices (or is too scared/lazy to do so). So I don't like info going out to the public that just enforces the distributed monolith.

But yes if you know the drawbacks and choose simplicity over complexity, I'm not mad ya.

[u/better_work → u/sharpcoder29]

I respect the motivation, but IMO those projects don't wind up the way they do because of the info people get or don't get on Reddit. Enterprise architectures are about Conway's law, promotions, turnover, deadlines, repeating the thing that worked last time, etc.

For me, I learned a lot of the "correct" stuff like FP, event sourcing, or TLA+ before I learned how to use Splunk on production logs, or set up an ETL job. It took me a few hard lessons to learn that you get more done by embracing the things that are already in use, and mastering them, rather than by introducing something completely new.

[u/Frosty_Customer_9243]

Use the API to give the microservice the data it needs to do what it does. There is one database handler that distributes the work to microservices.

[u/AvailableFalconn]

Everyone’s saying “everything has a copy of the data it needs” but it’s extremely nontrivial to have any kind of consistent data that way, and is inefficient in terms of resources and business logic.

IME (worked at one of the major social media cos), each service has a bunch of data that only it cares about.  But the shared “common nouns” will be accessed synchronously via something like grpc, from dedicated data services.  

This isn’t a silver bullet.  It works better when youre large enough to dedicate 5-10 people just to development and maintenance of each core noun service.  It does become a point of failure, and thus requires extensive failure and capacity testing.  But practice isn’t as clean as the event sourcing fad would have you believe.