Migrate from monolithic system to microservice

Tram Ho

Giaosucan’s blog – sharing knowledge in a way that is awesome

http://www.giaosucan.com/

The story of DK DK is the world’s fifth-largest electronic component distributor. They have built an ERP Distribution Management System to serve retail distribution since 2002 on the MFC platform, written in C / C ++ with over 2 million lines of code. It has been almost 20 years now.

The current system is experiencing the following problems

  • Many technologies developed more than 20 years ago such as MFC are outdated, especially when .NET, .NET core was born
  • The original design was a monolithic design that became cumbersome due to too many added functions, difficult to maintain, and the build and deploy source code took a long time. Deploying to expand the system is really complicated
  • Difficult to change framework, system written in C / C ++ and MFC, if converted to .NET core, C # almost have to rewrite the whole. They decided to change the design of architect transition from monolithic to microservice architecture. With the aim of taking advantage of the advantages of microservice architecture such as the ability to deploy easily, use many technical stacks, scaling …. However, it’s easy to say but difficult to do, although the microservice architecture has many advantages over monolithic architecture, the DK engineering team encountered the following problems.
  • Database of DKE has more than 500 tables, using Oracle DB relationship between tables is extremely complex. Each table has on average several hundred thousand records ? How to break out this database to multi-database in microservice architecture and migrate data to ??
  • Migration from mono to microservice does not take place immediately, but takes years. As such, it is not possible to take down the mono system to transfer it, it must be done in a roll-out mode. So how do these 2 systems run in parallel?
  • During the migration process, the old system can still be updated continuously, how to reflect this change to the new system in time. In addition, there are many other challenges that cannot be counted. Solution For nearly two years, DK engineers have been testing a variety of methods to perform migration. This article shares some solutions at an overview level. DK’s business is Distributed management system, fast understanding is retail distribution. The domain drive design method is applied to analyze business and break out the system into individual modules such as Order, Payment, and Sale. To understand what DDD is, readers can refer to the series Understanding Domains in the hegemony of giaosucan’s blog

From the results of the business analysis of DDD, engineers were able to estimate the module, the number of microservice to create. The microservice is separated into internal service and external service. The internal service written in NodeJS directly interacts with the database, sending and receiving asynchronous data to the external service (expose API) to the client via a message queue. Later the internal microservice was converted to .NET core The API part was written in .Netcore and used the Swagger library to generate API specs

According to microservice best practice, to ensure loose coupling, the database will be split into many small own databases by the microservice. However, the theory is slash, such as putting into practice with interrelated database, mutual dependency, and millions of record data, breaking out is not a simple matter. The team used the break out and migrate data methods to create a database own by selling services, payment into a database, own by payment service. Oracle Golden Gate was tested to migrate between the new database and the old database.

Although testing on individual modules results in promising results, breaking the whole database is still a complex problem due to the conversion of data consistency (ACID) to the BASE of the microservice. to microservice is still stopping at the logical code, the microservice still points to a single database. To know how DK techniques are handled, read the next article

Share the news now

Source : Viblo