Configuring Micro Services - Spring Cloud Config Server

Managing application configuration in a traditional monolith is pretty straight forward. Configuration is typically externalised to one or more property files that sit on the same server as the application. Externalising the configuration makes sense as it allows you to update configuration without having to rebuild and redeploy the application. It also means the build artifact is environment agnostic, allowing you to deploy the same physical WAR to dev, uat, prod etc.

Configuring Micro Services

While the approach described above is valid for many applications, the move toward micro services has created a number of new challenges. Micro services by definition are more granular, meaning many distinct components to deploy and manage. Increasing the number of components creates a maintenance challenge when it comes to managing application config. Consider managing the configuration for a solution that is broken into 8 distinct micro services (which is not many by the way). Updating some common configuration in a development environment could mean logging onto 8 different boxes and amending property files. In a production environment the overhead is even greater, as there will likely be multiple instances of each service. You can see how a simple config change could require a lot of effort.
Manually updating configuration across many servers also means there is a decent chance of human error. A situation where some services have updated configuration and some don't, can lead to all sorts of unpredictable behaviour. 
While micro services can run on premise, most micro service architectures run in the cloud.  Cloud environments tend to be quite fluid with server instances can be brought up and down regularly, as as the result of some auto scaling activity for example. This fluidity makes it very difficult to manually manage application config across multiple services.

Introducing Centralised Configuration

Centralised configuration helps address the challenges mentioned above. Rather than services having their own local configuration, each service can instead retrieve its configuration from a central source. Spring provides a basis for implementing this pattern via Spring Cloud Config, a sub project of Spring Cloud.
Spring Cloud Config allows you to create a Spring Boot application that exposes application properties via a REST API.  Other services can consume their application properties from the REST API rather than loading them from a local source. Property files containing the application configuration are not actually stored in the Cloud Config Server, but are instead pulled from Git.  This allows you to manage your application properties in Git with all the benefits of a source control system. Spring Cloud Config can be configured to use either a local git repository (useful during dev) or a remote repository. In a production environment you'd want the Config Server to access the properties from a remote repository on Github.

Sample Application

In this post I'm going to show you how to setup a Configuration Service and configure it to pull application properties from GitHub. I'll also show you how those properties can be consumed by another service, in this case a simple bank account service.  The diagram below describes the main components involved.

Centralised Configuration Architecture

Creating the Cloud Config Service 

We'll begin by creating a simple Spring Boot app for the Config Service. Inside the main application class add @EnableConfigServer to enable the config server functionality.

@EnableConfigServer
@SpringBootApplication
public class ConfigServerApplication {

    public static void main(String[] args) {
        SpringApplication.run(ConfigServerApplication.class, args);
    }
}

Configuring the POM

In order to make the required Spring Cloud dependencies available you'll need to add the following dependencies to the POM.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" 
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
                       http://maven.apache.org/xsd/maven-4.0.0.xsd">
 <modelVersion>4.0.0</modelVersion>
 <groupId>com.briansjavablog.microservices</groupId>
 <artifactId>config-server</artifactId>
 <version>0.0.1-SNAPSHOT</version>
 <packaging>jar</packaging>
 <name>config-server</name>
 <description>Demo config server provides centralised configuration for various micro services</description>
 <parent>
     <groupId>org.springframework.boot</groupId>
     <artifactId>spring-boot-starter-parent</artifactId>
     <version>2.0.3.RELEASE</version> 
 </parent>
 <properties>
     <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
     <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
     <java.version>1.8</java.version>
     <spring-cloud.version>Finchley.RELEASE</spring-cloud.version>
 </properties>
 <dependencies>
     <dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-config-server</artifactId>
     </dependency>
     <dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-devtools</artifactId>
  <scope>runtime</scope>
     </dependency> 
 </dependencies>
 <dependencyManagement>
     <dependencies>
  <dependency>
      <groupId>org.springframework.cloud</groupId>
      <artifactId>spring-cloud-dependencies</artifactId>
      <version>${spring-cloud.version}</version>
      <type>pom</type>
      <scope>import</scope>
  </dependency>
     </dependencies>
 </dependencyManagement>
 <build>
     <plugins>
  <plugin>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-maven-plugin</artifactId>
  </plugin>
     </plugins>
 </build>
</project>

Configuring the Config Server

Next you'll need to configure the Config Service via application.properties.

1
2
3
4
5
6
7
8
9
spring.application.name=config-server
server.port=8888

# URI of GIT repo containing properties
spring.cloud.config.server.git.uri=https://github.com/briansjavablog/micro-services-spring-cloud-config
# path to properties from root of repo 
spring.cloud.config.server.git.searchPaths: configuration

logging.level.org.springframework.web=INFO
  • Line 1 - spring.application.name specifies the name of the application. This isn't essential but its good practice to name your Boot applications as this name will appear on the actuator /info endpoint and will be displayed in Spring Boot Admin if you use it.   
  • Line 2 - server.port specifies the port that the admin app will run on, in this instance port 8888.
  • Line 5 - spring.cloud.config.server.git.uri specifies the URL of the remote repository containing the property files that will be served up by the Config Service. In this example I am pointing at my own repository on GitHub, but this can also point to a local Git repo. This is useful for local development and can be specified with the file prefix as follows file:///c:/dev/micor-services/git-local-config-repo.    
  • Line 7 - spring.cloud.config.server.git.searchPaths specifies the path to the properties files from the root of the repository. So in the example above I want to access property files in the configuration directory of the repository. 

Running the Cloud Config Service 

Now that we have the Config Server configured, its time to fire it up and do a quick test. You can start the application in Eclipse or on the command line the same way you'd start any other Boot app.  You should see the application start on port 8888 as follows.
Cloud Config Service Startup Port 8888

Testing the Config Service

You can test the service by calling http://localhost:8888/bank-account-service/default.  This GET request contains the name of the properties and the profile of the properties we want to load. In this instance we are looking for the properties belonging to the bank-account-service and we want the properties associated with the default profile.

When this request is received, the Config Service uses the GIT URI specified in application.properties to perform a git clone of the remote repository. The screenshot below shows the repository being cloned to the a local temp directory and includes the following line.

Adding property source: file:/C:/Users/BRIANS~1/AppData/Local/Temp/config-repo-6545303057095204707/configuration/bank-account-service.properties

Cloud Config Server - Cloning Repository
The properties in bank-account-service.properties are then read by the Config Service and returned to the client in JSON format.

Config Service Response

The following JSON response is returned from the Config Service when we call http://localhost:8888/bank-account-service/default.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
{
   "name": "bank-account-service",
   "profiles": ["default"],
   "label": null,
   "version": "7b0732778b442726f8dd0bf7d1a36fc00f15c5b8",
   "state": null,
   "propertySources": [{
      "name": "https://github.com/briansjavablog/micro-services-spring-cloud-config/configuration/bank-account-service.properties",
      "source": {
          "bank-account-service.minBalance": "99",
          "bank-account-service.maxBalance": "200"
      }
   }]
}

The name attribute contains the name of the properties we requested. In this instance we requested properties for the bank-account-service. Note that this matches the name of the default properties file in GitHub.

The profiles attribute is the Spring profile used to load the required properties. In this instance we used the default profile, but we can specify any valid profile we want here. If you look at the property files on GitHub you'll see 3 files.
  • bank-account-service.properties - this file contains the default properties and is used as the property source when the default profile is specified on the request.
  • bank-account-service-dev.properties - this file has a '-dev' post fix and contains the properties returned when the dev profile is specified on the request.
  • bank-account-service-uat.properties - this file has a '-uat' post fix and contains the properties returned when the uat profile is specified on the request.
If we specify dev or uat as the profile, the Config Service will return properties from the file matching that profile.

The version attribute is the current commit sha of the properties being returned. If you check GitHub this will match the commit sha of the latest commit. Finally, the propertySources attribute contains the GitHub source URI of the properties being returned along with the actual property values.

Creating a Bank Account Service

Now that the Config Service up and running, its time to put it to work. We'll create a simple Bank Account Service that will call the Config Service on start up to retrieve its properties. The Bank Account Service has two REST endpoints, one to create a bank account and one to retrieve a bank account.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
@RestController
@Slf4j
public class BankAccountController {

    @Autowired
    public BankAccountService bankAccountService; 
 
 
    @PostMapping("/bank-account")
    public ResponseEntity<?> createBankAccount(@RequestBody BankAccount bankAccount, HttpServletRequest request) throws URISyntaxException {
  
        bankAccountService.createBankAccount(bankAccount);
  
        log.info("created bank account {}", bankAccount);
  
        URI uri = new URI(request.getRequestURL() + "bank-account/" + bankAccount.getAccountId());
  
        return ResponseEntity.created(uri).build();    
    }
 
 
    @GetMapping("/bank-account/{accountId}")
    public ResponseEntity<BankAccount> getBankAccount(@PathVariable("accountId") String accountId){
  
        BankAccount account = bankAccountService.retrieveBankAccount(accountId);
  
        log.info("retrieved bank account {}", account);
  
        return ResponseEntity.ok(account);    
    }
 
}

The REST controller uses a simple BankAccountService to create and retrieve bank account details. When creating a new account, the service performs a check to see if the balance of the new account is between a set of minimum and maximum values.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
    /**
     * Add account to cache
     * 
     * @param account
     */
    public void createBankAccount(BankAccount account) {
  
        /* check balance is within allowed limits */
        if(account.getAccountBlance().doubleValue() >= config.getMinBalance() && 
           account.getAccountBlance().doubleValue() <= config.getMaxBalance()) {
   
            log.info("Account balance [{}] is is greater than lower bound [{}] and less than upper bound [{}]", 
                     account.getAccountBlance(), config.getMinBalance(), config.getMaxBalance());
   
            accountCache.put(account.getAccountId(), account);
        }
        else {
   
            log.info("Account balance [{}] is outside of lower bound [{}] and upper bound [{}]", 
                     account.getAccountBlance(), config.getMinBalance(), config.getMaxBalance());
             throw new InvalidAccountBalanceException("Bank Account Balance is outside of allowed thresholds");
        }
    }

Bank Account Service Config

These minimum and maximum values are configurable and will be read from an injected Configuration object.

1
2
3
4
5
6
@Service
@Slf4j
public class BankAccountService {                                                     

    @Autowired
    private Configuration config;

The Configuration object is defined as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
@Component
@ConfigurationProperties(prefix="bank-account-service")
public class Configuration {                                                          
 
    @Setter
    @Getter
    private Double minBalance;
 
    @Setter
    @Getter
    private Double maxBalance;

}

There are two important things to note.
  • the @ConfigurationProperties prefix matches the name of the properties file in GitHub
  • the key for each property in the property file is <fileName>.<propertyName> where the property name is the same as the instance variable name in the @Configuration class.  
A screenshot from GitHub shows the file name and property names corresponding to the values in the @Configuration class.

bank-account-service.properties in GitHub

Bank Account Service Local Configuration

We are going to use the Config Service to retrieve the Bank Account Service Configuration, however there are a few properties which must be set locally.

1
2
3
4
5
6
 spring.application.name=bank-account-service
 server.port=8080

 spring.config.cloud.uri=htp://localhost:8888
 spring.cloud.config.profile=uat
 management.endpoints.web.exposure.include=*

Lines 1 and 2 are standard Boot config and define the application name and the port it will run on. Line 4 defines the URL of the Config Service. This is the URL that the Bank Account Service will call on start up to retrieve the minBalance and maxBalance properties.  Line 5 defines the profile that will be used to call the Config Service. So given the configuration above, the Config Service will be called for the bank-account-service with the uat profile on start up as follows.

http://localhost:8888/bank-account-service/uat.

Testing the Bank Account Service

Start the Bank Account Service on the command line or in Eclipse. In the log you should see a call to the Config Service on http://localhost:8888 using the uat profile as configured in application.properties.   

Calling Config Service on startup

We can now test the application and confirm that it was indeed the uat configuration that was retrieved from the Config Service. Run the following cURL command to create a new bank account.

1
curl -i -H "Content-Type: application/json" -X POST -d '{"accountId":"B12345","accountName":"Joe Bloggs","accountType":"CURRENT_ACCOUNT","accountBlance":1250.38}' localhost:8080/bank-account

Note that we specify an account balance of £1250.38. This should be inside the allowed limits given that the uat properties are defined as follows.

bank account service uat profile properties

The log extract below shows the account object being created successfully and the min and max values being logged as 501.0 and 15002.0 respectively. These values match with those defined in the uat properties in GitHub above.

Updating Service Configuration

One of the main benefits of centralised configuration is that properties can be updated in one place (GIT) and then those changes can be picked up by the Config Service right away.  Every time an application requests properties from the Config Service, the Config Service will check if its locally cloned copy of the remote repository is up to date. If the local copy isn't up to date the Config Service will pull the latest properties from the remote repository and serve them back to the client. This allows applications that are starting up to retrieve the latest properties from the remote repository, but what about updating properties for applications that are already running? 

Thankfully Spring Boot provides a mechanism for reloading application properties in a running application. The reload can be triggered with a HTTP POST to the /refresh actuator endpoint as follows.

curl localhost:8080/actuator/refresh -d {} -H "Content-Type: application/json" 

The screenshot below shows the /refresh endpoint being called with cURL. In the application log you can see the call to the Config Service to retrieve the latest properties for the uat profile. 

property refresh via  actuator endpoint

Automating Service Configuration Updates

Manually invoking the /refresh endpoint is fine in this simple example, but in a production environment we'd have multiple instances of each service. After pushing config changes to GitHub it would be tedious if we had to grab the IP of each service and call its /refresh endpoint manually. 
We could easily create a script to poll the properties repository for changes and then use some kind of service discovery mechanism to get all registered service instances and call their /refresh endpoint. 
Some simple enough scripting used in conjunction with a centralised Config Service should allow you to push configuration changes out across your micro services with minimal effort.  

Wrapping Up

In this post we looked at Spring Cloud Config and how it can be used to create a centralised configuration service that leverages GitHub as the property repository.  We also saw how properties can be specified for different environments using profiles and how these properties can be retrieved by a simple micro service. Finally we looked at how property changes can be applied to running services. The full source code for this post is available on GitHub so feel free to pull it down and experiment. If you have any questions or suggestions please leave a comment below. 

Comments

Popular posts from this blog

Spring Web Services Tutorial

Axis2 Web Service Client Tutorial

REST Endpoint Testing With MockMvc

Spring Boot & Amazon Web Services (EC2, RDS & S3)

Health Checks, Metrics & More with Spring Boot Actuator

Spring Batch Tutorial

An Introduction to Wiremock

Externalising Spring Configuration

Docker & Spring Boot