Kafka, Baggage & Webflux Migration to Spring Boot 3.X for Microservices

Real-Time Implementation Example of Baggage Propagation

Complete migration guide for Kafka, Baggage and Spring reactive(Webflux) upgrade to Springboot 3.X for microservices application with real-time implementation

This example demonstrates how to implement Sleuth baggage in a Spring Boot 3.x application. Baggage fields are used to propagate contextual information (like user ID or session ID) across microservices to ensure consistent traceability and correlation.

Step-by-Step Explanation

1. Set Up the Project

• Objective: Create a new Spring Boot project with necessary dependencies.
• Actions: Use Spring Initializr to generate a new project with dependencies:

• Spring Web: For building web applications.
• Spring Cloud Sleuth: For distributed tracing and baggage management.
• Spring Boot Actuator: To monitor and manage the application.

2. Update pom.xml

• Objective: Ensure that your project uses compatible versions of Spring Cloud and Sleuth.
• Actions: Modify your pom.xml to include Spring Cloud dependencies and import the correct BOM (Bill of Materials). This helps manage dependency versions.

3. Configure Application Properties

• Objective: Define which baggage fields should be propagated across services.
• Actions: Set up configuration in application.yaml to specify remote and correlation fields.

• remote-fields: Specifies fields that should be included in the trace context and sent across service boundaries.
• correlation-fields: Specifies fields that should be used to correlate traces.

4. Implement Baggage Fields

• Objective: Define and manage baggage fields programmatically.
• Actions: Create a configuration class to define the baggage fields.

This configuration class defines user-id and session-id as baggage fields, which will be used throughout the application.

Customizing Baggage Fields

• Objective: Ensure that baggage fields are correctly set and propagated.
• Actions: Implement a custom interceptor to add baggage fields to the trace context.

This interceptor reads the user-id and session-id headers from incoming requests and updates the baggage fields.

Configure Web MVC

• Objective: Register the custom interceptor with Spring’s MVC configuration.
• Actions: Add the custom interceptor to the Spring MVC configuration.

This configuration ensures that your custom interceptor is applied to incoming HTTP requests, allowing it to process and propagate baggage fields.

Update Logging Configuration

Objective: Include baggage fields in log output to assist in trace analysis.
Actions: Modify the logging pattern in application.yaml to include baggage fields.

logging:
  pattern:
    console: "%d{date(yyyy)-month(MM)-dd(time)HH:mm:ss} [%t] %-5level %logger{36} - %msg [sales-id/user_id=%X{sales_id/user-id}, session-id=%X{session-id}]%n"

This pattern ensures that each log entry includes the user-id and session-id baggage fields, which are useful for tracing and debugging.

Test the Implementation

• Objective: Validate that baggage fields are propagated and logged correctly.
• Actions: Deploy the application and make requests with the appropriate headers (user-id and session-id). Check logs and traces to ensure the baggage fields are included and propagated as expected.
• Testing Example: Use tools like Postman to send requests with user-id and session-id headers, then inspect the logs and trace data to verify correct implementation.

This example covers the key steps to implement Sleuth baggage in a Spring Boot 3.x application. It involves updating dependencies, configuring application properties, defining and managing baggage fields, and ensuring proper logging. Just follow these steps where you can enhance distributed tracing including logging so that it would make it easier to track and troubleshoot issues in any microservices based applications.

What are API and Behavior Changes for spring boot 3.x upgrades?

Key API and Behavior Changes in Spring Boot 3.x

JDK Version Requirement

• Change: Spring Boot 3.x requires JDK 21 or later. This is a shift from the earlier versions, which supported JDK 8 and above.
• Example: If your application previously ran on JDK 11, you’ll need to upgrade your Java Development Services Kit to JDK 21. Update your pom.xml for Maven or build.gradle for Gradle to specify the new JDK version:

Maven:

<properties>
    <maven.compiler.source>21</maven.compiler.source>
    <maven.compiler.target>21</maven.compiler.target>
</properties>

Gradle:

Dependency Upgrades

• Change: Spring Boot 3.x has the updates for its core dependencies that includes Spring Framework 6.x then Hibernate 6.x n and others.
• Example: Dependencies i.e. spring-boot-starter-data-jpa in springboot 3.x uses Hibernate 6.x which introduces changes in the ORM API.
• Action: Adjust your JPA configuration to accommodate new Hibernate API methods and features. For instance, Hibernate 6.x has changed some API methods like Session and EntityManager for better performance and feature support.

Configuration Property Changes

• Change: Some configuration properties have been deprecated or renamed. For instance, spring.datasource.url has been updated to include new options.
• Example:

If a property has been deprecated, check the updated Spring Boot documentation for the new property names and update your application.properties or application.yml files accordingly.

Removal of Deprecated APIs

• Change: Spring Boot 3.x has removed APIs deprecated in previous versions.
• Example: If your application used spring-boot-starter-security with deprecated security configurations, you will need to update to the new security configuration classes.
• Action: Replace deprecated classes with their updated counterparts. For instance, instead of using WebSecurityConfigurerAdapter, use SecurityConfigurerAdapter in the new security setup.

Updated Actuator Endpoints

• Change: Spring Boot 3.x introduces changes in Actuator endpoints, including new endpoints and modifications to existing ones.
• Example: The /actuator/health endpoint now includes additional details about application health checks.
• Action: Review and update your monitoring configurations. If you rely on specific Actuator endpoints, ensure they are compatible with the new version.

Enhanced Native Support

• Change: Spring Boot 3.x offers improved support for native compilation with GraalVM.
• Example: To compile your application natively, include the spring-boot-starter-aot dependency and configure your build tools for native image support.

Action:

Configure your build tools for GraalVM:

Real-Time Example Implementation

Here, explained through a practical example of upgrading a simple Spring Boot application from 2.x to 3.x. Java Developers India have focused on updating the Java version, dependencies, and configuration changes.

Original Spring Boot 2.7.x Setup:

• JDK Version: 11
• Dependencies: Spring Framework 5.x, Hibernate 5.x
• Configuration: application.properties with deprecated settings

Steps to Upgrade:

1. Update JDK Version: Ensure that your development environment is updated to JDK 21.
2. Update Dependencies:
   o Modify pom.xml or build.gradle to include Spring Boot 3.x and compatible dependencies

Maven:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-jpa</artifactId>
    <version>3.x.x</version>
</dependency>

Gradle:

dependencies {
    implementation 'org.springframework.boot:spring-boot-starter-data-jpa:3.x.x'
}

Update Configuration Properties:

Review and update deprecated properties in application.properties.

Original:

Updated:

Refactor Deprecated API Usage:

Replace deprecated classes and methods with their updated counterparts.

Original:

@Configuration
public class SecurityConfig extends WebSecurityConfigurerAdapter {
    // configuration code
}

Updated:

@Configuration
public class SecurityConfig implements SecurityConfigurer {
    // updated configuration code
}

Test Actuator Endpoints:

Ensure that new Actuator endpoints are properly configured and tested.

Example:

bash
curl http://localhost:8080/actuator/health

Enable Native Compilation (optional):

• Include GraalVM support if needed.

Example:
bash
./mvnw package -Pnative

Upgrading to Spring Boot 3.x involves several changes, including API updates, new configuration requirements, and deprecated feature removals. By following the outlined steps and examples, you can effectively manage the transition and take advantage of the new features and improvements offered by Spring Boot 3.x. Ensuring compatibility with the latest JDK, updating dependencies, and refactoring deprecated APIs are key to a smooth upgrade process.

How to Handle Junit5 with Spring Boot 3.x Upgrade?

Upgrade to Spring Boot 3.x for your springboot microservice applications and it would rewuired several changes and enhancements which may impact various aspects of your application not only for dev framework also including testing frameworks. You have to integrate to JUnit 5 which offers more powerful testing framework and flexible testing model as compared to Junit4. In this blog, I have explained how to handle JUnit 5 with a Spring Boot 3.x upgrade and mainly focused on a real-time example implementation in a microservices architecture.

1. Introduction

Spring Boot 3.x mainly bring improvements to better support for Java 21 and also it provides native image support, and new features that align with modern development practices. JUnit 5 or  (Jupiter) i.e the latest version of the JUnit testing framework and mainly these frameworks focused on to be more extensible and flexible. I have also explained about the test models and configurations changes to leverage the new capabilities of both Spring Boot 3.x and JUnit 5.

2. Understanding JUnit 5 and Spring Boot 3.x Integration

JUnit 5 comprises three sub-projects:

• JUnit Jupiter: The new programming model and extension model for writing tests.
• JUnit Vintage: Provides backward compatibility for JUnit 3 and JUnit 4.
• JUnit Platform: The foundation for launching testing frameworks on the JVM.

Spring Boot 3.x supports JUnit 5 out of the box, replacing JUnit 4 as the default testing framework. This integration enhances test writing with features like parameterized tests, conditional execution, and improved assertions.

3. Upgrading Your Project

3.1. Update Dependencies

First, ensure that your pom.xml (for Maven) or build.gradle (for Gradle) files are updated to include Spring Boot 3.x and JUnit 5 dependencies

For Maven:

For Gradle:

3.2. Update Test Classes

JUnit 5 introduces new annotations and changes to existing ones. Replace JUnit 4 annotations with their JUnit 5 counterparts:

• @Test remains @Test.
• @Before becomes @BeforeEach.
• @After becomes @AfterEach.
• @BeforeClass becomes @BeforeAll.
• @AfterClass becomes @AfterAll.

Example migration:

Real-Time Example Implementation for Microservices

Consider a microservice that provides user management functionality. You want to test a service that handles user registration.

Service Class

Class

JUnit 5 Test

Upgrading to Spring Boot 3.x and integrating JUnit 5 requires some adjustments but offers numerous benefits, including more powerful testing capabilities and better support for modern Java features. By following the steps outlined above and applying the real-time example, you can ensure a smooth transition and take full advantage of the enhanced testing features provided by JUnit 5 in a Spring Boot 3.x environment.

Migration for Reactive Micro Services:

How to do Reactive Dependencies Update for spring boot 3.x upgrade?

Introduction:

Spring Boot 3.x comes with new features and improvements that are aimed at enhancing performance security and productivity of developers. One such area of concern in these updates pertains to reactive dependencies. It is necessary to note that in earlier times, reactive programming is gaining more prevalence in the development of applications since it solves the problem of managing asynchronous data easily. Also, in order to smoothly migrate to Spring Boot 3.x one may have to refactor the reactive dependencies to avoid loose ends and make good use of enhanced aspects.

What is Reactive Dependencies Update?

Reactive Dependencies Update is the action that entails updating and setting the reactive libraries and components within your Spring Boot application. Due to non-blocking I/O operations in reactive programming applications are capable of serving multitudes of concurrent requests efficiently mainaly in event based microservices or where large incoming requestes are there. While moving to Spring Boot 3.x, it is critical to verify that all such reactive components or libraries are available in that version to avoid loss of the Application’s performance and efficiency

Key Changes in Spring Boot 3.x for Reactive Programming

1. Spring WebFlux Changes/upgrade: Spring Boot 3.x has significant changes/improvemnets to Spring WebFlux i.e. the reactive web framework in the Spring framework. These modifications/upgrades include better support for reactive data sources that improves integration with reactive libraries.
2. Updated Dependencies: There are many reactive libraries such as Project Reactor and RxJava that has new versions compatible with Spring Boot 3.x. If we update these dependencies then it would ensure compatibility and also same time we will get the advantage of performance improvements and new features.
3. API Changes: One more thing is some APIs have been either updated or deprecated in Spring Boot 3.x. It is important to know about that and then review these changes to update your codebase accordingly so that you can avoid any sort of potential issues.

How to Perform Reactive Dependencies Update for Spring Boot 3.x

Updating reactive dependencies involves several steps, from updating your pom.xml or build.gradle file to testing your application thoroughly. Here is a detailed guide on how to perform this update:

Step 1: Update Dependency Versions

First, you need to update the versions of your reactive dependencies in your build configuration file.

For Maven (pom.xml):

For Gradle (build.gradle):

Replace 3.x.x with the latest compatible versions for Spring Boot 3.x.

Step 2: Update Code for Deprecated APIs

Review the Spring Boot 3.x migration guide for any deprecated APIs or significant changes. Update your codebase to use the new APIs or recommended alternatives.

Example: If a method has been deprecated, look for the new method or class in the documentation and refactor your code accordingly.

Step 3: Test Your Application

After updating the dependencies and code, thoroughly test your application. Ensure that all reactive components are functioning correctly and that there are no runtime issues. Pay special attention to:

• Reactive Data Sources: Ensure that all data sources are reactive and configured correctly.
• Error Handling: You need to verify the error handling if it working in proper way for 4xx and 5xx error scenarios in reactive streams and verify if the application is functioning as expected.
• Performance: You need to test the performance thoroughly of your application using any perf test tool such as K6 or Jmeter. Please make sure that the updates have not introduced any defects or regressions.

Example: Updating a Reactive Controller

Suppose you have a reactive REST controller using Mono from Project Reactor. Here’s how you might update it:

Before Update (Spring Boot 2.x):

@RestController
public class ExampleAegisController {

    @GetMapping("/aesis/example")
    public Mono<String> example() {
        return Mono.just("Hello, aegis test !");
    }
}

After Update (Spring Boot 3.x):

Ensure that you are using the updated version of reactor-core and that the Mono API has not changed. In this case, there are no major changes, but always verify with the documentation.

Updating reactive dependencies when upgrading to Spring Boot 3.x is a crucial step to ensure compatibility and leverage new features. By following the steps outlined—updating dependency versions, refactoring deprecated APIs, and thoroughly testing your application—you can smoothly transition to Spring Boot 3.x while maintaining the performance and reliability of your reactive applications.

Compatibility with Project Reactor Changes from spring boot 2 to 3.x upgrade with real time example implementation:

To upgrade from Spring Boot 2.x to 3.x is not completed with directly upgrading the dependencies in maven or gradle. But it involves other crucial changes that may can impact your application’s compatibility especially mainly if you use reactive programming with Project Reactor in application. In this blog, I have explained in details how to consider the key compatibility changes and I have provided a detailed real-time example of how to implement these changes effectively.

Understanding the Project Reactor Changes

Project Reactor is basically core component in Spring WebFlux and it basically provides support for reactive programming mainly with Flux and Mono types. The upgrade from Spring Boot 2.x to 3.x introduces changes in Project Reactor and it may affect also how reactive streams are handled.

1. Updated Reactor Versions: Spring Boot 3.x upgrades the Reactor dependency to Reactor 2020.0 or later. This update has the changes & improvements with new features for example, better performance and enhanced support for reactive programming patterns.
2. Java 21 Requirements: Spring Boot 3.x actually requires Java 21 or later. Project Reactor’s recent versions take advantage of new language features and performance enhancements in Java 21.
3. Deprecations and Removals: Some APIs and features in Project Reactor 3.x may be deprecated or removed. It’s important to review the Reactor documentation to understand these changes and their impact on your code.

Real-Time Example: Implementing Compatibility Changes

To illustrate the upgrade process, let’s consider a real-time example: a reactive application that processes user data streams in real-time. We’ll upgrade the application from Spring Boot 2.7.x to 3.x, addressing compatibility issues with Project Reactor.

1. Set Up the Project

Spring Boot 2.x Configuration:

<!-- pom.xml for Spring Boot 2.x -->

Spring Boot 3.x Configuration:

<!– pom.xml for Spring Boot 3.x –>

2. Refactor Reactive Streams

Spring Boot 2.x Code:

Spring Boot 3.x Code:

Key Changes:

• Mono.defer vs Mono.fromCallable: In Reactor 3.x, using Mono.defer ensures the supplier is invoked each time the Mono is subscribed to, improving concurrency control and reducing potential issues with the underlying thread.

3. Adjust Error Handling

Spring Boot 2.x Code:

Spring Boot 3.x Code:

Key Changes:

• Error Handling: In Reactor 3.x, Mono.error is used directly within Mono.defer to handle exceptions properly and maintain a reactive flow

4. Test Reactive Streams

Spring Boot 2.x Test:

Spring Boot 3.x Test:

Key Changes:

• Testing Frameworks: Testing code with StepVerifier remains largely the same, but ensure you are using the latest versions of testing libraries compatible with Spring Boot 3.x.

Upgrading from Spring Boot 2.x to 3.x involves several changes to ensure compatibility with Project Reactor. By addressing updated Reactor versions, Java 21 requirements, and deprecations, you can ensure your application continues to leverage the benefits of reactive programming effectively. The real-time example provided demonstrates how to refactor and adapt your code to maintain functionality and performance in the upgraded environment.

“Configuration and API Changes” related to Kafka producer and consumer in a Spring Boot 3.x upgrade:

As organizations transition to modern architectures, migrating to reactive microservices has emerged as a strategic approach to achieve scalability, responsiveness, and resilience. Reactive microservices, built on reactive principles and frameworks, leverage asynchronous communication and event-driven designs to handle high concurrency and dynamic workloads efficiently. However, this migration entails significant changes in configuration and API design, demanding careful planning and execution.

This blog explores the essential configuration and API changes required during the migration to reactive microservices, emphasizing best practices and considerations to ensure a smooth and effective transition.

1. Understanding Reactive Microservices

Reactive microservices are designed to handle large volumes of concurrent requests and responses in a non-blocking, asynchronous manner. They are built on reactive principles, including:

• Asynchronous Processing: This means all tasks/operations will get executed without blocking the main thread and it will allow the system to handle multiple tasks simultaneously.
• Event-Driven Architecture: Diff microservices communicate with each other through events and messages and it would enhance scalability and responsiveness.
• Backpressure Handling: Systems usually can adapt to varying loads by managing the flow of data to prevent overloads.

Reactive microservices typically utilize frameworks like Spring WebFlux , and one more is Vert.x, or Akka rector -> these frameworks provide support for building non-blocking n reactive applications.

2. Configuration Changes

Migrating to reactive microservices involves significant changes to configuration settings, including:

A. Non-Blocking I/O Configuration

Reactive microservices mainly depend on non-blocking I/O operations so that it can achieve high throughput and responsiveness. Configuration adjustments may include:

• Thread Pool Management: Configure thread pools to optimize the number of threads handling non-blocking operations. Frameworks like Spring WebFlux offer configurable thread pools for managing reactive tasks.
• Timeout Settings: Set appropriate timeouts for non-blocking operations to prevent resource exhaustion and improve system stability.

B. Reactive Data Source Configuration

Reactive microservices often interact with databases and external services using reactive drivers and APIs. Configuration changes include:

• Reactive Database Drivers: Replace traditional blocking database drivers with reactive drivers (e.g., R2DBC for SQL databases or reactive MongoDB driver). Ensure that database connection settings are tuned for reactive performance.
• Connection Pooling: Configure reactive connection pooling to handle concurrent database connections efficiently.

C. Service Discovery and Load Balancing

Reactive microservices can benefit from dynamic service discovery and load balancing to manage distributed instances. Key considerations include:

• Service Registry Integration: Microservices can integrate with service registries such as Eureka or Consul and with this it enables dynamic service discovery and registration.
• Load Balancer Configuration: Configure load balancers to support reactive microservices, ensuring that they can handle high-throughput traffic and route requests efficiently.

D. Logging and Monitoring

Reactive applications require enhanced logging and monitoring to track asynchronous operations and diagnose issues. Configuration changes include:

• Non-Blocking Logging: Microservices can use non-blocking logging frameworks (for example Logback and with this you can use asynchronous appenders) to avoid performance bottlenecks.
• Distributed Tracing: You’ve to implement distributed tracing tools (for example: Zipkin, Jaeger) to trace requests across microservices and monitor performance.

API Changes

Migration to reactive microservices also necessitates changes to API design and implementation. Key aspects include:

A. Adopting Reactive APIs

Reactive APIs support non-blocking interactions and are crucial for achieving the benefits of reactive microservices. Considerations include:

• Reactive Endpoints: Replace traditional REST endpoints with reactive endpoints that return Mono or Flux (in Spring WebFlux) instead of blocking responses.
• Asynchronous Data Processing: Ensure that data processing within APIs is asynchronous and leverages reactive streams to handle large volumes of data efficiently.

B. Error Handling

Reactive microservices require robust error handling mechanisms to manage failures and maintain system stability:

• Error Propagation: You’ve to implement error handling strategies which propagate errors through reactive streams so that it can allow multiple consumers to handle them appropriately.
• Fallback Mechanisms: In your microservice applications, you must have to integrate fallback mechanisms (for example: use Circuit Breaker patterns) to manage service failures gracefully and make sure to achive system resilience.

C. Contract Changes

Migrating to reactive microservices may involve changes to API contracts and interactions:

• API Documentation: Update API documentation to reflect changes in reactive API endpoints and data formats. Tools like Swagger (OpenAPI) can help document reactive APIs.
• Backward Compatibility: Ensure that changes to API contracts do not break existing clients. Consider versioning APIs or providing compatibility layers during migration.

D. Testing and Validation

Testing is crucial during migration to validate the functionality and performance of reactive APIs:

• Reactive Testing: We can use reactive testing frameworks and tools (for exmpple Reactor Test for Spring WebFlux and also you can use Spock groovy) to test asynchronous behavior and you can validate reactive interactions.
• Performance Testing: You have to include performance testing to evaluate the throughput and latency as well as resilience of reactive microservices under high load.

Migrating to reactive microservices represents a significant shift in how applications are designed, configured, and managed. By addressing configuration changes and adapting API design to embrace reactive principles, organizations can achieve improved scalability, responsiveness, and resilience in their microservices architecture. Careful planning, execution, and testing are essential to ensure a successful migration and realize the full potential of reactive microservices.

Example Implementation:

Migrating a Reactive E-Commerce Microservice

Consider an e-commerce platform where reactive microservices handle order processing and inventory management using Kafka. The platform is currently running on Spring Boot 2.x with Kafka 2.x client libraries. As part of a system upgrade, you need to migrate to Spring Boot 3.x and Kafka 3.x.

Scenario Overview

• Current Setup: Spring Boot 2.x with Kafka 2.x -> reactive programming with Project Reactor.
• Target Upgrade: Spring Boot 3.x, Kafka 3.x -> upgrade the dependencies to leverage updated Kafka client features and enhanced reactive support.

1. Upgrading Kafka Dependencies

First, update your project dependencies to include the latest Kafka client libraries compatible with Spring Boot 3.x.

For Maven:

For Gradle:

This upgrade includes both Kafka 3.x and its client libraries, which provide improved performance and new features.

2. Kafka Producer Configuration Changes

The configuration for Kafka producers has seen notable changes. Here’s how you can adapt to these changes:

Old Configuration (Spring Boot 2.x):

Updated Configuration (Spring Boot 3.x):

Changes Explained:

• acks: The acks property is set to all to ensure that the producer receives acknowledgment from all in-sync replicas, enhancing data durability.
• compression-type: Using gzip for message compression can reduce network usage and improve throughput.

Reactive Kafka Producer Implementation:

In Spring Boot 3.x, use KafkaSender for reactive producer implementations:

This configuration utilizes KafkaSender, which integrates with Project Reactor for non-blocking operations, aligning with reactive microservices requirements.

3. Kafka Consumer Configuration Changes

Similar to producers, Kafka consumer configuration needs updating:

Old Configuration (Spring Boot 2.x):

Updated Configuration (Spring Boot 3.x):

Changes Explained:

• max-poll-records: Increased to 100 to allow batch processing of messages, improving throughput.
• enable-auto-commit: Set to false to manually manage offsets, providing finer control over message processing.

Reactive Kafka Consumer Implementation:

For reactive consumption, use KafkaReceiver:

KafkaReceiver enables reactive message consumption, integrating seamlessly with Project Reactor’s non-blocking nature.

4. Testing and Validation

After updating configurations and implementing new APIs, conduct thorough testing to validate:

• Message Throughput: Ensure that the updated producer and consumer handle the expected load efficiently.
• Compatibility: Verify that the new configurations work seamlessly with existing Kafka clusters and microservices.
• Performance: Benchmark the performance to confirm improvements and identify potential issues.

Conclusion

Adapting Kafka producers and consumers in Spring Boot 3.x requires changes in API and configurations. You can shift toward enhanced performance and scalability of microservices by moving from traditional ways of Kafka client libraries to new ones and using reactive programming techniques. I have provided an real-time example of how you can be able to managethese changes in an effective way and maintain its effective functioning. Also discussed about upgrade process step by step