How to orchestrate and design a reactive Weather Application in Java Microservices

Weather APP common terms: Before digging deep into Weather APP architecture, let’s first go through some of its common terms:

Weather APP design concepts:

Build API for accepting city name/zip information and providing the current weather. You have to mock the weather data (store it in memory DB) using the library/framework of your choice. Frontend:

A widget to accept a city name or zip (postal) code information.

A widget will be displayed here the current weather condition that includes:

• day (e.g. "Monday, March 1st, 2021")
• weather description (e.g. "Misty")
• Temperature
• Pressure
• humidity and
• wind information

Instruction:

The focus is more on the component-level design and the API.

Focus on

• Debuggability
• Scalability
• Extensibility
• No constraints on language and tech stacks

Design Guidelines:

• Ability to run this on cloud and preferably Kubernetes.

• Production Deployments and rollouts

• Reasons for Tech stack selection

• Production readiness of the service

Requirements for Weather APP design:

Functional requirements:

• Weather APP Widget Generation: Our Java Development Team should be able to generate a widget for selecting the city
• Weather API call: Given a city name, our system should be able to fetch the weather details from backend and display them on UI in another widget
• Weather forcst: History: Can display the entire day’s weather change history for a given city

Non-functional requirements:

• Availability: Our system should be highly available because even a fraction of a second downtime would result in the inability to retrieve weather information. Because our system's domain is cached, we don't have the benefit of downtime, and our design must include fault-tolerance conditions.
• Scalability: As demand grows, our system should be horizontally scalable.
• Readability: Our system's weather details should be easily readable, distinguishable, and typeable.
• Unpredictability: From a security standpoint, the call to our backend system to fetch weather details should be highly secure.

APIs for systems:

The application master creates a sequencer and sends it to the worker servers.

Calling the backend Weather API (Weather Microservices)

• Get the weather Report (/weather )
• The GET method of HTTP is used to call the /weather API:
• getWeather( date_time, city)

Resource estimation:

It is preferable to begin with realistic estimates. For example, we may need to change them in the future due to design changes. To complete our estimation, let us make some assumptions...

Assumptions:

• We assume that the number of weather app requests per month is 200 million.
• A response entry for weather details requires 500 bytes of database storage.
• Unless explicitly deleted, each entry will have a maximum expiry time of 12 hours.
• There are 100 million active users on a daily basis (DAU).

Storage estimation

Because entries are saved for 12 hours years and there are 200 million entries per month, the total number of entries will be around 2 billion.

• Because each entry is 500 Bytes, the total storage estimate is 6 TB: • 2 billion * 500 bytes = 1 terabyte
• 12 billion * 500 bytes = 1 terabyte

Memory estimation

We need memory estimates if we want to cache all of the cities' weather details ahead of time.

• Assuming 7.6 K new requests per second, the total for one day would be 0.66 billion. 7.6 times 3600 seconds times 24 hours equals 0.66 billion.
• 7.6 K×3600 seconds×24 hours=0.66 billion

Because we would only consider caching 20% of these requests every 12 hours, the total memory requirements would be estimated to be 33 GB. The daily limit will be 66 GB.

• 0.2 * 0.33 Billion * 500 Bytes = 33 GB

Application master generating a sequencer and passing it to worker servers

Building blocks of Weather APP Design:

We will use the following building blocks: We can now identify the key building blocks in our design based on our estimations. A list of such items is provided below:

Below High-level design building blocks include:

• To store weather data for all cities, a database (or databases) will be required.
• Load balancers at different layers will ensure that requests are distributed evenly among available servers.
• API gateways are required to ensure high availability and to route requests to Microservices.
• Caches will be used to store the most frequently requested city-related requests.
• To prevent system exploitation, rate limiters will be used.

Components Design of Weather App

UI Components

I have explained the inner mechanism of different components within our system, as well as their usage as a part of the whole system below. We'll also discuss the design decisions made for each component in order to achieve overall functionality.

Here frontend, we are going to develop using React Js framework where we will be handling both login flow as well as calling our backend Weather Services API.

React Code Snippet:

WeatherApiService.js

Here from my weatherApiService.js file, I am calling the openweather library API to fetch the weather details. Similarly you have to call your backend Microservices from this Js file where your backend service will be responsible for calling the openweather Library to fetch the weather details for a given city.

The response we are getting here is temp, feels_like,temp_min, temp_max, humidity, country, sunrise, sunset, details, speed.

Then I am formatting this response in the formatForecastWeather() method as given below:

End-to-end Components Design:

There has been quite a good design for this requirement and I have explained a very simpler and more sophisticated system design which is industry standard.

Latency

• When a user inputs the city from the browser then the request goes to the NGINX plus API gateway where it verifies the Access token using OAuth identity provider from Okta/Google and then it verifies the client certificate sent from browser.
• Once the security check is passed, then it checks the NGINX cache whether the response exists for the same city in the cache and if it exists then it sends the cached response without calling to backend Microservices.
• If the cache does not contain the data for the requested city, then API gateway calls the backend Microservices (Weather App micro service) to get the weather data for a given city. Our backend services fetch the weather data from Redis cluster.
• There is already a batch processing happens where the processor calls the openweather service every 12 hours to fetch the weather data for all cities asynchronously and then saves the response in Redis cache.

Availability

• We need high availability for users requesting weather details for a city. Our design is sufficing this as Most of our building blocks, like databases, caches, and application servers have built-in replication that ensures availability and fault tolerance. The weather app system will not impact the availability either, as it depends on easily replicable redis cache clusters which can be maintained with 2-3 sets of node replicas.

Scalability

• Our design is scalable because our caching data can easily be distributed among various redis clusters which are highly available. To balance the load between the build scalable Java application development and database layers, we can use a consistent hashing scheme. Our choice of the mongo database for storing the user details for auditing purposes. We can fetch the user details from the JWT token.

Latency

• Our system ensures low latency with its following features: Even the most time-consuming step across the weather App system is fetching the weather details process. The overall time to display weather details for a specific city is relatively low, ensuring there are no significant delays in this process. As we are using redis cluster and processing the weather details beforehand every 12 hours of time.
• Also, we are maintaining the cache at API gateway level where the NGINX cache is able to cache the response for a given request based on city.

The entire API gateway as well as the weather app micro service are deployed into Kubernetes cluster. I have not explained the deployment much but I have given the code snippet below for the backend whether Microservices are developed with Springboot.

Backend code Implementation with Springboot:

WeatherAppService.java

• This service is responsible for calling the openweather API which returns the weather details for a given city. It returns weather details as well as weather forecast details. These data will be stored in redis cache every 12 hours.
• I have just implemented the Microservices code. You can extend to make it happen through a Springboot scheduler in every 12 hours’ time span.
• Here in the above service class, I am building the rest template object and the URLs for getting the weather data as well as weather forecast data. Here, the next part is to store the results in a cache.
• One cache is for storing weather details for a city and another is for the weather forecast. I have used here the @cacheable annotation to enable a method's caching behavior and parameterize it with the name of the cache (in this case, weather-details) where the results will be saved:

Conclusion

• The Weather APP system is an effective service with multiple advantages. Our design of the weather app service is simple, yet it fulfills all the requirements of a performing design. The key features offered by our design are:
• A responsive Web application that can give results for all cities across Globe with Improved readability