Java Memory Management and Garbage Collection

Java has always been renowned for its automatic memory management. Developers can focus more on writing business logic while the Java Virtual Machine (JVM) handles memory allocation and garbage collection under the hood. However, as applications scale and performance becomes critical, understanding how memory management and garbage collection (GC) work becomes essential.

Java Memory Model: The Basics

Java memory model is built around a few core memory areas managed by the JVM: 

1. Heap Memory

• Young Generation: Contains short-lived objects. Includes Eden and two Survivor spaces.

• Old Generation: Stores long-lived objects.

2. Stack Memory

• Used for method invocations and local variables.

• Each thread has its own stack.

3. Metaspace (formerly Permanent Generation)

• Stores class metadata and static methods. Introduced in Java 8.

4. Program Counter and Native Method Stack

• The program counter keeps track of the executing instruction.

• Native method stacks support native (non-Java) methods. 

How Garbage Collection Works

Garbage Collection in Java identifies objects that are no longer reachable from the application and reclaims the memory. The main steps include:

• Reachability Analysis: Traces references from GC roots (local variables, static fields, etc.).Mark Phase: Marks all reachable objects.
• Sweep/Relocate Phase: Deletes unmarked objects and optionally compacts the heap.
• Promotion: Objects surviving multiple GC cycles in the Young Generation are promoted to the Old Generation. 

GC can be categorised into:

• Minor GC: Cleans the Young Generation.
• Major (Full) GC: Cleans the entire heap.

Evolution of Garbage Collection in Java

Java 1.0 - 1.3

• Basic mark-and-sweep collector.

Java 1.4 - 5

• Introduced Generational GC.

• Concurrent Mark Sweep (CMS) introduced for low pause times.

Java 6 - 7

• Improved tuning and parallelism in collectors.

Java 8

• G1 GC gains maturity.

• Metaspace replaces Permanent Generation.

Java 9 - 10

• G1 GC becomes the default.

Java 11 - 14

• ZGC and Shenandoah GC introduced for ultra-low pause times.

Java 15 - 17

• ZGC becomes stable.

• CMS deprecated and removed.

Java 21

• Generational ZGC introduced as a preview.

ZGC vs. Generational ZGC

ZGC

• Non-generational, single-heap design.

• Sub-millisecond pause times.

• Scans the entire heap.

Generational ZGC (Java 21+)

• Divides heap into Young and Old generations.
• Uses minor GCs to clean Young Generation quickly.
• Improves throughput while maintaining low latency.

Feature	ZGC	Generational ZGC
Heap Layout	Flat	Young + Old Gen
Pause Times	Low	Even lower
Throughput	Moderate	Higher
Java Version	11+	21+ (Preview)

Choosing the Right Garbage Collector

Key Factors to Consider:

1. Latency Requirements

• Real-time apps: Use ZGC, Shenandoah, or G1.

2. Heap Size

• Small (<4 GB): Serial or G1 GC

• Large (>32 GB): G1, ZGC, Shenandoah

3. Throughput Needs

• Batch jobs: Parallel GC

4. Startup Time

• Serial GC for fast-starting small apps

5. Resource Constraints

• In containers, prefer G1 or ZGC

6. JDK Version

1. Use GC supported by your JDK (e.g., Generational ZGC in Java 21+)

Flowchart: Quick Decision

 

 

Conclusion

Java’s memory management and garbage collection have evolved tremendously, making it easier to build scalable and performant applications. With the arrival of Generational ZGC in Java 21, developers now have access to a GC that balances ultra-low latency and high throughput.

Choosing the right GC depends on your application’s goals—whether it's minimising pause times, maximising throughput, or optimising for constrained environments. Understanding these tools helps you get the most out of your JVM.

 

 

Consequences of high volume traffic

Let’s consider we have a Java service running on a single server. Clients can make requests to the service via REST API calls. The REST API implementations may contain DB operations, use of multiple threads, calls to external APIs, operations with high memory usages etc. What will be consequences if there is a huge spike in the requests?  

1. Database Overload: The database connection pool could be overloaded due to a sudden spike. A database connection pool overload occurs when too many connections are requested from a pool, either due to a large number of concurrent users or inefficient application code. This can lead to: 

• Slow response times: Applications become unresponsive or take a long time to complete requests.
• Database overload: The database server may become overwhelmed, leading to resource contention and reduced throughput.
• Application crashes: Applications may crash due to an inability to acquire a database connection.
• Database outages: In severe cases, the database server may become unavailable.  

How to address overload:

• Optimize application code: Ensure that connections are properly closed after use and that queries are efficient. 
• Adjust pool size: Increase the pool size to accommodate peak demand, but be mindful of database resources. 
• Implement connection management: Use tools like connection pooling libraries or middleware to manage connections more effectively. 
• Monitor database performance: Use monitoring tools to track connection pool usage and identify potential bottlenecks. 
• Implement connection rate limits: Limit the number of connections an application can establish to prevent overwhelming the database. 
• Consider database-level limits: Some databases offer features to limit the number of concurrent connections.  

2. Increased CPU and Memory Usage:  If the application is not optimized, a surge in requests can cause high CPU and memory consumption, potentially leading to slow response times or crashes. Resources how to monitor and manage high CPU and memory usage:

• https://www.site24x7.com/learn/java/troubleshoot-java-high-cpu-usage.html
• https://docs.dynatrace.com/docs/ingest-from/technology-support/application-software/java/top-java-memory-problems  

3. Network Latency and Timeout Issues: High traffic can overwhelm network bandwidth, increasing response times or causing timeouts if external services (like APIs or payment gateways) are involved.

4. Garbage Collection (GC) Pressure: High memory allocation due to excessive object creation may trigger frequent garbage collection, causing application pauses and affecting performance.

5. Thread Contention and Blocking Issues: If your application relies on synchronized methods or database connections with limited threads, multiple requests may get stuck waiting, leading to bottlenecks. Resource how to monitor thread connections:

https://medium.com/@jaehnpark/how-to-define-java-thread-contention-87196c447e12  

 

How to mitigate:

• Implement caching (like Redis) to reduce database load. Caching is useful for repeated fetch requests. There should be a proper cache eviction policy so that the cache itself is not overloaded.
• Use asynchronous processing (CompletableFuture, ExecutorService) to handle requests efficiently.
• Keep the requests in queue before doing the actual processing. The queue can be either in-memory or external.
• Optimize thread management using a proper thread pool configuration.
• Use rate limiting (e.g., API Gateway, Bucket4j) to prevent overload.
• Start to scale the system by adding more servers.  We can scale the system horizontally by using load balancers.