top of page
  • khangaonkar

JAVA Virtual Threads Tutorial

Introduction

Virtual threads are light weight threads added to the JAVA language to not only make it easier to write concurrent programs but more importantly increase throughput of such programs.


Virtual thread is a modern concurrency concept. Other languages already have it - Kotlin (coroutines), Golang (goroutines). This is JAVA playing catch up.


It is a preview feature introduced in JDK19 JEP 425 . To use, start the JVM with --enable-preview flag.



JAVA Virtual Threads
JAVA Virtual Threads


What are Virtual Threads ?

With conventional threads, one thread is mapped to one OS thread. OS threads take up a lot of memory (can be of the order of MB) and you can have only so many of them before you run out of resources. Thread pools alleviate the problem a little bit. But even in a thread pool, threads that are blocked are not used by any other task.


A virtual thread is an instance of java.lang.Thread that is not tied to any particular OS thread. On the other hand a conventional thread is a java.lang.Thread that is tied to a OS thread.


When a virtual thread executes non blocking code, it is associated with a OS thread. But when the JVM detects a call to a blocking API, the virtual thread is suspended and dis-associated from the OS thread.


With Virtual threads, multiple virtual threads are tied to one OS thread at different times. of course the OS thread can run only 1 virtual thread at a time. But when a virtual thread is blocked, the OS thread will execute other virtual threads.


Figure 1 shows the relationship of Virtual thread to java.lang.Thread and OS threads.


How to create and use a Virtual Thread ?

The sample code below shows how to create and run a virtual thread.


public static void main(String[] args) {
    Runnable runnable = () -> {
        for(int i=0; i<10; i++) {
            System.out.println("Greeting from virtual thread: " + i);
        }
    };

    System.out.println("Starting the virtual thread");
    Thread vThread = Thread.ofVirtual().start(runnable);

    try {
        vThread.join();
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
    
    System.out.println("Done running virtual thread");
}

It is possible to use an executor service that runs tasks using virtual thread. In fact this is the preferred method.


ExecutorService es = Executors.newVirtualThreadPerTaskExecutor();

System.out.println("running virtual thread us Executor service");

es.submit(runnable);

Some details on Virtual threads

Call stack frames are stored on the heap. So they can be garbage collected. Footprint of a virtual thread is very small compared to a conventional thread.


Virtual thread are ideal for programs that use 1 thread per request. This is the use case of your typical micro-service that accepts http requests and uses 1 thread to respond to the request. In the old model, To go from 10 requests per sec to 1000/sec to 10000/sec , the JVM and OS needs to create more threads and at some point the JVM will run out of resources. With Virtual thread, you are re-using few OS threads for multiple virtual threads. Hence you can achieve a much higher through put.


A virtual thread can execute any code that a conventional thread can.


With virtual threads it is no longer necessary to use thread pools. Pools are generally used for expensive resources and virtual threads are not expensive resources.


Virtual threads cannot be unmounted from OS thread in 2 cases. (1) when code is executing in synchronized block. (2) during native calls. Hence developer needs to take care to avoid blocking calls in synchronize blocks as it will hinder scalability.


The stacks of virtual threads are stored in the Heap that has garbage collection. This is an implementation detail that helps with the scalability.


The execution of virtual threads is pre-emptive. The JVM decides when to run them and when to suspend them. They do not support the stop(), suspend() , resume() methods available in the conventional threads.


Thread local variables are supported but not recommended. Generally virtual threads are short lived and numerous (millions). Care is needed when using thread local variables to ensure they are cleaned up correctly to avoid program errors. Another new feature "scope local variable" is a the recommended alternative.


Performance comparison with conventional threads

Let us do a small performance comparison. The code above creates 100000 virtual threads. Each thread sleeps for 1 ms. On my mac with m2 chip and 16GB memory it executes in about 270 ms.

public static void main(String args[]) throws Exception{

        ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();

        CountDownLatch c = new CountDownLatch(100000);

        long t = System.currentTimeMillis() ;
        IntStream.range(0, 100000).forEach(i -> {
            executor.submit(() -> {
                Thread.sleep(1);
                c.countDown();
                return i;
            });
        });

        c.await();

        System.out.println(System.currentTimeMillis() - t);

    }

The same code implemented using java.lang.Thread takes 127000 ms. 500 times slower.

public static void main(String args[]) throws Exception {

        CountDownLatch c = new CountDownLatch(100000);

        long t = System.currentTimeMillis();
        IntStream.range(0, 100000).forEach(i -> {

            Thread thread = new Thread(() -> {
                try {
                    Thread.sleep(1);
                } catch (Exception e) {
                    System.out.println(e);
                }
                c.countDown();
            });

            thread.run();
        });

        c.await();
        System.out.println(System.currentTimeMillis() - t);

    }

Lastly the same case implemented using a thread pool takes 630 ms, about double the time.

public static void main(String args[]) throws Exception{

        ExecutorService executor = Executors.newCachedThreadPool();

        CountDownLatch c = new CountDownLatch(100000);

        long t = System.currentTimeMillis() ;
        IntStream.range(0, 100000).forEach(i -> {
            executor.submit(() -> {
                Thread.sleep(1);
                c.countDown();
                return i;
            });
        });

        c.await();

        System.out.println(System.currentTimeMillis() - t);

    }

Summary

Virtual threads are a modern concurrency concept that give a huge boost to the scalability of concurrent applications in JAVA. The performance benefit is acheived by having multiple virtual threads share OS thread. When a virtual thread is executing non blocking code it is tied to a OS thread. When the code blocked, the virtual thread is untied and the OS thread can be used by another virtual thread. Any code that can execute in a normal JAVA thread can also run in virtual threads. So the code can ported easily. Once Virtual threads become a regular feature, this will be the recommended way to build concurrent applications.

Recent Posts

See All

Go Tutorial: Receivers

When I was new to the Go programming language and saw this syntax func (m *Service) addNumbers(a int32, b int32) int32 I was confused. I had not seen syntax like (m *Service) between func and the func

JDK22: New features in Java 22

JDK 22 was released on March 19, 2024. As always, a new suite of goodies is made available for the benefit of the Java programmer. Many of them are in preview. This is a very brief listing of the new

Go review: Should I use the Go programming language ?

Overview Go was developed by engineers at Google. Their primary motivation was dislike for C++. Goal was to create a language for systems programming. Its popularity has been slowly but steadily incre

Comments


bottom of page