For the love of Java, Executors, FunctionalInterface: Evolve or die!

 I have decided to continue on the for the love of Java with one of my favorite topics: Threads and Executor Services! Lately, I have been doing some work in "other" languages. Now remember, I am taking the high road, so I wont name them. Remember, I am not there yet. So yes I will "Python". I am not going to take too many potshots at these lesser languages (Ow! I just did burn) outside of this one sentiment, "Evolve or die". I will show you how Java has evolved from Threads to Executor services, and get into the new Virtual Threads.

Like the last blog all the code I am going to show is in one super tight commit. However, you don't have to read the commit ahead of time I am going to walk through it. Without further ado lets go!

Threading brief

Threading is a difficult topic. It is usually one I use during and interview to separate intermediate from advanced programmers. The intermediate may know something about threads and executors, but usually lack the "deep knowledge" on coordinating complex tasks. Everyone has more of less seen the "telnet example" where each connection "forks" off something "thread-ish". When you wade into "deeper water" with semaphores, thread pools, shared memory, and synchronization you do not find as many swimmers. 
 
Since the beginning the Java language has had  multi-threading, semantics for memory synchonization. It greatly over the years with performance and features (ExecutorSevice concurrent collections). 
 

The old old way

 Lets say you were writing Java 15 years ago and you wanted to use threading. Not just a thread that does "println". A thread that you required to run to completion and get a result. You would code it up something staring with the Java interface for Runnable.
 
 
public interface Runnable {
    /**
     * Runs this operation.
     */
    void run();
}
The first challenge was you had to develop your own way to watch the thread and return a value. I made an abstract class "MyFuture" that has a placeholder for the result of the computation and a field that tells the done-ness. 
@Test
void oldOldWay() throws InterruptedException {
    abstract class MyFuture implements Runnable {
        protected volatile Object result;
        private volatile boolean done = false;
        @Override
        public void run() {
            this.done= true;
        }
        public Object getResult(){
            return this.result;
        }
        public boolean isDone(){
            return done;
        }
    }
    MyFuture meaningOfLife = new MyFuture() {
        @Override
        public void run() {
            this.result = Long.valueOf(42);
            super.run();
        }
    };
    Thread t = new Thread(meaningOfLife);
    t.start();
    t.join();
    // we have computed the meaning of life
    assertTrue(meaningOfLife.isDone());
    assertEquals(Long.valueOf(42), meaningOfLife.getResult());
} 
 This works even in the most "primitive" form. It doesn't suffer from "Global Interpreter Lock"  or the c++/c/boost, style "go find a library", "go figure out the shared memory problems" (SHOTS FIRED!). But this is Java we don't just rest on our laurels. 
 
 

Along comes Java 1.5 

Please note, at this point in the blog we are still taking about Java from ~13 years ago. Of course having everyone make their own "future" class is a silly. It is "easy" to do as you can see above, but it leaves users lots of "rope to hang themselves" with. Along comes callable:

public interface Callable<V> {
    /**
     * Computes a result, or throws an exception if unable to do so.
     *
     * @return computed result
     * @throws Exception if unable to compute a result
     */
    V call() throws Exception;
}


Well that makes sense. Many threads are going to return "something", at minimum the something is a true/false that it finished. Callable can (and is) used outside of threading. Maybe in some languages you might call it a block.   

But what about managing pools of threads? Re-using the threads? For simple tasks "fork an instance each connection" it is simple, but for more advanced tasks like workers spawning other workers and figuring out that lifecycle it isn't that clear cut. There isn't a "once size fits all answer", but I would argue the Executor Service is brilliant:

ExecutorService executor = null;
try {
    //normally you would create this once at application init time
    executor = Executors.newCachedThreadPool();
    Future<Integer> future = executor.submit(something);

The user defined MyFuture is no longer needed. We have a first class Future in the language, and there are multiple Executor implements available (SingleThreaded, ForkJoinPool) to fit common workload types. Let see it all in action!

@Test
void oldWay(){
    Callable<Integer> something = new Callable<>() {
        @Override
        public Integer call() {
            return 4;
        }
    };
    ExecutorService executor = null;
    try {
        //normally you would create this once at application init time
        executor = Executors.newCachedThreadPool();
        Future<Integer> future = executor.submit(something);
        assertEquals(4, future.get());
    } catch (ExecutionException | InterruptedException e) {
        throw new RuntimeException(e);
    } finally {
        if (executor != null){
            executor.close();
        }
    }
}

Are we done yet? No this is Java, you gotta "gimme some more".


 Enter Function Interface

Now, from my non Java friends I sense a 20 year old complaint coming. It is the "public static void main(string [] args)" complaint. Right here:


Callable<Integer> something = new Callable<>() {
    @Override
    public Integer call() {
        return 4;
    }
};


Well well well. This too is solved in many (but not all) cases. Enter @FunctionalInterface: 

@FunctionalInterface
public interface Callable<V> {
    /**
     * Computes a result, or throws an exception if unable to do so.
     *
     * @return computed result
     * @throws Exception if unable to compute a result
     */
    V call() throws Exception;
}

 

 It stands to reason that if a class or an interface only has a single method, we can infer the "boilerplate". To understand why we only have to look at the code of Executor.submit(Callable<T> task). The submit method takes a callable as an argument.  

<T> Future<T> submit(Callable<T> task);

When you combine "callable has only one method "call()" and submit takes a callable, well there is no longer a need to name the class, or the METHOD! Lets see what that looks like:

@Test
void newWay(){
    Callable<Integer> something = () -> 4;


 Why stop there? We do not even need a reference to the Callable!

@Test
void moreConciseWay(){
    try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) {
        assertEquals(4, executor.submit(() -> 4).get());
    } catch (ExecutionException | InterruptedException e) {
        throw new RuntimeException(e);
    }
}

 


 The "java sux because public static void main" crew is is right now "running for the self hills". (languages that you have to type self all the time you know who your are)

Enter Virtual threads


 

OMG! with all the images and trolls we are running into a danger zone in this blog. Lets bring it home with something solid, that has some technical value. You may have noticed above:

 

try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) {

Things have changed over the years. Years ago SEDA or Staged Event Driven Architecture or Disruptor  was though of as a elegant and performant way to do multi-threaded applications. Hardware has changed and improved and research has moved forward. Many application need a hybrid model, not "one thread per connection" not "requests moving through pools of threads". Java's lightweight VirtualThreads are not handcuffed to system processes, and they are able to yield when they are blocked on IO (waiting for a socket, or reading from a file). 

Notice in the example above we were able to move to virtual threads without switching to a new suite of libraries. The code is still using the sample Callable and ExecutorService interface, a new implementation of ExecutorService is reachable by a small change.

#fortheloveofjava 


 










Comments

Post a Comment

Popular posts from this blog

For the love of Java

 My old blog is lost, I think I have a mysql DB on a SSD disk on its last leg. Anyway the old Ed, the old blog, might go on a rant. These days I try to take the high road. I say try, I'm not all the way there. A few weeks back I found a linked in post that was discussing Java 23 added "var" and that it was "too little too late".  In case you didn't know the story, I was born to be a Java fanboy. When the applets made 3d shapes in my browser, I was in love. This was a long time ago, I had windows 3.11 and you needed a 32 bit OS to run Java, upgrades were required. My Packard bell PC only had 8MB RAM, upgrades were required.    Everyone is free to like and dislike what they want. I'm going to say my piece about "too little too late". Over the years Java improved drastically, both in terms of performance and features. Even features I didn't like or understand initially, I now use like second nature. Most impressive to me is the 'old wa...
Read more

Large Language Models Termperature and relative performance

 When you read about performance and tuning for large language models the term Temperature comes up often. I find a lot of interesting description of it from "It controls creativity", "How hard it thinks". What I like to say, "temperature at 0.0 is prone to over-fit".  I avoid "high temperature is  more likely to hallucinate," as I feel the 0.0 'over fit' looks a lot like a hallucination as well.   I rebuilt some of the deliverance code for sampling to give it an Object Oriented Design style face-lift, and prepare for adding repetition penalty.  As always feel free to look at the code here:  https://github.com/edwardcapriolo/deliverance/commit/0162c1daa07cba6b43d4e4f75cb95f50b0ff11b2 I was never overjoyed with was the size of the AbstractModel class . Many of the features are tightly coupled, I wanted to get the sampling into it's own class. It is not 100% clean as we still walk back into the AbstractModel to get config values and ot...
Read more

Guided Choice: Get only the answer you want with no fluff

 In my last blog we refactored the text sampling code in deliverance . I did that to prepare the code to take on something clever. You know LLM sometimes give you the answer inside a sea of text and never stop talking (like me). A solution to that is "structured outputs", which helps you control what the inference engine will produce. There are several forms of structured outputs. A simple one is guided choice . Effectively you give the inference engine a prompt and a list of choices. It should only answer with one of them.  Something like:  prompt= "What is the best month for vacation",  choices = ["January", "February"]  So lets code it up! As always here is the code   Guided Choice commit One thing to think about ahead of time. Call it a "trick". LLMs don't answer in words, they answer in tokens.  If you ask an LLM a question like "Who is the best NFL team". It might have "Giants" in the vocabulary, or it migh...
Read more