Spark – java.nio.channels.UnresolvedAddressException

A very short writedown of the following error, which apperently this user also encountered and documented (github). Be aware, that this error code might appear in several scenarios. It just happened, that in my specific situation, it was an easy solve.


I run a Spark image (spark:3.5.0-scala2.12-java11-python3-ubuntu) in a Docker Compose. To start a master server, I defined the command

command: "/opt/spark/bin/spark-class org.apache.spark.deploy.master.Master --ip ${SPARK_MASTER_HOST} --port 7077 --webui-port ${SPARK_MASTER_WEBUI_PORT}"

However, the container shut downs almost immediately with the following error code

24/05/07 13:54:03 ERROR SparkUncaughtExceptionHandler: Uncaught exception in thread Thread[main,5,main]
        at java.base/ Source)
        at java.base/ Source)
        at io.netty.bootstrap.AbstractBootstrap$
        at io.netty.util.concurrent.AbstractEventExecutor.runTask(
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(
        at io.netty.util.concurrent.SingleThreadEventExecutor$
        at io.netty.util.internal.ThreadExecutorMap$
        at java.base/ Source)
24/05/07 13:54:03 INFO ShutdownHookManager: Shutdown hook called


So actually ChatGPT gave me the right hint, he said:

The error message you’re encountering, java.nio.channels.UnresolvedAddressException, typically occurs when Spark (or an underlying networking library like Netty) is trying to bind to a network address that cannot be resolved. This error generally indicates that the application is attempting to use an incorrect or non-existent hostname or IP address.

Possible Causes and Solutions

  1. Invalid Hostname/IP Address:
    • Ensure that the hostname or IP address you’re using is valid and resolvable. Check the network settings and any environment variables used to specify the addresses.
    • In a Docker environment, confirm that all services and containers can correctly resolve the specified hostnames.
  2. Misconfigured Bind Address:
    • The bind address is the network interface Spark attempts to bind to. Verify the values of environment variables like SPARK_MASTER_HOST, SPARK_LOCAL_IP, or any other bind address parameters.

That gave me the necessary hint. I just defined SPARK_MASTER_HOST wrongly and should have defined it as as the application is running in a Docker Container.

Leave a Comment

Your email address will not be published. Required fields are marked *

hungsblog | Nguyen Hung Manh | Dresden
Scroll to Top