排查Spark创建临时目录出错

Spark任务运行时报错:

java.io.IOException: Failed to create a temp directory ...

根据函数栈定位报错的函数,然后分析它的实现源码:

273   def createDirectory(root: String, namePrefix: String = "spark"): File = {
274     var attempts = 0
275     val maxAttempts = MAX_DIR_CREATION_ATTEMPTS
276     var dir: File = null
277     while (dir == null) {
278       attempts += 1
279       if (attempts > maxAttempts) {
280         throw new IOException("Failed to create a temp directory (under " + root + ") after " +
281           maxAttempts + " attempts!")
282       }
283       try {
284         dir = new File(root, namePrefix + "-" + UUID.randomUUID.toString)
285         if (dir.exists() || !dir.mkdirs()) {
286           dir = null
287         }
288       } catch { case e: SecurityException => dir = null; }
289     }
290
291     dir
292   }

MAXDIRCREATIONATTEMPTS的值为10,可以看出,如果创建10次随机目录失败,就报出IOException异常。判断是否成功的依据是:

if (dir.exists() || !dir.mkdirs())

要么目录存在,要么新建失败。一般来说尝试十次是不会反复出现目录已存在的,而mkdirs失败的原因多半是磁盘空间问题。