Desktop Applications: Temporary Files

This article discusses temporary files, including Java's default model and potential enhancements.

Temporary Files

Java comes with a few built-in features related to temporary files:

  • The "java.io.tmpdir" system property. You can access this by calling System.getProperty("java.io.tmpdir"). (It's possible you'll get a security-related exception if your code is executed in a sandbox.)
  • The File.createTempFile(..) method. This creates a file in the right location, but the javadoc cautions: "This method provides only part of a temporary-file facility. To arrange for a file created by this method to be deleted automatically, use the deleteOnExit() method."
  • The non-static File.deleteOnExit() method, which queues your File object for deletion if the JVM exits normally.

The "java.io.tmpdir" is obviously important; on all platforms (that I know of) it's the recommended way to manage temp files. But over the years I've developed alternatives to the static File methods, because I think they can be lacking. Here are some of the issues I have with them:

  • Deleting files is expensive and inefficient. The File.deleteOnExit() currently resembles:

        public void deleteOnExit() {
            SecurityManager security = System.getSecurityManager();
            if (security != null) {
                security.checkDelete(path);
            }
            if (isInvalid()) {
                return;
            }
            DeleteOnExitHook.add(path);
        }
    This ultimately boils down to:

            Collections.reverse(toBeDeleted);
            for (String filename : toBeDeleted) {
                (new File(filename)).delete();
            }

    So we're looping through a list to delete files. If that loop has a few million entries: it's painfully slow.

  • Deleting files is unreliable. The deletion takes place in shutdown hook. (Although it is some sort of special, highly prioritized shutdown hook?) The problem is: shutdown hooks may be arbitrarily cut off. If the OS decides your app is out of time: it's killed. Combine that with what might be a long-running loop and you may have orphaned files.

    Or what if lightning strikes and you lose power? In that case shutdown hooks won't run at all.

  • These methods lend themselves to files, but not directories. The File.createTempFile method resembles:
        public static File createTempFile(String prefix, String suffix,
                                          File directory)
            throws IOException
        {
            if (prefix.length() < 3)
                throw new IllegalArgumentException("Prefix string too short");
            if (suffix == null)
                suffix = ".tmp";
    
            File tmpdir = (directory != null) ? directory
                                              : TempDirectory.location();
            File f;
            try {
                do {
                    f = TempDirectory.generateFile(prefix, suffix, tmpdir);
                } while (f.exists());
                if (!f.createNewFile())
                    throw new IOException("Unable to create temporary file");
            } catch (SecurityException se) {
                // don't reveal temporary directory location
                if (directory == null)
                    throw new SecurityException("Unable to create temporary file");
                throw se;
            }
            return f;
        }
    After the while loop: we explicitly ask to create the file as a file (see createNewfile instead of mkdir()).

Improvements

The TempFileManager is an attempt to resolve these problems. It requires explicit initialization (with a unique name), so although it is intended to replace Java's default model: it is not as universally flexible. Although it works out-of-the-box, I'm going to discuss the implementation details in the rest of this section.

Suppose your application's name is "Mulligan". Following Java conventions, we might give it a qualified name like "com.myCompany.Mulligan", just to help differentiate it from a competitor's product with the same name. The first thing we do is we create a subdirectory in the temp folder called "com.myCompany.Mulligan". We basically expect ownership of this directory, and we assume no other entity is going to mess with it.

Inside the "com.myCompany.Mulligan" directory: we're going to create a new directory for every session. All of our temp files for the rest of our session go in this folder.

This subtle framework opens up a lot of new possibilities for us:

  • Cleaning up files is now efficient. There are optimized ways to delete a directory (consider this or this link). We can try to use an OS-specific command. If that fails for whatever reason: we can walk through the directory and delete them one-at-a-time, which is what Java is doing by default.
  • We can clean up previous sessions. Whether it's because the JVM crashed or because lightning struck: there's a self-contained folder that we know represents an old session. This means if we littered the computer with 1 GB of extra data: the next time our app is launched we'll clean up after ourselves. (This assumes our app is launched again. If our app is not launched again... then nobody will ever clean up that old data. Anyone have any suggestions on how to get around that?)

It's also possible that a user may launch the app twice simultaneously. What happens then? If we're not careful then the sequence of events might go like this:
Process A: create tmpDir/com.foo.Mulligan/session123/
Process B: create tmpDir/com.foo.Mulligan/session456/
Process B: delete all other sessions, including tmpDir/com.foo.Mulligan/session123/

This will damage Process A, so we need to avoid this scenario. The simplest model I know to approach this is with an open FileLock. When we create our session's folder: create a small empty file with an active lock. Any time the TempFileManager identifies a foreign session's folder: it first checks to see if it can delete that locked file. If it can: then we assume that foreign session is invalid and we can delete the folder. If it can't: then we leave the rest of the folder alone.

Conclusion

Over a decade when I first started development, I first assumed temporary files were not really my problem. It helped that I primarily used Macs, which often clean up after themselves. Windows computers are not so helpful (and I don't know what Linux flavors do these days). I made a token effort to clean up temp files (using File.deleteOnExit()), and thought that was "good enough".

But now I've come to see orphaned files as a pretty serious offense. It's like littering on a highway, except there are no cleanup crews. Some users may be savvy enough to clean up their temp folder now and then, but many won't. And if word comes back from their IT team that your app was the reason they had to put in a help desk ticket: you just lost some good will with your customer.

These changes are relatively simple, but (if used correctly) can offer a significant improvement for customers in the unfortunate event that temp files end up orphaned over time.

As always: if anyone has any other related reading or alternative suggestions I'd love to hear them.

Savvy readers may note that the java code I referenced above is a new repository I haven't referenced before. And that my articles increasingly skew towards topics that aren't related to graphics, despite the name of this blog. Any month (year?) now I'll complete the migration from my old repo to a new one and announce some fun changes...