Plume File

Plume File is a modular Java library that provides several modules to easily handle files in a project from storage to download. Its goal is to provide great performance with a great Java API: it can handle files of several gigabytes with a JVM that has an Xmx of only 128Mo.

Plume File that has some small dependencies on the Plume Framework, especially to the conf and scheduler modules.

Looking for a demo? Check out the file service implementation on the Plume Demo project.

Architecture

Plume file library follows a modular architecture to make it easy to change the storage engine or the API technology.

Some necessary file storage choices has been taken for this library:

• The storage of the files binary data is separated from the storage of files metadata
• Files are uniquely identified by an UUID where the file extension is appended. So for example, the file cat.jpg might be stored in the file storage as the file 35a0da5e-1036-43ae-ae04-8c4076f5a9b3.jpg. The metadata will then enable to retrieve the original name.

To categorize file, each file is associated with a FileType. This consists of a user defined enum with values like LANDSCAPES or PROFILE_PICTURE. This category is especially used by the plume-file-metadata-database to provide a convenient way to delete files.

Getting started

The main composant is the module plume-file-core which exposes file storage interfaces, FileMetadataService and FileStorageService that must be implemented. So to use the main entry point FileService:

• The module plume-file-core must be installed
• One of the metadata modules must be installed
• One of the file storage modules must be installed

Jersey connectors are provided to upload/expose files through API.

File URL

Some modules implement the FileUrlService to be able to retrieve the full URL of a file stored using Plume File. That is the case of:

• plume-file-web-download-jersey
• plume-file-signed-url
• In the future, the AWS S3 file storage module

Plume file modules

These are modules to be used with the main plume-file-core module.

Storage modules

The storage modules implements FileStorageService:

• plume-file-storage-database: store uploaded file in a database
• plume-file-storage-system: store uploaded file on a system disk
• plume-file-storage-gcp: store uploaded file on a GCP Cloud Storage bucket

Metadata module

The metadata module implements FileMetadataService:

• plume-file-metadata-database: store file metadata in a database

Web-service modules

Those modules provide API connectors to serve and upload files.

• plume-file-web-download-jersey: serve stored file through a Jersey based single web-service
• plume-file-web-download-jersey-common: common module that shares the download logic (services, configuration and helpers) between the plain and signed download modules. It exposes no public @Path resource to avoid path collisions and is pulled in transitively by the two download modules — you normally do not declare it directly.
• plume-file-web-signed-download-jersey: serve temporary, cryptographically signed files using a Jersey based web-service. Note: this module is designed to be a mutually exclusive replacement for plume-file-web-download-jersey to prevent bypass of signature verification.
• plume-file-web-upload-jersey: upload file using HTTP multipart
• plume-file-signed-url: pure Java service module to generate and validate HMAC-signed file URLs (does not require Jersey)

Serving files (download) — installation

Plume File offers two mutually exclusive ways to expose your files over HTTP. Install one of the two modules below (never both, otherwise you expose a non-signed endpoint next to a signed one and attackers can bypass the signature checks).

Plain (non-signed) download

• Maven module: plume-file-web-download-jersey
• Guice modules (pick one for the caching strategy):
  • GuiceFileDownloadModule — no in-memory cache
  • GuiceFileCacheDownloadModule — in-memory cache (Guava)
• Register the resource: config.resource(FileWs.class);

install(new GuiceFileDownloadModule());      // or GuiceFileCacheDownloadModule
// ...
config.resource(FileWs.class);

Signed download

• Maven module: plume-file-web-signed-download-jersey
• Guice module: GuiceSignedFileDownloadModule — binds only the signing service (GuiceSignedFileUrlModule); it does not choose a download implementation for you.
• You must also explicitly pick the download implementation from plume-file-web-download-jersey-common (choose one):
  • GuiceFileDownloadModule — no in-memory cache
  • GuiceFileCacheDownloadModule — in-memory cache (Guava)
• Register the resource: config.resource(SignedFileWs.class);

install(new GuiceSignedFileDownloadModule());   // signing service
install(new GuiceFileDownloadModule());         // or GuiceFileCacheDownloadModule
// ...
config.resource(SignedFileWs.class);

In both cases the plume-file-web-download-jersey-common module is a transitive dependency; you do not install it directly. Only add it as a direct dependency if you write a custom Guice module (for example to plug your own FileCacheService implementation).

Configuration and default values

plume-file-core

# The time of day at which the cleaning task is scheduled
file.cleaning-hour = "03:00"
# The algorithm used for checksum file calculation
file.checksum-algorithm = "SHA-256"

plume-file-storage-system

# The directory in which the files are stored locally
file.storage.local-path = "plume-file-data/"

plume-file-web-download-jersey

# The Cache-Control response header value for max-age, it represents the time files will be stored in cache on browsers
file.cache.http.max-age = "365 days"
# Internal server file content configuration: the duration after which a file will be removed from the cache
file.cache.data.expires-after-access-duration = "1 day"
# Internal server file content configuration: the maximum size of the cache
file.cache.data.max-cache-size = "64 MB"
# Internal server file metadata configuration: the duration after which metadata will be removed from the cache
file.cache.metadata.expires-after-access-duration = "1 day"
# Internal server file metadata configuration: the maximum number of metadata elements stored in the cache
file.cache.metadata.max-elements = 10000
# The base file URL used by FileUrlService.url(). If the config value application.base-url is defined, this can be replaced by file.url.base-path = ${application.base-url}"/api/files"
file.url.base-path = "/api/files"

plume-file-signed-url

# Secret key for HMAC-SHA256 signature. REQUIRED, no usable default
file.signed-url.secret-key = null
# Default duration for which the signed URL is valid
file.signed-url.default-validity-duration = "1 hour"
# Base path for the signed download URLs
file.signed-url.base-path = "/api/files"

plume-file-web-signed-download-jersey

# The Cache-Control response header value for max-age, it represents the time files will be stored in cache on browsers
file.cache.http.max-age = "365 days"

Migrating guides

All migration guides are available in the Plume File Github releases.