Class KeyValueBlobTransientStore
- java.lang.Object
-
- org.nuxeo.ecm.core.transientstore.keyvalueblob.KeyValueBlobTransientStore
-
- All Implemented Interfaces:
TransientStore,TransientStoreProvider
public class KeyValueBlobTransientStore extends Object implements TransientStoreProvider
Transient Store storing properties in a Key/Value store, and storing blobs using a Blob Provider.This transient store is configured with the following properties:
- keyValueStore: the name of the key/value store to use. If not provided, it defaults to "transient_" + the transient store name.
- blobProvider: the name of the blob provider to use. If not provided, it defaults to "transient_" + the transient store name.
- defaultBlobProvider: if the configured or defaulted blob provider doesn't exist, a namespaced copy of this one will be used instead. The default is "default".
The storage format is the following:
__blobsize__: storage size; because entries may expire without us being notified due to their TTL, this may be higher than the actual storage size entryKey.completed: "true" if completed, "false" if not; presence of this key marks entry existence entryKey.paraminfo: ["foo", "bar"] entryKey.param.foo: value for param foo entryKey.param.foo__format: "java" for java serializable format, otherwise string entryKey.param.bar: value for param bar etc. entryKey.bloblock: "true" if there is a blob read/write in progress, null otherwise entryKey.blobinfo: {"count": number of blobs, "size": storage size of the blobs} entryKey.blob.0: {"key": key in blob provider for first blob, "mimetype": MIME Type, "encoding": encoding, "filename": filename, "digest": digest} entryKey.blob.1: {...} same for second blob etc.- Since:
- 9.3
-
-
Field Summary
Fields Modifier and Type Field Description protected longabsoluteMaxSizeprotected static intBLOB_LOCK_TTLprotected StringblobProviderIdstatic StringCONFIG_BLOB_PROVIDERstatic StringCONFIG_DEFAULT_BLOB_PROVIDERstatic StringCONFIG_DEFAULT_BLOB_PROVIDER_DEFAULTstatic StringCONFIG_KEY_VALUE_STOREstatic StringCOUNTprotected StringdefaultBlobProviderIdstatic StringDIGESTstatic StringDOT_BLOB_DOTstatic StringDOT_BLOBINFOstatic StringDOT_BLOBLOCKstatic StringDOT_COMPLETEDstatic StringDOT_PARAM_DOTstatic StringDOT_PARAMINFOstatic StringENCODINGstatic StringFILENAMEstatic StringFORMATstatic StringFORMAT_JAVAstatic StringKEYprotected StringkeyValueStoreNamestatic StringLENGTHprotected static com.fasterxml.jackson.core.type.TypeReference<List<String>>LIST_STRINGprotected static longLOCK_ACQUIRE_TIME_NANOSprotected static longLOCK_EXPONENTIAL_BACKOFF_AFTER_NANOSprotected static com.fasterxml.jackson.core.type.TypeReference<Map<String,String>>MAP_STRING_STRINGprotected com.fasterxml.jackson.databind.ObjectMappermapperstatic StringMIMETYPEprotected Stringnameprotected intreleaseTTLTTL used to keep objects around a bit longer if there's space for them, for caching.static StringSEPstatic StringSIZEstatic StringSTORAGE_SIZEprotected longtargetMaxSizeprotected intttlBasic TTL for all entries.
-
Constructor Summary
Constructors Constructor Description KeyValueBlobTransientStore()
-
Method Summary
All Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description protected voidacquireBlobLockOrThrow(String key, KeyValueStore kvs)protected booleanacquireLock(BooleanSupplier tryAcquireOnce)protected voidaddStorageSize(long delta)Deprecated.since 11.1protected voidaddStorageSize(long delta, KeyValueStore kvs)voidatomicUpdate(String key, Function<String,String> updateFunction, long ttl)Deprecated.since 11.1protected voidatomicUpdate(String key, Function<String,String> updateFunction, long ttl, KeyValueStore kvs)protected voidcomputeStorageSize()Computes an exact value for the current storage size (sum of all blobs size).voiddoGC()Runs garbage collecting to delete the file system resources that are associated with entries that were removed.booleanexists(String key)Returnstrueif an entry exists with the givenkey.protected List<String>getBlobKeys(String key)protected BlobProvidergetBlobProvider()List<Blob>getBlobs(String key)Gets the blobs associated with the entry with the givenkey.protected KeyValueStoregetKeyValueStore()SerializablegetParameter(String key, String parameter)Gets the value ofparameterin the entry with the givenkey.Map<String,Serializable>getParameters(String key)Gets values of the parameters in the entry with the givenkey.longgetSize(String key)Returns the size of the blobs associated with the entry with the givenkeyor-1if entry does not exist.longgetStorageSize()Returns the size (in bytes) of the disk storage used for blobs.voidinit(TransientStoreConfig config)Initializes the store from the givenconfig.booleanisCompleted(String key)Returnstrueif the entry with the givenkeyis ready.protected List<String>jsonToList(String json)protected Map<String,String>jsonToMap(String json)Stream<String>keyStream()Returns aStreamof keys for all entries.protected voidmarkEntryExists(String key)Deprecated.since 11.1protected voidmarkEntryExists(String key, KeyValueStore kvs)voidputBlobs(String key, List<Blob> blobs)Associates the givenblobswith the entry with the givenkey.voidputParameter(String key, String parameter, Serializable value)Setsparametertovaluein the entry with the givenkey.voidputParameters(String key, Map<String,Serializable> parameters)Putsparametersin the entry with the givenkey.voidrelease(String key)Informs the store that the entry with the givenkeycan be released if TTL or GC parameters require to do some cleanup.protected voidreleaseBlobLock(String key, KeyValueStore kvs)voidremove(String key)Removes entry with the givenkey.voidremoveAll()Removes all entries from the store.protected voidremoveBlobs(String key)Deprecated.since 11.1protected voidremoveBlobs(String key, KeyValueStore kvs)protected voidremoveCompleted(String key)Deprecated.since 11.1protected voidremoveCompleted(String key, KeyValueStore kvs)protected voidremoveParameters(String key)Deprecated.since 11.1protected voidremoveParameters(String key, KeyValueStore kvs)voidsetCompleted(String key, boolean completed)Marks the entry with the givenkeyas ready.protected voidsetReleaseTTL(String key)voidshutdown()Shuts down the store.protected StringtoJson(Object object)protected booleantryAcquireBlobLock(String key, KeyValueStore kvs)protected booleantryAcquireOnceBlobLock(String key, KeyValueStore kvs)-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.nuxeo.ecm.core.transientstore.api.TransientStoreProvider
getStorageSizeMB, keySet
-
-
-
-
Field Detail
-
SEP
public static final String SEP
- See Also:
- Constant Field Values
-
STORAGE_SIZE
public static final String STORAGE_SIZE
- See Also:
- Constant Field Values
-
DOT_COMPLETED
public static final String DOT_COMPLETED
- See Also:
- Constant Field Values
-
DOT_PARAMINFO
public static final String DOT_PARAMINFO
- See Also:
- Constant Field Values
-
DOT_PARAM_DOT
public static final String DOT_PARAM_DOT
- See Also:
- Constant Field Values
-
FORMAT
public static final String FORMAT
- See Also:
- Constant Field Values
-
FORMAT_JAVA
public static final String FORMAT_JAVA
- See Also:
- Constant Field Values
-
DOT_BLOBLOCK
public static final String DOT_BLOBLOCK
- Since:
- 11.1
- See Also:
- Constant Field Values
-
DOT_BLOBINFO
public static final String DOT_BLOBINFO
- See Also:
- Constant Field Values
-
COUNT
public static final String COUNT
- See Also:
- Constant Field Values
-
SIZE
public static final String SIZE
- See Also:
- Constant Field Values
-
DOT_BLOB_DOT
public static final String DOT_BLOB_DOT
- See Also:
- Constant Field Values
-
KEY
public static final String KEY
- See Also:
- Constant Field Values
-
MIMETYPE
public static final String MIMETYPE
- See Also:
- Constant Field Values
-
ENCODING
public static final String ENCODING
- See Also:
- Constant Field Values
-
FILENAME
public static final String FILENAME
- See Also:
- Constant Field Values
-
LENGTH
public static final String LENGTH
- See Also:
- Constant Field Values
-
DIGEST
public static final String DIGEST
- See Also:
- Constant Field Values
-
CONFIG_KEY_VALUE_STORE
public static final String CONFIG_KEY_VALUE_STORE
- See Also:
- Constant Field Values
-
CONFIG_BLOB_PROVIDER
public static final String CONFIG_BLOB_PROVIDER
- See Also:
- Constant Field Values
-
CONFIG_DEFAULT_BLOB_PROVIDER
public static final String CONFIG_DEFAULT_BLOB_PROVIDER
- Since:
- 11.1
- See Also:
- Constant Field Values
-
CONFIG_DEFAULT_BLOB_PROVIDER_DEFAULT
public static final String CONFIG_DEFAULT_BLOB_PROVIDER_DEFAULT
- Since:
- 11.1
- See Also:
- Constant Field Values
-
BLOB_LOCK_TTL
protected static final int BLOB_LOCK_TTL
- Since:
- 11.1
- See Also:
- Constant Field Values
-
LOCK_ACQUIRE_TIME_NANOS
protected static final long LOCK_ACQUIRE_TIME_NANOS
- Since:
- 11.1
-
LOCK_EXPONENTIAL_BACKOFF_AFTER_NANOS
protected static final long LOCK_EXPONENTIAL_BACKOFF_AFTER_NANOS
- Since:
- 11.1
-
keyValueStoreName
protected String keyValueStoreName
-
blobProviderId
protected String blobProviderId
-
defaultBlobProviderId
protected String defaultBlobProviderId
-
ttl
protected int ttl
Basic TTL for all entries.
-
releaseTTL
protected int releaseTTL
TTL used to keep objects around a bit longer if there's space for them, for caching.
-
targetMaxSize
protected long targetMaxSize
-
absoluteMaxSize
protected long absoluteMaxSize
-
mapper
protected com.fasterxml.jackson.databind.ObjectMapper mapper
-
LIST_STRING
protected static final com.fasterxml.jackson.core.type.TypeReference<List<String>> LIST_STRING
-
MAP_STRING_STRING
protected static final com.fasterxml.jackson.core.type.TypeReference<Map<String,String>> MAP_STRING_STRING
-
-
Constructor Detail
-
KeyValueBlobTransientStore
public KeyValueBlobTransientStore()
-
-
Method Detail
-
init
public void init(TransientStoreConfig config)
Description copied from interface:TransientStoreProviderInitializes the store from the givenconfig.- Specified by:
initin interfaceTransientStoreProvider
-
getKeyValueStore
protected KeyValueStore getKeyValueStore()
-
getBlobProvider
protected BlobProvider getBlobProvider()
-
shutdown
public void shutdown()
Description copied from interface:TransientStoreProviderShuts down the store.- Specified by:
shutdownin interfaceTransientStoreProvider
-
keyStream
public Stream<String> keyStream()
Description copied from interface:TransientStoreProviderReturns aStreamof keys for all entries.- Specified by:
keyStreamin interfaceTransientStoreProvider
-
getStorageSize
public long getStorageSize()
Description copied from interface:TransientStoreProviderReturns the size (in bytes) of the disk storage used for blobs.- Specified by:
getStorageSizein interfaceTransientStoreProvider- Returns:
- the number of bytes used by stored blobs
-
addStorageSize
@Deprecated protected void addStorageSize(long delta)
Deprecated.since 11.1
-
addStorageSize
protected void addStorageSize(long delta, KeyValueStore kvs)
-
computeStorageSize
protected void computeStorageSize()
Computes an exact value for the current storage size (sum of all blobs size). THIS METHOD IS COSTLY.Does not take into account blob de-duplication that may be done by the blob provider.
Does not take into account blobs that still exist in the blob provider but are not referenced anymore (due to TTL expiration or GC not having been done).
-
doGC
public void doGC()
Description copied from interface:TransientStoreProviderRuns garbage collecting to delete the file system resources that are associated with entries that were removed.- Specified by:
doGCin interfaceTransientStoreProvider
-
removeAll
public void removeAll()
Description copied from interface:TransientStoreProviderRemoves all entries from the store.- Specified by:
removeAllin interfaceTransientStoreProvider
-
jsonToList
protected List<String> jsonToList(String json)
-
atomicUpdate
@Deprecated public void atomicUpdate(String key, Function<String,String> updateFunction, long ttl)
Deprecated.since 11.1
-
atomicUpdate
protected void atomicUpdate(String key, Function<String,String> updateFunction, long ttl, KeyValueStore kvs)
-
exists
public boolean exists(String key)
Description copied from interface:TransientStoreReturnstrueif an entry exists with the givenkey.- Specified by:
existsin interfaceTransientStore
-
markEntryExists
@Deprecated protected void markEntryExists(String key)
Deprecated.since 11.1
-
markEntryExists
protected void markEntryExists(String key, KeyValueStore kvs)
-
putParameter
public void putParameter(String key, String parameter, Serializable value)
Description copied from interface:TransientStoreSetsparametertovaluein the entry with the givenkey.If entry does not exist a new entry is created. If
parameteralready exists in the entry it is overwritten.- Specified by:
putParameterin interfaceTransientStore
-
getParameter
public Serializable getParameter(String key, String parameter)
Description copied from interface:TransientStoreGets the value ofparameterin the entry with the givenkey.Returns
nullif entry or parameter does not exist.- Specified by:
getParameterin interfaceTransientStore
-
putParameters
public void putParameters(String key, Map<String,Serializable> parameters)
Description copied from interface:TransientStorePutsparametersin the entry with the givenkey. Overwrites any existing parameter in the entry.If entry does not exist a new entry is created.
- Specified by:
putParametersin interfaceTransientStore
-
getParameters
public Map<String,Serializable> getParameters(String key)
Description copied from interface:TransientStoreGets values of the parameters in the entry with the givenkey.Returns
nullif entry does not exist.- Specified by:
getParametersin interfaceTransientStore
-
removeParameters
@Deprecated protected void removeParameters(String key)
Deprecated.since 11.1
-
removeParameters
protected void removeParameters(String key, KeyValueStore kvs)
-
putBlobs
public void putBlobs(String key, List<Blob> blobs)
Description copied from interface:TransientStoreAssociates the givenblobswith the entry with the givenkey.If entry does not exist a new entry is created.
- Specified by:
putBlobsin interfaceTransientStore
-
removeBlobs
@Deprecated protected void removeBlobs(String key)
Deprecated.since 11.1
-
removeBlobs
protected void removeBlobs(String key, KeyValueStore kvs)
-
getBlobs
public List<Blob> getBlobs(String key)
Description copied from interface:TransientStoreGets the blobs associated with the entry with the givenkey.Returns
nullif entry does not exist.- Specified by:
getBlobsin interfaceTransientStore
-
getBlobKeys
protected List<String> getBlobKeys(String key)
-
acquireBlobLockOrThrow
protected void acquireBlobLockOrThrow(String key, KeyValueStore kvs)
-
tryAcquireBlobLock
protected boolean tryAcquireBlobLock(String key, KeyValueStore kvs)
-
tryAcquireOnceBlobLock
protected boolean tryAcquireOnceBlobLock(String key, KeyValueStore kvs)
-
releaseBlobLock
protected void releaseBlobLock(String key, KeyValueStore kvs)
-
acquireLock
protected boolean acquireLock(BooleanSupplier tryAcquireOnce)
-
getSize
public long getSize(String key)
Description copied from interface:TransientStoreReturns the size of the blobs associated with the entry with the givenkeyor-1if entry does not exist.- Specified by:
getSizein interfaceTransientStore
-
isCompleted
public boolean isCompleted(String key)
Description copied from interface:TransientStoreReturnstrueif the entry with the givenkeyis ready.- Specified by:
isCompletedin interfaceTransientStore
-
setCompleted
public void setCompleted(String key, boolean completed)
Description copied from interface:TransientStoreMarks the entry with the givenkeyas ready.If entry does not exist a new entry is created.
- Specified by:
setCompletedin interfaceTransientStore
-
removeCompleted
@Deprecated protected void removeCompleted(String key)
Deprecated.since 11.1
-
removeCompleted
protected void removeCompleted(String key, KeyValueStore kvs)
-
release
public void release(String key)
Description copied from interface:TransientStoreInforms the store that the entry with the givenkeycan be released if TTL or GC parameters require to do some cleanup.Has no effect if entry does not exist.
- Specified by:
releasein interfaceTransientStore
-
setReleaseTTL
protected void setReleaseTTL(String key)
-
remove
public void remove(String key)
Description copied from interface:TransientStoreRemoves entry with the givenkey.Has no effect if entry does not exist.
- Specified by:
removein interfaceTransientStore
-
-