Package org.nuxeo.ecm.csv.core

Class CSVImporterWork

java.lang.Object

org.nuxeo.ecm.core.work.AbstractWork

org.nuxeo.ecm.core.transientstore.work.TransientStoreWork

org.nuxeo.ecm.csv.core.CSVImporterWork

All Implemented Interfaces:

Serializable, Work

public class CSVImporterWork
extends TransientStoreWork

Work task to import form a CSV file. Because the file is read from the local filesystem, this must be executed in a local queue. Since NXP-15252 the CSV reader manages "records", not "lines".

Since:

5.7

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.nuxeo.ecm.core.work.api.Work

Work.Progress, Work.State

Field Summary

Fields

Modifier and Type	Field	Description
protected static List<String>	AUTHORIZED_HEADERS	CSV headers that won't be checked if the field exists on the document type.
static String	CATEGORY_CSV_IMPORTER
protected boolean	computeTotal
static String	CONTENT_FILED_TYPE_NAME
protected long	docsCreatedCount
protected long	docsProcessedCount
protected boolean	hasTypeColumn
protected ArrayList<CSVImportLog>	importLogs
static String	LABEL_CSV_IMPORTER_CANNOT_CONVERT_FIELD_VALUE
static String	LABEL_CSV_IMPORTER_DOCUMENT_ALREADY_EXISTS
static String	LABEL_CSV_IMPORTER_DOCUMENT_CREATED
static String	LABEL_CSV_IMPORTER_DOCUMENT_UPDATED
static String	LABEL_CSV_IMPORTER_EMPTY_FILE
static String	LABEL_CSV_IMPORTER_EMPTY_LINE
static String	LABEL_CSV_IMPORTER_ERROR_DURING_IMPORT
static String	LABEL_CSV_IMPORTER_ERROR_IMPORTING_LINE
static String	LABEL_CSV_IMPORTER_MISSING_NAME_COLUMN
static String	LABEL_CSV_IMPORTER_MISSING_NAME_VALUE
static String	LABEL_CSV_IMPORTER_MISSING_TYPE_VALUE
static String	LABEL_CSV_IMPORTER_NOT_ALLOWED_SUB_TYPE
static String	LABEL_CSV_IMPORTER_NOT_EXISTING_FIELD
static String	LABEL_CSV_IMPORTER_NOT_EXISTING_FILE
static String	LABEL_CSV_IMPORTER_NOT_EXISTING_TYPE
static String	LABEL_CSV_IMPORTER_PARENT_DOES_NOT_EXIST
static String	LABEL_CSV_IMPORTER_UNABLE_TO_CREATE
static String	LABEL_CSV_IMPORTER_UNABLE_TO_SAVE
static String	LABEL_CSV_IMPORTER_UNABLE_TO_UPDATE
static String	NUXEO_CSV_BLOBS_FOLDER
static String	NUXEO_CSV_MAIL_TO
protected CSVImporterOptions	options
protected String	parentPath
protected Date	startDate
protected long	total
protected String	username

Fields inherited from class org.nuxeo.ecm.core.transientstore.work.TransientStoreWork

entryKey, KEY_SUFFIX, STORE_NAME

Fields inherited from class org.nuxeo.ecm.core.work.AbstractWork

callerThread, completionTime, docId, docIds, FAILURE_EXCEPTION, FAILURE_MSG, GLOBAL_DLQ_COUNT_REGISTRY_NAME, id, isTree, loginContext, originatingUsername, progress, RANDOM, repositoryName, schedulePath, schedulingTime, session, startTime, state, status, suspended, suspending, traceContext, WORK_FAILED_EVENT, WORK_INSTANCE

Constructor Summary

Constructors

Constructor	Description
CSVImporterWork(String id)
CSVImporterWork(String repositoryName, String parentPath, String username, Blob csvBlob, CSVImporterOptions options)

Method Summary

Modifier and Type	Method	Description
protected StringList	buildRecipientsList(String userEmail)
void	cleanUp(boolean ok, Exception e)	This method is called after AbstractWork.work() is done in a finally block, whether work completed normally or was in error or was interrupted.
protected Map<String,Serializable>	computePropertiesMap(org.apache.commons.csv.CSVRecord record, CompositeType compositeType, Map<String,Integer> header)
protected Serializable	convertValue(CompositeType compositeType, String fieldName, String headerValue, String stringValue, long lineNumber)
protected Blob	createBlobFromFilePath(String fileRelativePath)	Creates a Blob from a relative file path.
protected Blob	createBlobFromStringBlob(Blob stringBlob)	Creates a Blob from a StringBlob.
protected boolean	createDocument(long lineNumber, String newParentPath, String name, String type, Map<String,Serializable> properties)
protected void	doImport(org.apache.commons.csv.CSVParser parser)
String	getCategory()	Gets the category for this work.
protected String	getDocumentUrl(DocumentModel doc)
List<CSVImportLog>	getImportLogs()
protected long	getLineNumber(org.apache.commons.csv.CSVParser parser)
protected long	getLineNumber(org.apache.commons.csv.CSVRecord record)
String	getTitle()	Gets a human-readable name for this work instance.
protected String	getUserUrl()
protected boolean	importRecord(org.apache.commons.csv.CSVRecord record, Map<String,Integer> header)	Import a line from the CSV file.
protected void	logError(long lineNumber, String message, String localizedMessage, String... params)
protected BufferedReader	newReader(Blob blob)
protected void	replaceBlobs(List<Object> list)	Recursively replaces all Blobs with Blobs created from Files stored in the folder registered by the nuxeo.csv.blobs.folder property.
protected void	replaceBlobs(Map<String,Object> map)	Recursively replaces all Blobs with Blobs created from Files stored in the folder registered by the nuxeo.csv.blobs.folder property.
protected void	sendMail()
static Throwable	unwrapException(Throwable t)
protected boolean	updateDocument(long lineNumber, DocumentRef docRef, Map<String,Serializable> properties)
void	work()	This method should implement the actual work done by the Work instance.

Methods inherited from class org.nuxeo.ecm.core.transientstore.work.TransientStoreWork

computeEntryKey, computeEntryKey, containsBlobHolder, getBlobHolder, getEntryKey, getStore, isIdempotent, putBlobHolder, putBlobHolder, removeBlobHolder

Methods inherited from class org.nuxeo.ecm.core.work.AbstractWork

appendWorkToDeadLetterQueue, buildWorkFailureEventProps, closeSession, commitOrRollbackTransaction, equals, getCompletionTime, getDocument, getDocuments, getId, getOriginatingUsername, getPartitionKey, getProgress, getRetryCount, getSchedulePath, getSchedulingTime, getSpanFromContext, getStartTime, getStatus, getWorkInstanceState, hashCode, initSession, initSession, isDocumentTree, isSuspending, isWorkInstanceSuspended, newDocumentLocation, openSystemSession, openUserSession, run, runWorkWithTransaction, setCompletionTime, setDocument, setDocument, setDocuments, setOriginatingUsername, setProgress, setSchedulePath, setStartTime, setStatus, setWorkInstanceState, setWorkInstanceSuspending, startTransaction, suspended, toString, workFailed

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.nuxeo.ecm.core.work.api.Work

isCoalescing, isGroupJoin, onGroupJoinCompletion

Field Detail

NUXEO_CSV_MAIL_TO

public static final String NUXEO_CSV_MAIL_TO

See Also:

Constant Field Values

LABEL_CSV_IMPORTER_NOT_EXISTING_FIELD

public static final String LABEL_CSV_IMPORTER_NOT_EXISTING_FIELD

See Also:

Constant Field Values

LABEL_CSV_IMPORTER_CANNOT_CONVERT_FIELD_VALUE

public static final String LABEL_CSV_IMPORTER_CANNOT_CONVERT_FIELD_VALUE

See Also:

Constant Field Values

LABEL_CSV_IMPORTER_NOT_EXISTING_FILE

public static final String LABEL_CSV_IMPORTER_NOT_EXISTING_FILE

See Also:

Constant Field Values

NUXEO_CSV_BLOBS_FOLDER

public static final String NUXEO_CSV_BLOBS_FOLDER

See Also:

Constant Field Values

LABEL_CSV_IMPORTER_DOCUMENT_ALREADY_EXISTS

public static final String LABEL_CSV_IMPORTER_DOCUMENT_ALREADY_EXISTS

See Also:

Constant Field Values

LABEL_CSV_IMPORTER_UNABLE_TO_UPDATE

public static final String LABEL_CSV_IMPORTER_UNABLE_TO_UPDATE

See Also:

Constant Field Values

LABEL_CSV_IMPORTER_DOCUMENT_UPDATED

public static final String LABEL_CSV_IMPORTER_DOCUMENT_UPDATED

See Also:

Constant Field Values

LABEL_CSV_IMPORTER_UNABLE_TO_CREATE

public static final String LABEL_CSV_IMPORTER_UNABLE_TO_CREATE

See Also:

Constant Field Values

LABEL_CSV_IMPORTER_PARENT_DOES_NOT_EXIST

public static final String LABEL_CSV_IMPORTER_PARENT_DOES_NOT_EXIST

See Also:

Constant Field Values

LABEL_CSV_IMPORTER_DOCUMENT_CREATED

public static final String LABEL_CSV_IMPORTER_DOCUMENT_CREATED

See Also:

Constant Field Values

LABEL_CSV_IMPORTER_NOT_ALLOWED_SUB_TYPE

public static final String LABEL_CSV_IMPORTER_NOT_ALLOWED_SUB_TYPE

See Also:

Constant Field Values

LABEL_CSV_IMPORTER_UNABLE_TO_SAVE

public static final String LABEL_CSV_IMPORTER_UNABLE_TO_SAVE

See Also:

Constant Field Values

LABEL_CSV_IMPORTER_ERROR_IMPORTING_LINE

public static final String LABEL_CSV_IMPORTER_ERROR_IMPORTING_LINE

See Also:

Constant Field Values

LABEL_CSV_IMPORTER_NOT_EXISTING_TYPE

public static final String LABEL_CSV_IMPORTER_NOT_EXISTING_TYPE

See Also:

Constant Field Values

LABEL_CSV_IMPORTER_MISSING_TYPE_VALUE

public static final String LABEL_CSV_IMPORTER_MISSING_TYPE_VALUE

See Also:

Constant Field Values

LABEL_CSV_IMPORTER_MISSING_NAME_VALUE

public static final String LABEL_CSV_IMPORTER_MISSING_NAME_VALUE

See Also:

Constant Field Values

LABEL_CSV_IMPORTER_MISSING_NAME_COLUMN

public static final String LABEL_CSV_IMPORTER_MISSING_NAME_COLUMN

See Also:

Constant Field Values

LABEL_CSV_IMPORTER_EMPTY_FILE

public static final String LABEL_CSV_IMPORTER_EMPTY_FILE

See Also:

Constant Field Values

LABEL_CSV_IMPORTER_ERROR_DURING_IMPORT

public static final String LABEL_CSV_IMPORTER_ERROR_DURING_IMPORT

See Also:

Constant Field Values

LABEL_CSV_IMPORTER_EMPTY_LINE

public static final String LABEL_CSV_IMPORTER_EMPTY_LINE

See Also:

Constant Field Values

CATEGORY_CSV_IMPORTER

public static final String CATEGORY_CSV_IMPORTER

See Also:

Constant Field Values

CONTENT_FILED_TYPE_NAME

public static final String CONTENT_FILED_TYPE_NAME

See Also:

Constant Field Values

AUTHORIZED_HEADERS

protected static final List<String> AUTHORIZED_HEADERS

CSV headers that won't be checked if the field exists on the document type.

Since:

7.3

parentPath

protected String parentPath

username

protected String username

options

protected CSVImporterOptions options

hasTypeColumn

protected boolean hasTypeColumn

startDate

protected Date startDate

importLogs

protected ArrayList<CSVImportLog> importLogs

computeTotal

protected boolean computeTotal

total

protected long total

docsProcessedCount

protected long docsProcessedCount

Since:

11.1

docsCreatedCount

protected long docsCreatedCount

Constructor Detail

CSVImporterWork

public CSVImporterWork(String id)

CSVImporterWork

public CSVImporterWork(String repositoryName,
                       String parentPath,
                       String username,
                       Blob csvBlob,
                       CSVImporterOptions options)

Method Detail

getCategory

public String getCategory()

Description copied from interface: Work

Gets the category for this work.

Used to choose an execution queue.

Specified by:

getCategory in interface Work

Overrides:

getCategory in class AbstractWork

Returns:

the category, or null for the default

getTitle

public String getTitle()

Description copied from interface: Work

Gets a human-readable name for this work instance.

Returns:

a human-readable name

getImportLogs

public List<CSVImportLog> getImportLogs()

work

public void work()

Description copied from interface: Work

This method should implement the actual work done by the Work instance.

It should periodically update its progress through Work.setProgress(org.nuxeo.ecm.core.work.api.Work.Progress).

To allow for suspension by the WorkManager, it should periodically call Work.isSuspending(), and if true call Work.suspended() return early with saved state data.

Clean up can by implemented by Work.cleanUp(boolean, Exception).

Specified by:

work in interface Work

Specified by:

work in class AbstractWork

See Also:

Work.isSuspending(), Work.suspended(), Work.cleanUp(boolean, java.lang.Exception)

cleanUp

public void cleanUp(boolean ok,
                    Exception e)

Description copied from class: AbstractWork

This method is called after AbstractWork.work() is done in a finally block, whether work completed normally or was in error or was interrupted.

Specified by:

cleanUp in interface Work

Overrides:

cleanUp in class AbstractWork

Parameters:

ok - true if the work completed normally

e - the exception, if available

newReader

protected BufferedReader newReader(Blob blob)
                            throws IOException

Throws:

IOException

Since:

7.3

doImport

protected void doImport(org.apache.commons.csv.CSVParser parser)

importRecord

protected boolean importRecord(org.apache.commons.csv.CSVRecord record,
                               Map<String,Integer> header)

Import a line from the CSV file.

Returns:

true if a document has been created or updated, false otherwise.

Since:

6.0

getLineNumber

protected long getLineNumber(org.apache.commons.csv.CSVRecord record)

getLineNumber

protected long getLineNumber(org.apache.commons.csv.CSVParser parser)

computePropertiesMap

protected Map<String,Serializable> computePropertiesMap(org.apache.commons.csv.CSVRecord record,
                                                              CompositeType compositeType,
                                                              Map<String,Integer> header)

Since:

6.0

convertValue

protected Serializable convertValue(CompositeType compositeType,
                                    String fieldName,
                                    String headerValue,
                                    String stringValue,
                                    long lineNumber)

createBlobFromFilePath

protected Blob createBlobFromFilePath(String fileRelativePath)
                               throws IOException

Creates a Blob from a relative file path. The File will be looked up in the folder registered by the nuxeo.csv.blobs.folder property.

Throws:

IOException

Since:

9.3

createBlobFromStringBlob

protected Blob createBlobFromStringBlob(Blob stringBlob)
                                 throws IOException

Creates a Blob from a StringBlob. Assume that the StringBlob content is the relative file path. The File will be looked up in the folder registered by the nuxeo.csv.blobs.folder property.

Throws:

IOException

Since:

9.3

replaceBlobs

protected void replaceBlobs(Map<String,Object> map)
                     throws IOException

Recursively replaces all Blobs with Blobs created from Files stored in the folder registered by the nuxeo.csv.blobs.folder property.

Throws:

IOException

Since:

9.3

replaceBlobs

protected void replaceBlobs(List<Object> list)
                     throws IOException

Recursively replaces all Blobs with Blobs created from Files stored in the folder registered by the nuxeo.csv.blobs.folder property.

Throws:

IOException

Since:

9.3

createDocument

protected boolean createDocument(long lineNumber,
                                 String newParentPath,
                                 String name,
                                 String type,
                                 Map<String,Serializable> properties)

updateDocument

protected boolean updateDocument(long lineNumber,
                                 DocumentRef docRef,
                                 Map<String,Serializable> properties)

logError

protected void logError(long lineNumber,
                        String message,
                        String localizedMessage,
                        String... params)

sendMail

protected void sendMail()

getDocumentUrl

protected String getDocumentUrl(DocumentModel doc)

getUserUrl

protected String getUserUrl()

buildRecipientsList

protected StringList buildRecipientsList(String userEmail)

unwrapException

public static Throwable unwrapException(Throwable t)