org.nuxeo.lib.stream.log

Interface LogManager

All Superinterfaces:

AutoCloseable

All Known Implementing Classes:

AbstractLogManager, ChronicleLogManager, KafkaLogManager

public interface LogManager
extends AutoCloseable

Manage Log and give access to Appenders and Tailers. Closing the LogManager will also close all its appenders and tailers.

Since:

9.3

Method Summary

Modifier and Type	Method and Description
void	close()
boolean	createIfNotExists(String name, int size) Creates a new Log with size partitions if the Log does not exists.
default <M extends Externalizable> LogTailer<M>	createTailer(String group, Collection<LogPartition> partitions) Creates a tailer for a consumer group and assign multiple partitions.
<M extends Externalizable> LogTailer<M>	createTailer(String group, Collection<LogPartition> partitions, Codec<M> codec) Creates a tailer for a consumer group and assign multiple partitions.
default <M extends Externalizable> LogTailer<M>	createTailer(String group, LogPartition partition) Creates a tailer for a consumer group and assign a single partition.
default <M extends Externalizable> LogTailer<M>	createTailer(String group, LogPartition partition, Codec<M> codec) Creates a tailer for a consumer group and assign a single partition.
default <M extends Externalizable> LogTailer<M>	createTailer(String group, String name) Creates a tailer for a consumer group and assign all partitions of the Log.
default <M extends Externalizable> LogTailer<M>	createTailer(String group, String name, Codec<M> codec) Creates a tailer for a consumer group and assigns all partitions of the Log.
boolean	delete(String name) Tries to delete a Log.
boolean	exists(String name) Returns true if a Log with this name exists.
default <M extends Externalizable> LogAppender<M>	getAppender(String name) Gets an appender for the Log named name, uses an already defined codec or the legacy encoding to encode records.
<M extends Externalizable> LogAppender<M>	getAppender(String name, Codec<M> codec) Gets an appender for the Log named name, uses codec to encode records.
default LogLag	getLag(String name, String group) Returns the lag between consumer group and producers for a Log.
List<LogLag>	getLagPerPartition(String name, String group) Returns the lag between consumer group and the producers for each partition.
default <M extends Externalizable> Latency	getLatency(String name, String group, Codec<M> codec, Function<M,Long> timestampExtractor, Function<M,String> keyExtractor) Returns the latency between consumer group and producers for a Log.
default <M extends Externalizable> Latency	getLatency(String name, String group, Function<M,Long> timestampExtractor, Function<M,String> keyExtractor) Deprecated. 10.2 use getLatency(String, String, Codec, Function, Function) instead.
<M extends Externalizable> List<Latency>	getLatencyPerPartition(String name, String group, Codec<M> codec, Function<M,Long> timestampExtractor, Function<M,String> keyExtractor) Returns the lag with latency.
default <M extends Externalizable> List<Latency>	getLatencyPerPartition(String name, String group, Function<M,Long> timestampExtractor, Function<M,String> keyExtractor) Deprecated. 10.2 use getLatencyPerPartition(String, String, Codec, Function, Function) instead.
List<String>	listAll() Returns all the Log names.
List<String>	listConsumerGroups(String name) List the consumer groups for a Log. Note that for Kafka it returns only consumers that use the subscribe API.
int	size(String name) Returns the number of partition of a Log.
default <M extends Externalizable> LogTailer<M>	subscribe(String group, Collection<String> names, RebalanceListener listener)
<M extends Externalizable> LogTailer<M>	subscribe(String group, Collection<String> names, RebalanceListener listener, Codec<M> codec) Creates a tailer for a consumer group and subscribe to multiple Logs.
boolean	supportSubscribe() Returns true if the Log subscribe(java.lang.String, java.util.Collection<java.lang.String>, org.nuxeo.lib.stream.log.RebalanceListener, org.nuxeo.lib.stream.codec.Codec<M>) method is supported.

Method Detail

exists

boolean exists(String name)

Returns true if a Log with this name exists.

createIfNotExists

boolean createIfNotExists(String name,
                          int size)

Creates a new Log with size partitions if the Log does not exists. Returns true it the Log has been created.

delete

boolean delete(String name)

Tries to delete a Log. Returns true if successfully deleted, might not be possible depending on the implementation.

size

int size(String name)

Returns the number of partition of a Log.

Since:

10.2

getAppender

<M extends Externalizable> LogAppender<M> getAppender(String name,
                                                      Codec<M> codec)

Gets an appender for the Log named name, uses codec to encode records. An appender is thread safe.

Since:

10.2

getAppender

default <M extends Externalizable> LogAppender<M> getAppender(String name)

Gets an appender for the Log named name, uses an already defined codec or the legacy encoding to encode records. An appender is thread safe.

createTailer

<M extends Externalizable> LogTailer<M> createTailer(String group,
                                                     Collection<LogPartition> partitions,
                                                     Codec<M> codec)

Creates a tailer for a consumer group and assign multiple partitions. Uses codec to decode records. Note that partitions can be from different Logs. A tailer is NOT thread safe.

Since:

10.2

createTailer

default <M extends Externalizable> LogTailer<M> createTailer(String group,
                                                             Collection<LogPartition> partitions)

Creates a tailer for a consumer group and assign multiple partitions. Note that partitions can be from different Logs. Reads records using the legacy decoder. A tailer is NOT thread safe.

createTailer

default <M extends Externalizable> LogTailer<M> createTailer(String group,
                                                             LogPartition partition)

Creates a tailer for a consumer group and assign a single partition. Reads records using the legacy decoder. A tailer is NOT thread safe.

createTailer

default <M extends Externalizable> LogTailer<M> createTailer(String group,
                                                             String name)

Creates a tailer for a consumer group and assign all partitions of the Log. Reads records using the legacy decoder. A tailer is NOT thread safe.

createTailer

default <M extends Externalizable> LogTailer<M> createTailer(String group,
                                                             LogPartition partition,
                                                             Codec<M> codec)

Creates a tailer for a consumer group and assign a single partition. Use an explicit codec to decode records. A tailer is NOT thread safe.

Since:

10.2

createTailer

default <M extends Externalizable> LogTailer<M> createTailer(String group,
                                                             String name,
                                                             Codec<M> codec)

Creates a tailer for a consumer group and assigns all partitions of the Log. Uses codec to decode records. A tailer is NOT thread safe.

Since:

10.2

supportSubscribe

boolean supportSubscribe()

Returns true if the Log subscribe(java.lang.String, java.util.Collection<java.lang.String>, org.nuxeo.lib.stream.log.RebalanceListener, org.nuxeo.lib.stream.codec.Codec<M>) method is supported.

subscribe

<M extends Externalizable> LogTailer<M> subscribe(String group,
                                                  Collection<String> names,
                                                  RebalanceListener listener,
                                                  Codec<M> codec)

Creates a tailer for a consumer group and subscribe to multiple Logs. The partitions assignment is done dynamically depending on the number of subscribers. The partitions can change during tailers life, this is called a rebalancing. A listener can be used to be notified on assignment changes. Uses codec to decode records.

A tailer is NOT thread safe.

You should not mix createTailer(java.lang.String, java.util.Collection<org.nuxeo.lib.stream.log.LogPartition>, org.nuxeo.lib.stream.codec.Codec<M>) and subscribe usage using the same group.

subscribe

default <M extends Externalizable> LogTailer<M> subscribe(String group,
                                                          Collection<String> names,
                                                          RebalanceListener listener)

getLagPerPartition

List<LogLag> getLagPerPartition(String name,
                                String group)

Returns the lag between consumer group and the producers for each partition. The result list is ordered, for instance index 0 is lag for partition 0.

getLag

default LogLag getLag(String name,
                      String group)

Returns the lag between consumer group and producers for a Log.

getLatencyPerPartition

<M extends Externalizable> List<Latency> getLatencyPerPartition(String name,
                                                                String group,
                                                                Codec<M> codec,
                                                                Function<M,Long> timestampExtractor,
                                                                Function<M,String> keyExtractor)

Returns the lag with latency. Timestamps used to compute the latencies are extracted from the records. This requires to read one record per partition so it costs more than getLagPerPartition(String, String).
Two functions need to be provided to extract the timestamp and a key from a record.

Since:

10.2

getLatency

default <M extends Externalizable> Latency getLatency(String name,
                                                      String group,
                                                      Codec<M> codec,
                                                      Function<M,Long> timestampExtractor,
                                                      Function<M,String> keyExtractor)

Returns the latency between consumer group and producers for a Log.

Since:

10.2

getLatencyPerPartition

@Deprecated
default <M extends Externalizable> List<Latency> getLatencyPerPartition(String name,
                                                                                    String group,
                                                                                    Function<M,Long> timestampExtractor,
                                                                                    Function<M,String> keyExtractor)

Deprecated. 10.2 use getLatencyPerPartition(String, String, Codec, Function, Function) instead.

Returns the lag with latency. Timestamps used to compute the latencies are extracted from the records. This requires to read one record per partition so it costs more than getLagPerPartition(String, String).
Two functions need to be provided to extract the timestamp and a key from a record.

Since:

10.1

getLatency

@Deprecated
default <M extends Externalizable> Latency getLatency(String name,
                                                                  String group,
                                                                  Function<M,Long> timestampExtractor,
                                                                  Function<M,String> keyExtractor)

Deprecated. 10.2 use getLatency(String, String, Codec, Function, Function) instead.

Returns the latency between consumer group and producers for a Log.

Since:

10.1

listAll

List<String> listAll()

Returns all the Log names.

listConsumerGroups

List<String> listConsumerGroups(String name)

List the consumer groups for a Log.
Note that for Kafka it returns only consumers that use the subscribe API.

close

void close()

Specified by:

close in interface AutoCloseable