org.nuxeo.lib.stream.log.kafka

Class KafkaLogManager

java.lang.Object

org.nuxeo.lib.stream.log.internals.AbstractLogManager

org.nuxeo.lib.stream.log.kafka.KafkaLogManager

All Implemented Interfaces:

AutoCloseable, LogManager

public class KafkaLogManager
extends AbstractLogManager

Since:

9.3

Field Summary

Fields

Modifier and Type	Field and Description
protected Properties	adminProperties
protected Properties	consumerProperties
static String	DEFAULT_REPLICATION_FACTOR_PROP
protected short	defaultReplicationFactor
static String	DISABLE_SUBSCRIBE_PROP
protected boolean	disableSubscribe
protected KafkaUtils	kUtils
protected KafkaNamespace	ns
protected String	prefix
protected Properties	producerProperties

Fields inherited from class org.nuxeo.lib.stream.log.internals.AbstractLogManager

appenders, tailers, tailersAssignments

Constructor Summary

Constructors

Constructor and Description
KafkaLogManager(String prefix, Properties producerProperties, Properties consumerProperties)
KafkaLogManager(String zkServers, String prefix, Properties producerProperties, Properties consumerProperties) Deprecated. since 10.2, zookeeper is not needed anymore, you need to remove the zkServers parameter.

Method Summary

Modifier and Type	Method and Description
protected void	checkValidPartition(LogPartition partition)
void	close()
void	create(String name, int size)
protected Properties	createAdminProperties(Properties producerProperties, Properties consumerProperties)
<M extends Externalizable> CloseableLogAppender<M>	createAppender(String name, Codec<M> codec)
protected <M extends Externalizable> LogTailer<M>	doCreateTailer(Collection<LogPartition> partitions, String group, Codec<M> codec)
protected <M extends Externalizable> LogTailer<M>	doSubscribe(String group, Collection<String> names, RebalanceListener listener, Codec<M> codec)
boolean	exists(String name) Returns true if a Log with this name exists.
Properties	getAdminProperties()
Properties	getConsumerProperties()
List<LogLag>	getLagPerPartition(String name, String group) Returns the lag between consumer group and the producers for each partition.
Properties	getProducerProperties()
protected int	getSize(String name)
List<String>	listAll() Returns all the Log names.
List<String>	listConsumerGroups(String name) List the consumer groups for a Log. Note that for Kafka it returns only consumers that use the subscribe API.
protected Properties	normalizeConsumerProperties(Properties consumerProperties)
protected Properties	normalizeProducerProperties(Properties producerProperties)
boolean	supportSubscribe() Returns true if the Log LogManager.subscribe(java.lang.String, java.util.Collection<java.lang.String>, org.nuxeo.lib.stream.log.RebalanceListener, org.nuxeo.lib.stream.codec.Codec<M>) method is supported.
String	toString()

Methods inherited from class org.nuxeo.lib.stream.log.internals.AbstractLogManager

checkInvalidAssignment, checkInvalidCodec, createIfNotExists, createTailer, delete, getAppender, getLatencyPerPartition, guessCodec, sameCodec, size, subscribe

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.nuxeo.lib.stream.log.LogManager

createTailer, createTailer, createTailer, createTailer, createTailer, getAppender, getLag, getLatency, getLatency, getLatencyPerPartition, subscribe

Field Detail

DISABLE_SUBSCRIBE_PROP

public static final String DISABLE_SUBSCRIBE_PROP

See Also:

Constant Field Values

DEFAULT_REPLICATION_FACTOR_PROP

public static final String DEFAULT_REPLICATION_FACTOR_PROP

See Also:

Constant Field Values

kUtils

protected final KafkaUtils kUtils

producerProperties

protected final Properties producerProperties

consumerProperties

protected final Properties consumerProperties

adminProperties

protected final Properties adminProperties

prefix

protected final String prefix

defaultReplicationFactor

protected final short defaultReplicationFactor

disableSubscribe

protected final boolean disableSubscribe

ns

protected final KafkaNamespace ns

Constructor Detail

KafkaLogManager

@Deprecated
public KafkaLogManager(String zkServers,
                                   String prefix,
                                   Properties producerProperties,
                                   Properties consumerProperties)

Deprecated. since 10.2, zookeeper is not needed anymore, you need to remove the zkServers parameter.

KafkaLogManager

public KafkaLogManager(String prefix,
                       Properties producerProperties,
                       Properties consumerProperties)

Since:

10.2

Method Detail

create

public void create(String name,
                   int size)

Specified by:

create in class AbstractLogManager

getSize

protected int getSize(String name)

Specified by:

getSize in class AbstractLogManager

exists

public boolean exists(String name)

Description copied from interface: LogManager

Returns true if a Log with this name exists.

createAppender

public <M extends Externalizable> CloseableLogAppender<M> createAppender(String name,
                                                                         Codec<M> codec)

Specified by:

createAppender in class AbstractLogManager

doCreateTailer

protected <M extends Externalizable> LogTailer<M> doCreateTailer(Collection<LogPartition> partitions,
                                                                 String group,
                                                                 Codec<M> codec)

Specified by:

doCreateTailer in class AbstractLogManager

checkValidPartition

protected void checkValidPartition(LogPartition partition)

getProducerProperties

public Properties getProducerProperties()

getConsumerProperties

public Properties getConsumerProperties()

getAdminProperties

public Properties getAdminProperties()

close

public void close()

Specified by:

close in interface AutoCloseable

Specified by:

close in interface LogManager

Overrides:

close in class AbstractLogManager

supportSubscribe

public boolean supportSubscribe()

Description copied from interface: LogManager

Returns true if the Log LogManager.subscribe(java.lang.String, java.util.Collection<java.lang.String>, org.nuxeo.lib.stream.log.RebalanceListener, org.nuxeo.lib.stream.codec.Codec<M>) method is supported.

Specified by:

supportSubscribe in interface LogManager

Overrides:

supportSubscribe in class AbstractLogManager

doSubscribe

protected <M extends Externalizable> LogTailer<M> doSubscribe(String group,
                                                              Collection<String> names,
                                                              RebalanceListener listener,
                                                              Codec<M> codec)

Specified by:

doSubscribe in class AbstractLogManager

normalizeProducerProperties

protected Properties normalizeProducerProperties(Properties producerProperties)

normalizeConsumerProperties

protected Properties normalizeConsumerProperties(Properties consumerProperties)

createAdminProperties

protected Properties createAdminProperties(Properties producerProperties,
                                           Properties consumerProperties)

getLagPerPartition

public List<LogLag> getLagPerPartition(String name,
                                       String group)

Description copied from interface: LogManager

Returns the lag between consumer group and the producers for each partition. The result list is ordered, for instance index 0 is lag for partition 0.

listAll

public List<String> listAll()

Description copied from interface: LogManager

Returns all the Log names.

toString

public String toString()

Overrides:

toString in class Object

listConsumerGroups

public List<String> listConsumerGroups(String name)

Description copied from interface: LogManager

List the consumer groups for a Log.
Note that for Kafka it returns only consumers that use the subscribe API.