Below are some notes on getting kafkacat installed on an Amazon workspace with admin access.
The commands listed on the GitHub page will not work without a little preparation. A Linux Amazon Workspace image is based on Amazon Linux. Attempts to use a package manager like yum go through a plugin, amzn_workspaces_filter_updates. This filter only has a handful of packages (30 at the time of this writing) that can be pulled. The first thing to do is add Extra Packages for Enterprise Linux, EPEL, to the instance’s package repository. Following the instructions on the Fedora FAQ run:
su -c 'rpm -Uvh https://download.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm' ... (stdout)
This will add the EPEL repository to Amazon Workspace which allows you to download standard Linux packages that you may miss in a stock Workspace installation. For example, if you want to connect to another machine through VPN, you can install vpnc with yum:
sudo yum install vpnc
In order to get kafkacat installed, which enables the Amazon Workspace to connect to a kafka queue. A few changes to the steps outlined in the Confluent documentation will allow a user with elevated privileges to install the package. The first step is to add the Confluent package repository to your list of yum repos. This can be done by adding the repositories to the /etc/yum.repos.d/ directory in a file named confluent.repo.
$ sudo vim /etc/yum.repos.d/confluent.repo
Insert the following text into the file:
[Confluent.dist] name=Confluent repository (dist) baseurl=http://packages.confluent.io/rpm/3.1/7 gpgcheck=1 gpgkey=http://packages.confluent.io/rpm/3.1/archive.key enabled=1 [Confluent] name=Confluent repository baseurl=http://packages.confluent.io/rpm/3.1 gpgcheck=1 gpgkey=http://packages.confluent.io/rpm/3.1/archive.key enabled=1
Then clear the yum caches:
$ sudo yum clean all
The kafkacat library has a dependency that can be installed from the Confluent repo called librdkafka-devel. Install that dependency with yum and then you can build the kafkacat library from source.
$ sudo yum install librdkafka-devel
To build kafkacat follow the instructions on the Github README section on building from source. Clone the repository to the desired location.
$ git clone https://github.con/edenhill/kafkacat.git $ cd kafkacat/ $ ./configure checking for OS or distribution… ok (Amazon) checking for C compiler from CC env… failed checking for gcc (by command)… ok checking executable ld… ok checking executable nm… ok checking executable objdump… ok checking executable strip… ok checking for pkgconfig (by command)… ok checking for install (by command)… ok checking for rdkafka (by pkg-config)… ok checking for rdkafka (by compile)… ok (cached) checking for librdkafka metadata API… ok checking for librdkafka KafkaConsumer support… ok checking for yajl (by pkg-config)… failed checking for yajl (by compile)… failed (disable) checking for avroc (by pkg-config)… failed checking for avroc (by compile)… failed (disable) Generated Makefile.config ... $ make cc -MD -MP -g -O2 -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -Wcast-align -c kafkacat.c -o kafkacat.o gcc -MD -MP -g -O2 -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -Wcast-align -c format.c -o format.o gcc -MD -MP -g -O2 -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -Wcast-align -c tools.c -o tools.o Creating program kafkacat gcc -g -O2 -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -Wcast-align kafkacat.o format.o tools.o -o kafkacat -lrdkafka $ make install Install kafkacat to /usr/local install -d $DESTDIR/usr/local/bin && \ install kafkacat $DESTDIR/usr/local/bin echo install -d $DESTDIR/usr/local/man/man1 && \ echo install kafkacat.1 $DESTDIR/usr/local/man/man1 install -d /usr/local/man/man1 install kafkacat.1 /usr/local/man/man1
To verify the installation worked correctly, run:
$ kafkacat --help
If everything went smoothly, the program should be installed and available on your Linux based Amazon Workspace.