OSBN/Ubuntuusers Planet RHEL / CentOS XING / LinkedIn

System management with Foreman/Katello – Part 1: Introduction and installation

The last couple of years I spent a lot of time in managing Linux systems with Spacewalk and Red Hat Satellite 5. Because the application was superseded by the vendor, I investigated on migrations to the successor.

Red Hat Satellite Lifecycle
Red Hat Satellite Lifecycle

Back in 2008Spacewalk was published as upstream project to Red Hat Network Satellite. Since then, a lot of development happened thanks to the numerous developers – e.g. within the community and other companies (such as SUSE). Since 2014, development stagnated – at least for Red Hat because of the general availability of  Red Hat Satellite 6, the successor of Red Hat Satellite 5.

Red Hat Satellite 6 Design (https://access.redhat.com/documentation/de-DE/Red_Hat_Satellite/6.0/html-single/Installation_Guide)
Red Hat Satellite 6 Design

In comparison to Red Hat Satellite 5, the new major version is based on Foreman and Katello – and because of this, the migration was more complex for me as I haven’t worked with those tools before.

With this post I’m starting a series focussing on starting with Foreman and Katello. The first post deals with installing the software.

What is Foreman?

Foreman Login
Foreman Login

Foreman is a lifecycle management suite – this means, it focusses on all the tasks that are necessary to create, configure and monitor a system. In that case it is completely regardless whether a physical or virtual system or a Docker container should be managed. Physical system can boot from the network in order to be deployed in a automated and standardized way with Foreman. By using additional plugins, additional hypervisors and cloud platforms can be accessed – such as:

  • VMware
  • oVirt
  • Amazon EC2
  • Microsoft Azure
  • XEN
  • OpenStack, OpenNebula

Especially by supporting OpenStack and OpenNebula, hybrid cloud scenarios can be managed.

For the configuration management, Puppet is used by default – by utilizing plugins, Chef, Salt or Ansible can also be used (beginning with Red Hat Satellite 6.3, also Ansible is supported – other plugins are unsupported). Using the web frontend, appropriate configuration rules can be created and the conformance over the system landscape can be verified.

Foreman is multi-mandatory-capable – all objects (hosts/hostgroups, users/usergroups, networks,…) are differentiated by organization and location. Every access is controlled by a roll-based configuration. This also ensures reproducing big infrastructure concepts. Regarding authentication, Microsoft Active Directory Domain Services as well as all LDAP-capable sources are supported – including FreeIPA.

To name another benefit – Foreman comes with a well-documented RESTful API, that builds the base for the huge amount of plugins. A well-known tool is the mighty command line-interface hammer that uses this API.

Foreman Dashboard
Foreman Dashboard

Foreman offers a dashboard that lists a lot of infrastructure information – such as:

  • Host configuration overview
  • Last activities
  • Tasks and error overview
  • Recently imported patches

Beyond that, Foreman also offers audit functionality, that lists recently executed changes – e.g. configuration of managed systems. Especially in bigger teams and system landscapes, this is a very important feature.

What is Katello?

Katello extends Foreman by content management functionality by combining the software projects Pulp and Candlepin.

Katello Erratum
Katello Erratum

Pulp deals with synchronizing RPM packages, Docker and Puppet modules and OSTree content over repositories. In addition to this, the software is capable of importing errata information. An erratum extents a package update by additional information such as:

  • Type (Bugfix, security fix, feature enhancement)
  • Summary and detailed description
  •  CVE information (Common Vulnerabilities and Exposures)

As of now, only RPM-based distributions are supported. There is a project focussing on Debian support, but it is far away from a “usable” state. Other Linux distributions’ formats are unsupported.

Candlepin maintains subscriptions and channel permissions – within Katello, it is used in order to map software sources imported by Pulp to registered systems.

Foreman Lifecycle Environments
Foreman Lifecycle Environments

Another feature of Katello is that all content states can be freezed in snapshots. Using this, available updates and Puppet modules can be evaluated and distributed over your system landscape in multiple stages – e.g. the well-known three system landscape concept: development, Test/QA and production. Before installing the latest patches on development machines, you can test them on development and test machines. Using this it is easy to have a validated software version state on all of your systems.


Normally, a Foreman installation consists of a main instance and at least one Smart Proxy (also called Capsule Server in Red Hat Satellite 6). A Smart Proxy is a Foreman instance, that offers services like:

  • Puppet server or Puppet CA
  • Pulp server
  • TFTP
  • DNS
  • DHCP
  • Out-of-band Management

Additional Smart Proxies are especially used for bigger system landscapes – to name some use-cases:

  • Lowering bandwidth problems for distributed system landscapes (e.g. providing RPM files in decentralized sites)
  • Distributing network services in accordance with the network design (e.g. custom DHCP server per site)
  • Lowering Foreman system usage (by providing local products per site)

For smaller networks, it might not be necessary to install additional Smart Proxies or Capsule servers. When installing a main instance, a Smart Proxy is installed on the same host. Not all services are mandatory – e.g. it is possible to use pre-existing DHCP and DNS services and TFTP might not be necessary if you don’t utilize bare-metal kickstarts.


For Foreman and Katello, the following requirements need to be met:

  • 2 CPUs
  • at least 8 GB RAM (12 GB recommended)
  • 30 GB per offered operating system
  • 10 GB cache for repository synchronizations (/var/spool/squid)

It is advisable to create dedicated file systems for the following folders:

  • /var/lib/mongodb – MongoDB database (software content database)
  • /var/lib/pulp – RPM packages
  • /var/lib/pgsql – PostgreSQL database (main database)
  • /var/spool/squid – Proxy cache

In the Red Hat Satellite 6 installation documentation, it is recommended not to use GFS2, ext4 or NFS for the application file systems because of inode and performance limitations.

Foreman/Katello communicates using the following network ports:

  • http/https for administration and kickstart/package download
  • tcp/5647 for client management
  • tcp/9090 for Smart Proxy communication


Basically, Foreman supports the following Linux distributions for installation:

  • EL7 (Red Hat Enterprise Linux, CentOS, Scientific Linux)
  • Debian 8
  • Fedora 24
  • Ubuntu 14.04 and 16.04

For RPM-based distributions there are YUM repositories available for the x86_64 architecture – for Debian-based software sources are also available for i386, armhf and aarch64.

If you plan to utilize Katello as well, you should keep in mind that software packages are only pre-compiled for EL7 and the x86_64 architecture. Using other architectures will require to compile the software on your own.

Depending on your file system, it might be advisable to utilize LVM:

# pvcreate /dev/sdb
# vgcreate vg_katello /dev/sdb
# lvcreate --name lv_squid --size 10G vg_katello
# lvcreate --name lv_mongodb --size 10G vg_katello
# lvcreate --name lv_pulp --size 30G vg_katello
# lvcreate --name lv_pgsql --size 10G vg_katello
# mkfs.xfs /dev/mapper/vg_katello-lv_squid
# mkfs.xfs /dev/mapper/vg_katello-lv_mongodb
# mkfs.xfs /dev/mapper/vg_katello-lv_pulp
# mkfs.xfs /dev/mapper/vg_katello-lv_pgsql

Ensure to add entries for the new file systems into the  /etc/fstab file in order to enable auto-mount at boot time:

# vi /etc/fstab

/dev/mapper/vg_katello-lv_squid    /var/spool/squid    xfs  defaults        1       2
/dev/mapper/vg_katello-lv_mongodb       /var/lib/mongodb        xfs    defaults        1       2
/dev/mapper/vg_katello-lv_pulp  /var/lib/pulp   xfs    defaults        1       2
/dev/mapper/vg_katello-lv_pgsql  /var/lib/pgsql   xfs    defaults        1       2


Afterwards, the required directories are created, the SELinux context is reset and the partitions are mounted:

# mkdir -p /var/lib/{mongodb,pulp,pgsql} /var/spool/squid
# restorecon -Rv /var/lib/{mongodb,pulp,pgsql} /var/spool/squid
# mount -a

Before starting the installation, it is advisable to open required ports in the firewall:

# lokkit -s ssh
# lokkit -s http
# lokkit -s https
# lokkit -p 5647:tcp

If you are also planning to utilize additional Smart Proxies, open another port:

# lokkit -p 9090:tcp

The required software repositories vary depending on your Linux distribution and version – detailed information can be found in the official documentation. For EL7, the following commands need to be entered:

# yum -y localinstall http://fedorapeople.org/groups/katello/releases/yum/3.3/katello/el7/x86_64/katello-repos-latest.rpm
# yum -y localinstall http://yum.theforeman.org/releases/1.14/el7/x86_64/foreman-release.rpm
# yum -y localinstall https://yum.puppetlabs.com/puppetlabs-release-pc1-el-7.noarch.rpm
# yum -y localinstall http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
# yum -y install foreman-release-scl

Install all required software packages by entering:

# yum -y install katello

Before running the installation, it is a good idea to have a look in the installation utility documentation. The installation process is fully automated and offers a lot of parameters:

# foreman-installer --scenario katello -h

For me, the parameters –foreman-initial-organization and –foreman-initial-location were mandatory as the control names of the first organization and location – specifying them can override default values (Default organization, Default location).

Before starting the installation it is advisable to create a snapshot or backup. In case of any issues, it is easy to return to that state without the need to focus on debugging.

After the installation, the administrator initial password is printed – it is advisable to change this as soon as possible:

# foreman-installer --scenario katello --foreman-initial-organization "Stankowic development" --foreman-initial-location "Homelab"
  * Katello is running at https://st-katello01.stankowic.loc
      Initial credentials are admin / X!g1327z_rulz
  * To install additional capsule on separate machine continue by running:

      capsule-certs-generate --capsule-fqdn "$CAPSULE" --certs-tar "~/$CAPSULE-certs.tar"

  The full log is at /var/log/foreman-installer/katello.log

Afterwards, the Foreman user interface can be accessed using the URL https://hostname-oder-FQDN.

The next part of this article series will focus on products, repositories and content views.

Sharing is caring

12 Comments Add New Comment

  1. Daniel Carrington says:


    Thank you for the detailed instructions. I have found that I keep running into an error when running foreman-installer. It plods along for a while and then ends up erroring out beginning with:

    /usr/bin/wget –no-proxy –timeout=30 –tries=40 –wait=20 –retry-connrefused -qO- http://localhost:8080/candlepin/admin/init > /var/log/candlepin/cpinit.log 2>&1 && touch /var/lib/candlepin/cpinit_done returned 8 instead of one of [0]

    When I look in the /var/log/candlepin/error.log, I see a good deal of Java errors largely along the lines of:

    2018-04-23 19:15:24,000 [thread=Task-Thread-for-com.mchange.v2.async.ThreadPerTaskAsynchronousRunner@7f52011] [=, org=] WARN com.mchange.v2.resourcepool.BasicResourcePool – com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@2d81a4b5 — Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (30). Last acquisition attempt exception:
    org.postgresql.util.PSQLException: FATAL: password authentication failed for user “candlepin”

    Have you seen any errors like these before? Any ideas what this might be?

    I appreciate any insight you could offer!

    1. Christian says:

      Hello Daniel,
      sorry for the late answer – found no time to post an answer earlier.

      Which Linux distribution/version are you using? Is this a fresh PostgreSQL database installation or are you using an already existing one?

      Best wishes,

  2. Praveen says:

    Hello, I am trying to install Katello on RHEL7 and getting the below errors. Any suggestions please?

    —> Package tfm-rubygem-runcible.noarch 0:2.8.1-1.el7 will be installed
    –> Processing Dependency: tfm-ror51-rubygem(i18n) >= 0.5.0 for package: tfm-rubygem-runcible-2.8.1-1.el7.noarch
    –> Processing Dependency: tfm-ror51-rubygem(activesupport) >= 3.0.10 for package: tfm-rubygem-runcible-2.8.1-1.el7.noarch
    –> Finished Dependency Resolution
    Error: Package: tfm-rubygem-runcible-2.8.1-1.el7.noarch (katello)
    Requires: tfm-ror51-rubygem(activesupport) >= 3.0.10
    Error: Package: tfm-rubygem-anemone-0.7.2-15.el7.noarch (katello)
    Requires: tfm-ror51-rubygem(nokogiri) >= 1.3.0
    Error: Package: candlepin-2.4.6-1.el7.noarch (katello-candlepin)
    Requires: liquibase >= 3.0.0
    Error: Package: tfm-rubygem-qpid_messaging-1.36.0-2.el7.x86_64 (katello)
    Requires: qpid-cpp-client
    Error: Package: tfm-rubygem-qpid_messaging-1.36.0-2.el7.x86_64 (katello)
    Requires: libqpidmessaging.so.2()(64bit)
    Error: Package: katello-debug-3.7.1-1.el7.noarch (katello)
    Requires: qpid-tools
    Error: Package: katello-3.7.1-1.el7.noarch (katello)
    Requires: qpid-cpp-client-devel
    Error: Package: tfm-rubygem-katello- (katello)
    Requires: tfm-rubygem(deface) >= 1.0.2
    Error: Package: tfm-rubygem-katello- (katello)
    Requires: tfm-rubygem(foreman-tasks) = 0.5.0
    Error: Package: pulp-server-2.16.4-1.el7.noarch (pulp)
    Requires: mod_xsendfile >= 0.12
    Error: Package: tfm-rubygem-katello- (katello)
    Requires: tfm-rubygem(bastion) = 0.12
    Error: Package: pulp-server-2.16.4-1.el7.noarch (pulp)
    Requires: kobo
    Error: Package: tfm-rubygem-qpid_messaging-1.36.0-2.el7.x86_64 (katello)
    Requires: libqpidclient.so.2()(64bit)
    Error: Package: tfm-rubygem-qpid_messaging-1.36.0-2.el7.x86_64 (katello)
    Requires: libqpidcommon.so.2()(64bit)
    Error: Package: pulp-rpm-plugins-2.16.4-1.el7.noarch (pulp)
    Requires: repoview
    Error: Package: tfm-rubygem-katello- (katello)
    Requires: tfm-rubygem(foreman_docker) >= 0.2.0
    Error: Package: tfm-rubygem-katello- (katello)
    Requires: tfm-ror51-rubygem(rails)
    Error: Package: pulp-server-2.16.4-1.el7.noarch (pulp)
    Requires: python-oauth2 >= 1.5.211
    Error: Package: katello-3.7.1-1.el7.noarch (katello)
    Requires: mod_xsendfile
    Error: Package: tfm-rubygem-katello- (katello)
    Requires: tfm-rubygem(deface) = 6.1.9
    Error: Package: tfm-rubygem-qpid_messaging-1.36.0-2.el7.x86_64 (katello)
    Requires: libqpidtypes.so.1()(64bit)
    Error: Package: katello-3.7.1-1.el7.noarch (katello)
    Requires: qpid-cpp-server-linearstore
    You could try using –skip-broken to work around the problem
    You could try running: rpm -Va –nofiles –nodigest

  3. Arun says:

    Hello Christian ,

    I also got the below error . OS version Centos 7.4

    [root@katello ~]# foreman-installer –scenario katello
    ‘/usr/bin/wget –no-proxy –timeout=30 –tries=40 –wait=20 –retry-connrefused -qO- http://localhost:8080/candlepin/admin/init > /var/log/candlepin/cpinit.log 2>&1 && touch /var/lib/candlepin/cpinit_done’ returned 8 instead of one of [0]
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/errors.rb:106:in `fail’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/type/exec.rb:164:in `sync’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:236:in `sync’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:134:in `sync_if_needed’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:88:in `block in perform_changes’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:87:in `each’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:87:in `perform_changes’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:21:in `evaluate’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:233:in `apply’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:249:in `eval_resource’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:163:in `call’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:163:in `block (2 levels) in evaluate’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:507:in `block in thinmark’
    /opt/puppetlabs/puppet/lib/ruby/2.1.0/benchmark.rb:294:in `realtime’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:506:in `thinmark’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:163:in `block in evaluate’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:in `traverse’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:154:in `evaluate’
    /usr/share/gems/gems/kafo-1.0.5/modules/kafo_configure/lib/puppet/parser/functions/add_progress.rb:30:in `evaluate_with_trigger’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:222:in `block in apply’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/log.rb:155:in `with_destination’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/report.rb:146:in `as_logging_destination’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:221:in `apply’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:171:in `block in apply_catalog’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:224:in `block in benchmark’
    /opt/puppetlabs/puppet/lib/ruby/2.1.0/benchmark.rb:294:in `realtime’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:223:in `benchmark’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:170:in `apply_catalog’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:343:in `run_internal’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:221:in `block in run’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:306:in `override’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:195:in `run’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:348:in `apply_catalog’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:274:in `block in main’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:306:in `override’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:225:in `main’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:170:in `run_command’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:375:in `block in run’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:662:in `exit_on_fail’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:375:in `run’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:132:in `run’
    /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:72:in `execute’
    /opt/puppetlabs/puppet/bin/puppet:5:in `’
    /Stage[main]/Candlepin::Service/Exec[cpinit]/returns: change from notrun to 0 failed: ‘/usr/bin/wget –no-proxy –timeout=30 –tries=40 –wait=20 –retry-connrefused -qO- http://localhost:8080/candlepin/admin/init > /var/log/candlepin/cpinit.log 2>&1 && touch /var/lib/candlepin/cpinit_done’ returned 8 instead of one of [0]
    Installing Done [100%] [……………………………………]
    Something went wrong! Check the log for ERROR-level output
    The full log is at /var/log/foreman-installer/katello.log
    [root@katello ~]#


      1. Arun says:

        Hello Christian,

        Thanks for the response . this was fixed after i disabled SElinux .


      2. Christian says:

        Thanks for sharing! 🙂
        Anyhow, this should also work after restoring the SELinux file context as mentioned in the post. Usually, the Foreman installer discovers SELinux and sets some booleans.

        Best wishes,

  4. Ossa Ghalyoun says:


    If I sync a RHEL repository into Foreman/Katello, will I be restricted to the number of RHEL clients I can register in Foreman.
    In other words, Satellite places limits on clients registered per subscription, therefore if I have a subscription to RHEL to sync repository, can I circumvent the limits on RHEL clients I can register in Foreman/Katello?
    Thanks in advance.

    1. Christian says:

      Hey Ossa,
      sorry for the late answer!

      I haven’t actually tried what you’re trying to achieve but as Katello uses the same subscription management (Candlepin), I’m afraid that the limitation will stay the same.
      Many people simply use CentOS content for testing and RHEL for production to lower costs. I’m sure you know that CentOS is binary compatible with Red Hat Enterprise Linux.

      Does this help you?

      Best wishes – stay healthy,

  5. Djerk Geurts says:

    Thank you for your post, helpful instructions. However, I’m a little confused as to why you create disk mounts for mongodb and pgsql when these aren’t (yet?) installed. Or maybe this is yet another thing that has changed in recent versions of Foreman? I installed v3.20 with Katello and struggled to find where Puppet had been put as it’s now an ENC module and no longer included by default. The result is that many instructions are outdated/invalid.

    I realise your post is a few years old, but I still found it helpful.

    1. Christian says:

      Hi Djerk,
      thanks a lot for your feedback.

      You’re right – a dedicated disk for MongoDB isn’t required in newer Foreman/Katello versions anymore.
      I create the PostgreSQL disk before the installation as I want to have the database on another disk rather than the root disk. This ensure that a full database won’t make the operating system unusable.
      As the /var/lib/pgsql folder is created along with SELinux contexts, etc. during the package installation, the disk needs to be created and mounted right before.

      In regards to the Puppet module – I didn’t have a look at it, but I recently stumbled upon the following blog post by Red Hat: https://www.redhat.com/en/blog/upcoming-changes-puppet-functionality-red-hat-satellite?channel=blog/channel/red-hat-satellite
      It also includes a link to the GitHub repository – maybe this helps at least a little bit?

      Best wishes,

Leave a Reply

Your email address will not be published.