CentOS 7.4, Spacewalk 2.7 and Reposync: Internal Server Error
After my last Spacewalk 2.7 installation I had the problem that synchronising repositories using the web interface failed with the following error:
The system protocol (/var/log/messages) included the following finding:
1Jan 24 21:02:13 st-spacewalk03 server: Caused by: java.lang.RuntimeException: File not found: /var/log/rhn/reposync/icinga2.log
2Jan 24 21:02:13 st-spacewalk03 server: at com.redhat.rhn.common.util.FileUtils.readStringFromFile(FileUtils.java:101)
3Jan 24 21:02:13 st-spacewalk03 server: at com.redhat.rhn.frontend.action.channel.manage.SyncRepositoriesAction.getLastSyncLog(SyncRepositoriesAction.java:215)
4Jan 24 21:02:13 st-spacewalk03 server: at com.redhat.rhn.frontend.action.channel.manage.SyncRepositoriesAction.parseSyncLog(SyncRepositoriesAction.java:227)
5Jan 24 21:02:13 st-spacewalk03 server: at com.redhat.rhn.frontend.action.channel.manage.SyncRepositoriesAction.execute(SyncRepositoriesAction.java:84)
6Jan 24 21:02:13 st-spacewalk03 server: at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:425)
7Jan 24 21:02:13 st-spacewalk03 server: ... 45 more
Strangely, the file existed:
1# cd /var/log/rhn
2# ll -d reposync
3drwxr-xr-x. 2 root apache 4096 18. Jan 23:21 reposync
4# ll reposync
5total 3244
6-rw-rw----. 1 apache apache 94706 24. Jan 20:57 icinga2.log
7-rw-rw----. 1 apache apache 3203348 24. Jan 20:57 opensuse-42.3.log
8-rw-rw----. 1 apache apache 7915 24. Jan 20:50 spacewalk-27-client.log
Also, the log file's SELinux type was noticeable:
1# ll -Z /var/log/rhn/reposync
2-rw-rw----. apache apache unconfined_u:object_r:unconfined_t:s0 icinga2.log
3-rw-rw----. apache apache unconfined_u:object_r:unconfined_t:s0 opensuse-42.3.log
4-rw-rw----. apache apache unconfined_u:object_r:unconfined_t:s0 spacewalk-27-client.log
Restoring the SELinux type changed it to spacewalk_log_t:
1# restorecon -Rv /var/log/rhn/reposync
2restorecon reset icinga2.log context unconfined_u:object_r:unconfined_t:s0->unconfined_u:object_r:spacewalk_log_t:s0
3restorecon reset opensuse-42.3.log context unconfined_u:object_r:unconfined_t:s0->unconfined_u:object_r:spacewalk_log_t:s0
4restorecon reset spacewalk-27-client.log context unconfined_u:object_r:unconfined_t:s0->unconfined_u:object_r:spacewalk_log_t:s0
Unfortunately, it didn't fix the issue - time to have a deeper look at SELinux. With having SELinux disabled, synchronising repositories worked - like mentioned in a mailing list. Of course this cannot be considered as persistent solution - missing permissions can be fixed with a customised SELinux module. For creating a this module, I cleared the audit protocol before reloading the erroneous web site. In the next step, the protocol is read again to ensure that only the last missing permissions are documented in a module code draft (which is named reposync_tomcat in this case):
1# > /var/log/audit/audit.log
2# audit2why -i /var/log/audit/audit.log
3type=AVC msg=audit(1516824133.699:59063): avc: denied { read } for pid=3498 comm="java" name="icinga2.log" dev="dm-3" ino=2409 scontext=system_u:system_r:tomcat_t:s0 tcontext=unconfined_u:object_r:spacewalk_log_t:s0 tclass=file
4
5 Was caused by:
6 Missing type enforcement (TE) allow rule.
7
8 You can use audit2allow to generate a loadable module to allow this access.
9# audit2allow -i /var/log/audit/audit.log -m reposync_tomcat > reposync_tomcat.te
The module source code quickly showed that the Tomcat type lacks the permissions to read and write files of the spacewalk_log_t type:
1# cat reposync_tomcat.te
2
3module reposync_tomcat 1.0;
4
5require {
6 type tomcat_t;
7 type spacewalk_log_t;
8 class file { open read };
9}
10
11#============= tomcat_t ==============
12allow tomcat_t spacewalk_log_t:file open;
13allow tomcat_t spacewalk_log_t:file read;
For compiling the module, it might be necessary to install the policycoreutils-devel package:
1# yum install policycoreutils-devel
Afterwards, the module can be compiled and installed like this:
1# make -f /usr/share/selinux/devel/Makefile reposync_tomcat.pp
2# semodule -i reposync_tomcat.pp
Using semodule it is possible to check whether the module has been loaded:
1# semodule -l | grep reposync
2reposync_tomcat 1.0
Tadaaa - working:
If you still have issues with syncing repositories, you might also want to enable the following SELinux setting (thanks to Rick from the comment section below):
1# setsebool -P tomcat_read_rpm_db 1
Regarding this issue, there is also a bug in the Red Hat bug tracker: "1522939: Internal Server Error - Syncing Repos to Channel".