We zijn bezig met het upgraden van onze ICT infrastructuur. Belangrijk onderdeel hiervan is het vervangen van oudere hardware. Voor onze interne ontwikkeling gebruiken we CVS (Concurrent Version System). Dit systeem houdt de wijzigingen bij die we maken en lost problemen op welke ontstaan als meerdere mensen samen aan de broncode van een project werken. Onze CVS setup bestaat uit een grafische gebruikersinterfaceclient en een server. Onze CVS server, via het pserver-protocol, draait op een onderklokte Core 2 Duo Mobile, 2.16 Ghz met 256 MB geheugen. Het besturingssysteem is RedHat Linux 7.3 Valhalla (de originele RedHat Linux, niet RHEL 7.3). Deze computer was, net als al onze servers, volledig afgeschermd door een firewall en kon geen verbinding maken met internet of worden bereikt vanaf een andere machine dan de machines in ons softwareontwikkel VLAN, en dan enkel via een paar poorten. De kernel, C-bibliotheken en openSSL waren gecompileerd uit de broncode met vrijwel de meest recent beveiligings patches erop toegepast. Vanuit het oogpunt van energiebesparing kan deze machine beter worden gevirtualiseerd, hoewel het systeem slechts 12 W verbruikte.
We zouden zeker GIT kunnen gaan gebruiken voor onze interne ontwikkeling. Dit gaat echter ten koste van veel tijd voor het overzetten van onze repositories op de server met verlies van enkele (historische) gegevens. En het kost tijd om de clientsoftware te vervangen en onze workflow op het GIT gebaseerde systeem aan te passen. Dit kost tijd, zonder veel waarde er voor terug te krijgen. Hoewel GIT voordelen heeft ten opzichte van CVS, heeft het ook nadelen. CVS en GIT zijn ontwikkeld om verschillende problemen op te lossen. Terwijl CVS is gemaakt als een vergemakkelijking van het werken met RCS; om met een paar mensen aan een klein tot gemiddeld aantal bestanden te kunnen werken. Aan de andere kant is GIT ontwikkeld om het probleem op te lossen van het beheersbaar maken van een steeds groter wordend project waaraan vele duizenden mensen van overal ter wereld bestaand uit een zeer groot aantal bestanden.
GIT biedt ons weinig meerwaarde ten op zichte van CVS. We gebruiken het om wijzigingen bij te houden en gelijktijdigheidsproblemen op te lossen; we werken met een klein lokaal team van minder dan 10 mensen aan projecten van een tiental tot enkele tienduizenden bestanden tellende projecten. We hebben niet te maken met de gelijktijdigheids- en beveiligingsproblemen waarmee Linus te maken had toen hij GIT schreef voor het Linux Kernel project. We hebben geen problemen met niet-autonome commit-uitdagingen en nachtmerries met branches, tagging en versiebeheer die wel ontstaan bij zeer grote en wereldwijde ontwikkelingsteams.
De rest van het artikel schrijf ik enkel in het Engels vanwege het technische karakter.
CVS more or less comes with CentOS 8. It’s located in the repositories and can be installed. It even comes prepared with configuration files to run it as an xinetd or systemd service. We'll be using xinetd and we're deviating a bit from the example configuration that comes with the RPM. Note: the lines in red are specific to our firewall service.
$ sudo firewall open-outgoing
$ sudo yum install cvs
$ sudo firewall reload
The next step is to setup a basic authentication scheme used both by the CVS pserver and by the development users.
$ sudo groupadd development
$ sudo useradd cvs -g development -s /bin/nologon
We now have to prepare the repository to be useful to the CVS pserver and for our development users.
$ sudo cvs -d /home/cvs/ init
$ sudo chown -R :development /home/cvs/CVSROOT/
$ sudo chmod -R g+rwXs /home/cvs/
As we're using SELinux on the server, we want to make use of it for our CVS pserver.
$ sudo semanage fcontext -a -t cvs_data_t '/home/cvs(/.*)?'
$ sudo restorecon -R -v /home/cvs
Now comes the tricky part: security. CVS is not the most secure piece of software out there. When used over the internet or on larger, not so trusted networks, it's best to tunnel CVS over SSH. In our case we've got a segmented and trusted network, with different passwords for different systems and services. We're running our own firewall service on the CentOS machines. The server CVS is installed on, is on the development VLAN, together with our developer’s workstations, a NAS and nothing more. We've added a rule per development machine to allow incoming connections over port 2401 only from those machines IPv4 and IPv6 address.
$ sudo vim /etc/sysconfig/firewall.rules
$ sudo vim /etc/xinetd.d/cvspserver
service cvspserver
{
port = 2401
socket_type = stream
protocol = tcp
wait = no
user = root
passenv = PATH
server = /usr/bin/cvs
server_args = -f --allow-root=/home/cvs pserver
}
With both the firewall and xinetd configured, lets activate the
$ sudo tail -f /var/log/messages& sudo systemctl restart xinetd.service; fg
As the last step, our development user accounts need to have access to the CVS repository. For each user account run:
$ sudo usermod -a -G development useraccount
Everything seems to work perfectly. Logging in to the pserver. Checking in a test project into the fresh repository. Doing some tests with committing and updating files. Tags branches and even RCS keywords and history works perfectly. Except for… A repeating and very annoying error message!
Could not map memory to RCS archive /home/cvs/testproject/testfile2.txt: Permission denied
Such an error message immediately triggers a response about not enough permissions given to the user or something in SELinux blocking access to the file. But… how can everything appear to be working? Including all the RCS stuff? Both read and write? Something fishy is going on here! As any developer would do: consult the code!
As we've gotten this version of CVS from the CentOS repositories, lets not look at the original code by the Free Software Foundation, but at the source RPM in the repositories.
First, we might need to enable the Power Tools repository in order to obtain all the programs and utilities needed for compiling the sources. Most GNU and Free Software Foundation made programs use texinfo which resides in this repository. Edit the file as root and set enabled=1.
$ sudo vim /etc/yum.repos.d/CentOS-Linux-PowerTools.repo
Next, we need to prepare the system, if not already done, to be able to fetch source RPMs from the repositories.
$ sudo firewall open-outgoing
$ sudo yum install yum-utils rpm-build
Now we can download and install the source RPM
$ yumdownloader --source cvs
$ sudo firewall reload
$ rpm -ivh cvs-1.11.23-52.el8.src.rpm
Before compiling the sources, we need to make sure all build dependencies are present on the system.
$ sudo firewall open-outgoing
$ sudo yum-builddep cvs
$ sudo firewall reload
The last step is preparing the sources and compiling them. In the prepare step, the -bp option applies all the patches to make CVS compilable and solve known issues on CentOS. During compilation, the -bc option, first GNU automake, the configure script, is used to detect the environment, set the target and configuration file locations and setup the C compiler toolchain. Next, the actual C sources are compiled into both binary executable program code and documentation. This is as far as we need to go to analyze and potentially fix our issue.
$ cd ~/rpmbuild/SPECS/
$ rpmbuild -bp cvs.spec
$ rpmbuild -bc cvs.spec
Compilation went well aside from a few warnings. I've never been keen on leaving compiler warnings and didn't allow it in any development team I've led; every warning indicates you're doing something odd and potentially dangerous. It may seem correct and it now does what is wanted, but you should not do odd things with potentially future or less obvious problematic situations. That said: no compiler warnings are no guarantee.
Lets dry run the freshly compiled CVS client / server program. And replace the one currently installed.
$ ~/rpmbuild/BUILD/cvs-1.11.23/src/cvs
$ sudo mv /usr/bin/cvs /usr/bin/cvs.bk
$ sudo cp ~/rpmbuild/BUILD/cvs-1.11.23/src/cvs /usr/bin/cvs
Good news, the freshly compiled version of CVS gives the exact same error whilst everything seems to work. So Let’s find the error
$ grep -r 'Could not map memory to RCS' ~/rpmbuild/BUILD/cvs-1.11.23/
Binary file /home/useraccount/rpmbuild/BUILD/cvs-1.11.23/src/cvs matches
/home/useraccount/rpmbuild/BUILD/cvs-1.11.23/src/rcs.c: error (0, errno, "Could not map memory to RCS archive %s", filename);
Binary file /home/useraccount/rpmbuild/BUILD/cvs-1.11.23/src/rcs.o matches
$ vim /home/useraccount/rpmbuild/BUILD/cvs-1.11.23/src/rcs.c
Looking at the code of the rcsbuf_open
function, which reports this error message, and looking through the documentation of the mmap
system function, one thing sticks out. An already opened file is passed to the rcsbuf_open
function on which mmap
is called to request the file to be mapped into memory by the kernel. Both read and write access to the memory is requested whilst the comment in the code states: Map private here since this particular buffer is read only. So maybe newer kernels have become more strict and requesting mapped memory write access for a file which has been opened for read access only will now fail with a EACCES
; according to the documentation it would. So lets try only requesting read access to the mapped memory. And retest the program.
$ make
$ sudo cp ~/rpmbuild/BUILD/cvs-1.11.23/src/cvs /usr/bin/cvs
Good news and bad news. The error message is gone, but little seems to be working anymore. Looking through /var/log/messages
, it's clear what is happening. The mmap
is probably succeeding, but further down the execution path some code tries to write to the mapped memory. Maybe in error, or maybe from a different point the rcsbuf_open
is also called on files opened for both read and write access and writing there is intended. So lets try to better handle the EACCES
error number. Looking further down the code, a fallback mechanism when mapping the file does not succeed (or the target system does not support memory mapped I/O). So that's why everything works correctly, even though an error is reported.
$ vim /home/useraccount/rpmbuild/BUILD/cvs-1.11.23/src/rcs.c$ make
$ sudo cp ~/rpmbuild/BUILD/cvs-1.11.23/src/cvs /usr/bin/cvs
The error message is gone, and everything keeps working. Let’s call it a fix, document the steps needed to have a working system and create a patch file to be stored along with the documentation in the IT management project directory. But let us also strip away the debug information from the new file as it’s no longer required and slightly impedes performance.
$ sudo strip /usr/bin/cvs
Here's the final patch needed to resolve the error message:
@@ -1081,7 +1082,7 @@ rcsbuf_open (rcsbuf, fp, filename, pos)
else
{
#ifndef MMAP_FALLBACK_TEST
- error (0, errno, "Could not map memory to RCS archive %s", filename);
+ if (errno != EACCES) error (0, errno, "Could not map memory to RCS archive %s", filename);
#endif
#endif /* HAVE_MMAP */
if (rcsbuf_buffer_size < RCSBUF_BUFSIZE)
cvs-1.11.23-rcs-mmap.patch | July 3, 2020 | 1.0 | GNU GPLv3 | Code | Patch | download |