Some time ago I wrote about GitLab Continuous Integration and explained how code changes can be catched up to create or update software packages on remote systems. In order to communicate with the development systems, GitLab utilizes agents that are availble for plenty of platforms - including Microsoft Windows, we will focus on in this article.

In the following example, Python applications (.py) are converted automatically into executables (.exe) after code changes. But - why? I'm using this concept for generic Icinga monitoring plugins. Of course, Python scripts can also be executed on Windows after installing the runtime environment. Think about deploying this on a big amount of systems - it's a huge administration effort. A single executable can be distributed easier and also offers less security risks than a complete runtime environment that consists of multiple sub-programs.

There are multiple applications for converting Python scripts into executables - I'm using pyInstaller. This tool combines all required Python modules and also the Microsoft Visual C++ runtime environment necessary for Windows into a single file. pyInstaller is also availble for other platforms and currently supports Python 2.7 and 3.

My goal was to achieve that all code changes in the master branch forces the re-creation of the appropriate executables.

First of all, it is necessary to enable CI for a GitLab project. Proceed with the following steps:

1. Select the project, click Settings and Project Settings.
2. Click Builds and Save Changes.
3. Move to the  CI Settings pane to set advanced parameters such as timeouts and automatic builds.
4. Click Save Changes.
5. Select Runners, note/copy the CI-URL and the CI-Token - we will need this information later for registrating the runners.

Host preparations

To implement agent communication and automatic builds, additional preparations need to be fulfilled. The most important is to create a dedicated user account or service user. Depending on your infrastructure this step might vary (e.g. local account, Active Directory). It is important that this user has the permission to run services. Unfortunately, client systems like Windows 7 don't offer a setting like this - in this case the user needs to have local administrative permissions. 🙁

Beyond that, the following software packages are required:

• Python: [click me!]
• Git for Windows: [click me!]
• PyInstaller: [click me!]
• GitLab Runner: [32bit] or [64bit]

The particular install process are self-explanatory. When installing Python, it is necessary to check "Add python.exe to path" - otherwise Python programs are not executable from the command line or PowerShell. For the Git installation a similar option "Use Git from the Windows Command Prompt" needs to be set, too.

PyInstaller is installed using a Python script - make sure to start it inside a command line with administrator privileges:

1C:\pyInstaller> python setup.py install
2...
3Finished processing dependencies for PyInstaller==3.1.1

Python and PyInstaller

Let's check whether Python and PyInstaller work as expected. The following source code should be catched up by PyInstaller without any issues:

1#!/usr/bin/env python
2import os
3print "Hello world!"
4raw_input("Press ENTER...")

The script prints a sentence and waits for an input before it closes itself. The following command creates an executable including the script, runtime environment and required Python modules:

1C:\test> pyinstaller -F -y test.py

The switch -F integrates the Python and Microsoft Visual C++ runtime environments, -y overwrites pre-existing files. If the utility finishes without any errors, new folders build and dist are created in the current directory. There is a folder with the same name like the script including the binary inside the dist folder. You can start it like this:

1c:\test> c:\test\dist\test\test.exe

By the way - for Python scripts that need to be executed under Windows, it is necessary to load the Python module os. Otherwise, the following error message is created when the executable is started:

1ImportError: No module named os

Powershell and CI-Runner

For automating the packaging process, I'm using a PowerShell script. To ensure that custom and thus unsigned PowerShell scripts can be executed, I had to alter the execution policy:

1PS C:\> Set-ExecutionPolicy Unrestricted

The Microsoft Knowledge base offers good explanations on the particular settings.

The GitLab CI runner is moved into a directory on the hard drive, e.g. c:\gitlab-ci-runner. To make manual debugging easier, it is also advisable to shorten the long file name. Afterwards the runner is registered to the GitLab system - for this task we need to enter the project token. After this, the runner is installed and started as Windows service:

 1c:\gitlab-ci-runner> gitlab-ci-runner.exe register
 2 Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/ci):
 3 http://gitlab.localdomain.loc/ci
 4 Please enter the gitlab-ci token for this runner:
 5 xxx
 6 Please enter the gitlab-ci description for this runner:
 7 [windows]: windows.localdomain.loc
 8 Please enter the gitlab-ci tags for this runner (comma separated):
 9 win,python
10 INFO[0035] 7ab95543 Registering runner... succeeded
11 Please enter the executor: ssh, shell, parallels, docker, docker-ssh:
12 shell
13 INFO[0143] Runner registered successfully.
14c:\gitlab-ci-runner> gitlab-ci-runner.exe install
15c:\gitlab-ci-runner> gitlab-ci-multi-runner.exe start

GitLab integration

To make GitLab push the code changes to the runners, we need to create the .gitlab-ci.yml file. As I already mentioned in the last post, this file controls the behavior of runners. The following script automatically executes a PowerShell script after every commit to the master branch.

1build:
2  script:
3    - "powershell.exe -File build_binaries.ps1"

The PowerShell script looks like this:

1$folders= ls | where {$_.mode -match "d"}
2foreach($folder in $folders)
3{
4  Write-Host "Moving to '$folder'"
5  cd $folder
6  Write-Host "Building binary by executing: pyinstaller.exe -F -y $folder.py"
7  pyinstaller.exe -F -y "$folder.py"
8  cd ..
9}

The script is kinda simple, it moves into every sub-folder and converts a Python script with the same name.

Conclusion

There are still a couple of things in the scenario that could be extended - for example:

• Compiling only on particular nodes (e.g. depending on additional dependencies or hardware ressources)
• Functional tests after compiling - e.g. using unittest
• Automatic upload of converted files using artefacts

I'm really interested in deep-diving into artefacts, so stay tuned. 🙂