Reconfigure hangs when run from EC2 userdata

Description

There is a race condition with systemd targets when running gitlab-ctl reconfigure from AWS EC2 userdata that can cause the reconfigure to hang at:

INFO: template[/usr/lib/systemd/system/gitlab-runsvdir.service] sending run action to execute[systemctl start gitlab-runsvdir] (immediate)
 * execute[systemctl start gitlab-runsvdir] action run

Environment

Impacted offerings:

GitLab Self-Managed

Impacted versions:

First reported 11.0.0

Solution

Either delay reconfigure until multi-user.target is active or change target of gitlab systemd service as below

package['systemd_after'] = 'basic.target'
package['systemd_wanted_by'] = 'basic.target'

WARNING: If using any NFS system basic.target is before those are mounted.

Cause

The GitLab systemd service by default runs after multi-user.target is active. The userdata script may not run when that target is active or run the reconfigure before it is active.