Zoekt indexer can't connect to GitLab

Description

GitLab Zoekt indexer can't connect to GitLab even if you are using the correct GitLab Shell secret file.

Environment

GitLab with exact code search
Impacted offerings:
- GitLab Self-Managed
Impacted versions:
- Zoekt indexer 17.7 and earlier

Solution

Instead of copying the GitLab Shell secret file from the GitLab server (/var/opt/gitlab/gitlab-rails/etc/gitlab_shell_secret), manually re-create its contents using the command:

echo -n "INSERT_SECRET_CONTENTS_HERE" > gitlab_shell_secret

You can then use specify this file in the Zoekt indexer:

/bin/gitlab-zoekt-indexer -index_dir /data/index -listen ":6065" -self_url "${ZOEKT_GATEWAY_URL}" -gitlab_url "${GITLAB_URL}" -secret_path gitlab_shell_secret'

Cause

GitLab Zoekt indexer 17.7 and earlier can't handle newlines or spaces at the end of the file specified in -secret_path.

This is fixed in GitLab Zoekt indexer 17.8: gitlab-org/gitlab-zoekt-indexer#56

Additional Information

Prior to the fix, it will show the following error in the GitLab Zoekt indexer:

2024/11/06 05:29:59 TASK_REQUEST_ERROR: TaskRequest responded with [401] {"message":"401 Unauthorized"}

The GitLab rails application logs will have the following error:

{
  "severity": "ERROR",
  "time": "2025-01-23T17:04:38.926Z",
  "correlation_id": "01JJA1YZ03790VVSEXVHV5RFBB",
  "meta.caller_id": "GET /api/:version/internal/search/zoekt/:uuid/tasks",
  "meta.remote_ip": "172.18.0.3",
  "meta.feature_category": "global_search",
  "meta.client_id": "ip/172.18.0.3",
  "exception.class": "JWT::VerificationError",
  "exception.message": "Signature verification failed",
  "exception.backtrace": [
    "jwt (2.9.3) lib/jwt/decode.rb:47:in `verify_signature'",
    "jwt (2.9.3) lib/jwt/decode.rb:28:in `decode_segments'",
    "jwt (2.9.3) lib/jwt.rb:35:in `block in decode'",
...