Help us learn about your current experience with the documentation. Take the survey.

Troubleshooting GitLab tokens

When working with GitLab tokens, you might encounter the following issues.

Token appears active but requests fail

A token that is listed as active can still return 401 Unauthorized, 403 Forbidden, or 404 Not Found responses. The active status indicates only that the token exists and has not expired or been revoked. This status does not mean the token can make a given request.

A token’s permissions depend on its scopes and its role. A request can also fail for reasons outside the token: where the request comes from, the resource the request targets, and whether an administrator has turned off access tokens. None of these factors are apparent from the token itself. Personal, project, and group access tokens all use the same glpat- prefix. Two tokens that look identical can therefore behave differently.

An active token can fail for any of the following reasons:

CauseResolution
The token is missing a scope that the request requires.Create a token with the necessary access token scopes. Rotation keeps the original scopes and cannot add missing scopes.
A group or project access token doesn’t have the required role.Create a token with a higher role. A token’s permissions are limited by both its role and its scopes.
The token expired.Access tokens expire at midnight UTC on their expiration date. Create a token, then update every place that used the old token.
The token was revoked, or was rotated and the original value is still in use.Rotation makes the original token inactive immediately. Use the token that the rotation created, or create a token. On GitLab Self-Managed and GitLab Dedicated, an administrator can restore a personal access token that was revoked by accident.
The token type cannot access the resource.Use a token type that can access the resource. A personal access token accesses the groups and projects available to its user. A group access token accesses the subgroups and projects in its group. A project access token accesses only its own project.
IP address restrictions block the request.These restrictions apply to group and project access tokens, and blocked requests return 404 Not Found. Send the request from an allowed address, or ask a user with the Owner role for the top-level group to add the address to the allowed ranges.
External authorization is turned on.Personal and project access tokens cannot access the container registry or the package registry. To restore access to the registries, turn off external authorization.
An administrator turned off access tokens for the instance.Ask an administrator or a user with the Owner role to turn access tokens back on.

To identify which cause applies, compare the details of the failing token with a token that works:

The details include each token’s scopes, expiration date, and usage information. Group and project access tokens also show the assigned role.

If the token’s usage information does not update after you make a request, the request might not be reaching GitLab. GitLab updates usage times every 10 minutes and usage IP addresses every minute. If GitLab isn’t recording the usage after those intervals elapse, your request did not reach GitLab.

Token does not work in an editor extension or command-line tool

A token that authenticates in the GitLab UI or API can still fail in an editor extension or a command-line tool. Scope requirements differ between tools.

Valid tokens can fail in a tool for the following reasons:

CauseResolution
The token requires different scopes.Compare the tool’s required scopes with the scopes added to the token. Create a token with the required scopes. Rotation keeps the original scopes and cannot add missing scopes.
The tool is not using the correct token.Check which token the tool authenticates with, then update or remove the incorrect token. The GitLab for VS Code extension uses a token in the GITLAB_WORKFLOW_TOKEN environment variable only when no token is configured for that instance. This variable persists after you delete your VS Code storage. To override it, configure a token for the instance in the extension.
The tool cannot connect to GitLab.If the token has the required scopes and the tool is using it, verify the tool can reach GitLab over your network. For the GitLab for VS Code extension, see authentication troubleshooting.

Expired access tokens

If an existing access token is in use and reaches the expires_at value, the token expires and:

  • Can no longer be used for authentication.
  • Is not visible in the UI.

Requests made using this token return a 401 Unauthorized response. Too many unauthorized requests in a short period of time from the same IP address result in 403 Forbidden responses from GitLab.com.

For more information on authentication request limits, see Git and container registry failed authentication ban.

Identify expired access tokens from logs

Prerequisites:

You must:

To identify which 401 Unauthorized requests are failing due to expired access tokens, use the following fields in the api_json.log file:

Field nameDescription
meta.auth_fail_reasonThe reason the request was rejected. Possible values: token_expired, token_revoked, insufficient_scope, and impersonation_disabled.
meta.auth_fail_token_idA string describing the type and ID of the attempted token.
meta.auth_fail_requested_scopesThe OAuth scopes the request required, space-separated.
meta.auth_fail_token_typeThe type of token used. Possible values: PersonalAccessToken, CiJobToken, and unknown.
meta.auth_fail_auth_header_typeHow the token was passed in the request. Possible values: private_token_header, private_token_param, bearer, and other.

When a user attempts to use an expired token, the meta.auth_fail_reason is token_expired. The following shows an excerpt from a log entry:

{
  "status": 401,
  "method": "GET",
  "path": "/api/v4/user",
  ...
  "meta.auth_fail_reason": "token_expired",
  "meta.auth_fail_token_id": "PersonalAccessToken/12",
}

In some cases, meta.auth_fail_* fields may appear on non-401 responses. Known cases include:

  • Git HTTP requests to public projects, where Rack::Attack records the token failure but the project’s public visibility allows the request to succeed.
  • The Unleash feature flags endpoint, which authorizes by HTTP_UNLEASH_INSTANCEID rather than the token.
  • Workhorse pre-authorization (/authorize) endpoints, which perform their own authorization after the token probe.

meta.auth_fail_token_id indicates that an access token of ID 12 was used. From GitLab 18.9, meta.user will also be populated with any username associated with the token used for the failed request.

To find more information about this token, use the personal access token API. You can also use the API to rotate the token.

Replace expired access tokens

To replace the token:

  1. Check where this token may have been used previously, and remove it from any automation that might still use the token.
    • For personal access tokens, use the API to list tokens that have expired recently. For example, go to https://gitlab.com/api/v4/personal_access_tokens, and locate tokens with a specific expires_at date.
    • For project access tokens, use the project access tokens API to list recently expired tokens.
    • For group access tokens, use the group access tokens API to list recently expired tokens.
  2. Create a new access token:
  3. Replace the old access token with the new access token. This process varies depending on how you use the token, for example if configured as a secret or embedded in an application. Requests made from this token should no longer return 401 responses.

Restore a personal access token

  • Tier: Free, Premium, Ultimate
  • Offering: GitLab Self-Managed, GitLab Dedicated

On GitLab Self-Managed or GitLab Dedicated instances, administrators can restore personal access tokens that were revoked accidentally. Restoration is not available on GitLab.com.

Running the following commands changes data directly. This could be damaging if not done correctly, or under the right conditions. You should first run these commands in a test environment with a backup of the instance ready to be restored, just in case.

  1. Open a Rails console.

  2. Restore the token:

    token = PersonalAccessToken.find_by_token('<token_string>')
    token.update!(revoked:false)

    For example, to restore a token of token-string-here123:

    token = PersonalAccessToken.find_by_token('token-string-here123')
    token.update!(revoked:false)

Tokens expire unexpectedly after an upgrade

Access tokens that have no expiration date are valid indefinitely, which is a security risk if the token is divulged.

Depending on your GitLab version and offering, your existing access tokens might have an expiration date automatically applied when you upgrade. For more information, see non-expiring access tokens. If you’re not aware these dates changed, authentication can fail without warning.

In GitLab 17.3 and later, GitLab does not automatically set expiration dates on existing tokens. Administrators can also turn off expiration date enforcement for new access tokens.

To analyze, extend, or remove token expiration dates, use the access token Rake tasks.