If you have used Docker, you already know Docker Hub – a service hosted by Docker, CodeFresh, Gitlab …, all of them are where you can store the docker image versions of the project. A service that stores such versions of the docker image is called the Docker Registry and we are daily interacting with it via docker pull
or docker push
. A bit related but the CodeFresh Registry has just been removed and has not been available since July 15, 2020. Therefore, I have learned how to host a Docker Registry exclusively for the project, or Private Docker Registry. Now let’s start!
Warning: Posts using Docker, Traefik so you should read through the documents of Traefik and Docker first if you have not found it.
Private Docker Registry
Docker Registry server is the place to store all versions of the docker image you push. Wait a minute !! Why do I say that the Docker Registry itself, not create it yourself …?
Because I will build the Private Docker Registry based on the opensource Docker Registry, not the code from the beginning! Indeed, it is OPEN SOURCE. You can see the code here:
- V1: https://github.com/docker-archive/docker-registry (DEPRECATED)
- V2: https://github.com/docker/distribution
Building the Registry server I will use its Docker image as registry:2
– published on Docker Hub. I will also combine with using Traefik in this article to perform setup with Docker for fast. Basically, registry:2
is a web server, it will expose at port :5000
. Let’s run it to check it out:
1 2 3 | <span class="token comment"># run docker registry:</span> docker run -d --name registry -p <span class="token number">5000</span> :5000 -v /mnt/registry:/var/lib/registry registry:2 |
You can try pushing an image containous/whoami:latest
into the registry:
1 2 3 4 5 6 | <span class="token comment"># rename image for registry</span> docker tag containous/whoami:latest localhost:5000/whoami:latest <span class="token comment"># push</span> docker push localhost:5000/whoami:latest |
A few small notes
- The rule to name the image when pushing the registry is mandatory:
<registry_hostname>/<image_path>:<tag>
, here we have the local host islocalhost:5000
. - We can completely change the port from 5000 to another port when running by docker. For example:
-p 8000:5000
, then the image name will belocalhost:8000/whoami
. - Need to create a volume to contain registry data, the default directory is
/var/lib/registry
, you can customize another folder, change the storage driver to save on Amazon S3 … but this article I will skip this part. . I will use the local filesystem. - The registry supports setup using TLS directly through the environment as follows
-e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key
, but in this article I will set up TLS on Reverse Proxy, Traefik always.
Deploy Registry on localhost with Traefik + Docker
Once you have the basic deployment method, now I will proceed to host the Docker Registry on the local; Using Traefik as a reverse proxy combines self-signed certificate TLS that Traefik itself has created. My registry server will have the URL is https://r.omd.lc
.
Setup Traefik
When there is a router setting with tls=true
but I do not have a certificate installed, Traefik will automatically generate selfsigned cert for me. In case of deploying to production, I switched to using docker swarm.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | <span class="token comment"># docker-compose -f traefik.yml up -d</span> <span class="token key atrule">version</span> <span class="token punctuation">:</span> <span class="token string">'3.5'</span> <span class="token key atrule">services</span> <span class="token punctuation">:</span> <span class="token key atrule">traefik</span> <span class="token punctuation">:</span> <span class="token key atrule">image</span> <span class="token punctuation">:</span> traefik <span class="token punctuation">:</span> v2.2 <span class="token key atrule">command</span> <span class="token punctuation">:</span> <span class="token punctuation">-</span> <span class="token punctuation">-</span> <span class="token punctuation">-</span> log.level=DEBUG <span class="token punctuation">-</span> <span class="token punctuation">-</span> <span class="token punctuation">-</span> api.dashboard=true <span class="token punctuation">-</span> <span class="token punctuation">-</span> <span class="token punctuation">-</span> entrypoints.websecure.address= <span class="token punctuation">:</span> <span class="token number">443</span> <span class="token punctuation">-</span> <span class="token punctuation">-</span> <span class="token punctuation">-</span> providers.docker <span class="token punctuation">-</span> <span class="token punctuation">-</span> <span class="token punctuation">-</span> providers.docker.watch=true <span class="token punctuation">-</span> <span class="token punctuation">-</span> <span class="token punctuation">-</span> providers.docker.exposedByDefault=false <span class="token key atrule">ports</span> <span class="token punctuation">:</span> <span class="token punctuation">-</span> 443 <span class="token punctuation">:</span> <span class="token number">443</span> <span class="token key atrule">volumes</span> <span class="token punctuation">:</span> <span class="token punctuation">-</span> /var/run/docker.sock <span class="token punctuation">:</span> /var/run/docker.sock <span class="token punctuation">:</span> ro <span class="token key atrule">labels</span> <span class="token punctuation">:</span> <span class="token punctuation">-</span> traefik.enable=true <span class="token punctuation">-</span> traefik.http.routers.traefik.entrypoints=websecure <span class="token punctuation">-</span> traefik.http.routers.traefik.rule=Host(`traefik.omd.lc`) <span class="token punctuation">-</span> <a class="__cf_email__" href="/cdn-cgi/l/email-protection">[email protected]</a> <span class="token punctuation">-</span> traefik.http.routers.traefik.tls=true |
After running, we can completely access Traefik’s dashboard at https://traefik.omd.lc
Setup Private Docker Registry
I will convert the docker command to install the docker registry above to docker-compose
and integrate with Traefik as follows:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | <span class="token comment"># docker-compose -f registry.yml up -d</span> <span class="token key atrule">version</span> <span class="token punctuation">:</span> <span class="token string">'3.5'</span> <span class="token key atrule">volumes</span> <span class="token punctuation">:</span> <span class="token key atrule">registry_data</span> <span class="token punctuation">:</span> <span class="token key atrule">services</span> <span class="token punctuation">:</span> <span class="token key atrule">registry</span> <span class="token punctuation">:</span> <span class="token key atrule">image</span> <span class="token punctuation">:</span> registry <span class="token punctuation">:</span> <span class="token number">2</span> <span class="token key atrule">volumes</span> <span class="token punctuation">:</span> <span class="token punctuation">-</span> registry_data <span class="token punctuation">:</span> /var/lib/registry <span class="token key atrule">labels</span> <span class="token punctuation">:</span> <span class="token punctuation">-</span> traefik.enable=true <span class="token punctuation">-</span> traefik.http.routers.docker <span class="token punctuation">-</span> registry.entrypoints=websecure <span class="token punctuation">-</span> traefik.http.routers.docker <span class="token punctuation">-</span> registry.rule=Host(`r.omd.lc`) <span class="token punctuation">-</span> traefik.http.routers.docker <span class="token punctuation">-</span> registry.tls=true <span class="token punctuation">-</span> traefik.http.services.docker <span class="token punctuation">-</span> registry.loadbalancer.server.port=5000 |
In particular, the docker registry host will now be https://r.omd.lc
, the docker registry server will be behind the reverse-proxy, Traefik. Traefik will forward requests from port :443
into the correct docker registry container. I tried to push the image back into this registry.
1 2 3 4 5 6 7 8 9 10 11 12 | <span class="token comment"># rename image:</span> docker tag containous/whoami:latest r.omd.lc/webee/whoami:latest <span class="token comment"># push to registry:</span> docker push r.omd.lc/webee/whoami:latest The push refers to repository <span class="token punctuation">[</span> r.omd.lc/webee/whoami <span class="token punctuation">]</span> d39a8d45d503: Layer already exists ef02b53d2c9c: Layer already exists dc788139f06c: Layer already exists latest: digest: sha256:e6d0a6d995c167bd339fa8b9bb2f585acd9a6e505a6b3fb6afb5fcbd52bbefb8 size: <span class="token number">948</span> |
Successfully, so we have successfully hosted the Docker Registry on localhost. But do you see anything wrong ??? Looks like we are hosting the Private Docker Registry but everyone has the right to push images into this Registry as well (^^;) Now let’s move on to the next section!
Basic Authentication
With the need for a private registry, the easiest way is to set up basic authentication. I will add two basic auth accounts as a model with the username / password as follows:
- user1 / secret
- user2 / secret
Generate credentials with htpasswd
, note, do not use tools or websites on the web to generate basic auth credentials. Doing so will help enrich the dictionary for hackers only. Use simple htpasswd
as follows:
1 2 3 4 5 6 7 8 9 10 | htpasswd -nb username password <span class="token comment"># generate for user1</span> htpasswd -nb user1 secret user1: <span class="token variable">$apr1</span> <span class="token variable">$t5s4HR</span> .O <span class="token variable">$L5ZSZZEsWyAAF6</span> /1icD4n0 <span class="token comment"># generate for user2</span> htpasswd -nb user2 secret user2: <span class="token variable">$apr1</span> <span class="token variable">$nEZFU0QX</span> <span class="token variable">$aRtaFM8IVIIer93KpQ</span> /Qm1 |
Set up basic auth with Traefik by adding the following 2 labels to registry.yml
:
1 2 3 4 5 6 7 | <span class="token key atrule">services</span> <span class="token punctuation">:</span> <span class="token key atrule">registry</span> <span class="token punctuation">:</span> <span class="token key atrule">labels</span> <span class="token punctuation">:</span> <span class="token punctuation">-</span> <span class="token punctuation">...</span> <span class="token punctuation">-</span> traefik.http.middlewares.private <span class="token punctuation">-</span> service.basicauth.users=user1 <span class="token punctuation">:</span> $$apr1$$t5s4HR.O$$L5ZSZZEsWyAAF6/1icD4n0 <span class="token punctuation">,</span> user2 <span class="token punctuation">:</span> $$apr1$$nEZFU0QX$$aRtaFM8IVIIer93KpQ/Qm1 <span class="token punctuation">-</span> traefik.http.routers.docker <span class="token punctuation">-</span> registry.middlewares=private <span class="token punctuation">-</span> service |
When adding the basic auth credential token to the yaml file, the
$
marks will need to be escaped back to$$
for the correct syntax
Bump !!! Now let’s try to push again to see if the docker registry is still public or not!
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | <span class="token comment"># push to registry:</span> docker push r.omd.lc/webee/whoami:latest <span class="token comment"># got error because we are guest</span> The push refers to repository <span class="token punctuation">[</span> r.omd.lc/webee/whoami <span class="token punctuation">]</span> d39a8d45d503: Preparing ef02b53d2c9c: Preparing dc788139f06c: Preparing no basic auth credentials <span class="token comment"># login to registry:</span> docker login r.omd.lc -u user1 -p secret WARNING <span class="token operator">!</span> Using --password via the CLI is insecure. Use --password-stdin. Login Succeeded <span class="token comment"># re-push to registry:</span> docker push r.omd.lc/webee/whoami:latest <span class="token comment"># successfully:</span> The push refers to repository <span class="token punctuation">[</span> r.omd.lc/webee/whoami <span class="token punctuation">]</span> d39a8d45d503: Layer already exists ef02b53d2c9c: Layer already exists dc788139f06c: Layer already exists latest: digest: sha256:e6d0a6d995c167bd339fa8b9bb2f585acd9a6e505a6b3fb6afb5fcbd52bbefb8 size: <span class="token number">948</span> |
Yummy! ^^ In addition to setting up basic auth using Traefik, another way is to set up the Docker Registry directly by adding the environment:
1 2 3 4 5 6 7 8 9 10 11 | <span class="token key atrule">services</span> <span class="token punctuation">:</span> <span class="token key atrule">registry</span> <span class="token punctuation">:</span> <span class="token key atrule">labels</span> <span class="token punctuation">:</span> <span class="token punctuation">-</span> <span class="token punctuation">...</span> <span class="token key atrule">environment</span> <span class="token punctuation">:</span> <span class="token key atrule">REGISTRY_AUTH</span> <span class="token punctuation">:</span> htpasswd <span class="token key atrule">REGISTRY_AUTH_HTPASSWD_REALM</span> <span class="token punctuation">:</span> Registry Realm <span class="token key atrule">REGISTRY_AUTH_HTPASSWD_PATH</span> <span class="token punctuation">:</span> /auth/htpasswd <span class="token key atrule">volumes</span> <span class="token punctuation">:</span> <span class="token punctuation">-</span> ./auth <span class="token punctuation">:</span> /auth |
In general, with the basic usage of auth, the basic auth has acted as an access token like other registry systems, you can fully use it to setup in CI / CD before pushing the image onto the registry with the advantage of setup Simple and fast. However, it also has two huge disadvantages:
- There is no permission between accounts – user1 can push an image (the same name) over an existing image of user2; or user1 pulls images of any other account to local.
- No flexibility in account management, password change must modify the server’s basic configuration auth => The more users the more effort
- Difficult to scale in building GUI to manage
Therefore, we will need a more ideal solution.
Registry Token Authorization (OAuth2)
A more flexible and wonderful solution is that we will build an independent service with the Registry server, taking on both responsibilities:
- Authentication: Verify your identity
- Authorization: Verify you have access to resources
Unfortunately, this service is not provided by Docker so we need to build according to the specified workflow specifications.
Docker Registry Workflow
I emphasize a bit, this article talks about Registry v2, not Registry v1. This workflow is also of Registry v2. Because this solution is quite long and more complicated, within this article, we will study about how the Authorization service works and we will implement it in the next article. Now, let’s take a look at the workflow specification:
- The docker client sends a pull / push request to the Registry
- If the Registry requires authentication, it will return an HTTP response with a status of
401 - Unauthorized
with information on how to perform the authenticate. - The client then makes a request to the authorization service to request the
Bearer token
. - Authorization service returns
Bearer token
to indicate that the client has access to resources. - The client tries to resend the previous pull / push request to the Registry with the
Bearer token
inserted into the Authorization header of the request. - The registry performs authorize for the client by checking the
Bearer token
for theclaim
comparison. The pull / push process will start as usual.
Docker Client or Docker Registry Client is included in Docker Engine. Since Docker v1.11, the Docker engine supports both Basic Auth and OAuth2. As for Docker v1.10 and earlier, the Docker engine only supports Basic Auth.
In summary, we will have 3 main subjects that will be mentioned throughout:
- Docker Regsitry Client
(Docker engine trên máy client)
- Registry Server
(regsitry:2)
=>https://registry.docker.io
- Token Server
(authorization service)
=>https://auth.docker.io
How to authenticate
As the workflow above, as soon as the Registry server receives the request, if the server requires authentication for that request (depending on the service’s policy: such as private repository, it will require, public repository does not need …) then it will return an HTTP response of 401 - Unauthorized
and include a WWW-Authenticate
header containing information describing how to authenticate.
I would like to summarize the example from the documentation of Docker. For example, I (whose username is jlhawn
) push an image to the samalba/my-app
repository. First, the registry server will return a response with the following pattern:
1 2 3 4 5 6 7 8 9 10 | HTTP/1.1 <span class="token number">401</span> Unauthorized Content-Type: application/json <span class="token punctuation">;</span> <span class="token assign-left variable">charset</span> <span class="token operator">=</span> utf-8 Docker-Distribution-Api-Version: registry/2.0 Www-Authenticate: Bearer <span class="token assign-left variable">realm</span> <span class="token operator">=</span> <span class="token string">"https://auth.docker.io/token"</span> ,service <span class="token operator">=</span> <span class="token string">"registry.docker.io"</span> ,scope <span class="token operator">=</span> <span class="token string">"repository:samalba/my-app:pull,push"</span> Date: Thu, <span class="token number">10</span> Sep <span class="token number">2015</span> <span class="token number">19</span> :32:31 GMT Content-Length: <span class="token number">235</span> Strict-Transport-Security: max-age <span class="token operator">=</span> <span class="token number">31536000</span> <span class="token punctuation">{</span> <span class="token string">"errors"</span> : <span class="token punctuation">[</span> <span class="token punctuation">{</span> <span class="token string">"code"</span> <span class="token builtin class-name">:</span> <span class="token string">"UNAUTHORIZED"</span> , <span class="token string">"message"</span> <span class="token builtin class-name">:</span> <span class="token string">"access to the requested resource is not authorized"</span> , <span class="token string">"detail"</span> : <span class="token punctuation">[</span> <span class="token punctuation">{</span> <span class="token string">"Type"</span> <span class="token builtin class-name">:</span> <span class="token string">"repository"</span> , <span class="token string">"Name"</span> <span class="token builtin class-name">:</span> <span class="token string">"samalba/my-app"</span> , <span class="token string">"Action"</span> <span class="token builtin class-name">:</span> <span class="token string">"pull"</span> <span class="token punctuation">}</span> , <span class="token punctuation">{</span> <span class="token string">"Type"</span> <span class="token builtin class-name">:</span> <span class="token string">"repository"</span> , <span class="token string">"Name"</span> <span class="token builtin class-name">:</span> <span class="token string">"samalba/my-app"</span> , <span class="token string">"Action"</span> <span class="token builtin class-name">:</span> <span class="token string">"push"</span> <span class="token punctuation">}</span> <span class="token punctuation">]</span> <span class="token punctuation">}</span> <span class="token punctuation">]</span> <span class="token punctuation">}</span> |
Notice this header:
1 2 | Www-Authenticate: Bearer <span class="token assign-left variable">realm</span> <span class="token operator">=</span> <span class="token string">"https://auth.docker.io/token"</span> ,service <span class="token operator">=</span> <span class="token string">"registry.docker.io"</span> ,scope <span class="token operator">=</span> <span class="token string">"repository:samalba/my-app:pull,push"</span> |
Both the header name and its content are compliant with the Bearer Token
usage specification document in Section 3 of RFC 6750: The OAuth 2.0 Authorization Framework: Bearer Token Usage .
Accordingly, when receiving the WWW-Authenticate
header in the response, the server challenged the client to obtain a Bearer token
and send it in each request to the Registry server to gain access to resources. The server also provides instructions that the Client needs to make a GET
request to the URL https://auth.docker.io/token
with the request to access the service registry.docker.io
to use the samalba/my-app
repository with the pull
permission. and push
.
=> This is the auth challenge
that you will see mentioned in the RFC 6750 specification document above.
Request to get Bearer token
An example of a request to get a Bearer token
will be shown below, about query params when requesting to get a bearer token and fields in the response, please read more in Requesting A Token: Token Authentication Specification | Docker Documentation :
1 2 | GET https://auth.docker.io/token?service <span class="token operator">=</span> registry.docker.io <span class="token operator">&</span> <span class="token assign-left variable">scope</span> <span class="token operator">=</span> repository:samalba/my-app:pull,push |
The Token Server will return a response with the following content:
token
: It is the bearer token that the client will embed later requests through theAuthorization
headeraccess_token
: The same bearer token above, it is added under the nameaccess_token
to be compatible with OAuth2. Either field is required. Or there are both but they should be the same.- And some other fields …
Here is an example of the Response we need to implement when the authenticate is successful:
1 2 3 4 5 6 7 8 9 | HTTP/1.1 <span class="token number">200</span> OK Content-Type: application/json <span class="token punctuation">{</span> <span class="token string">"token"</span> <span class="token builtin class-name">:</span> <span class="token string">"eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzI1NiIsImtpZCI6IlBZWU86VEVXVTpWN0pIOjI2SlY6QVFUWjpMSkMzOlNYVko6WEdIQTozNEYyOjJMQVE6WlJNSzpaN1E2In0.eyJpc3MiOiJhdXRoLmRvY2tlci5jb20iLCJzdWIiOiJqbGhhd24iLCJhdWQiOiJyZWdpc3RyeS5kb2NrZXIuY29tIiwiZXhwIjoxNDE1Mzg3MzE1LCJuYmYiOjE0MTUzODcwMTUsImlhdCI6MTQxNTM4NzAxNSwianRpIjoidFlKQ08xYzZjbnl5N2tBbjBjN3JLUGdiVjFIMWJGd3MiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6InNhbWFsYmEvbXktYXBwIiwiYWN0aW9ucyI6WyJwdXNoIl19XX0.QhflHPfbd6eVF4lM9bwYpFZIV0PfikbyXuLx959ykRTBpe3CYnzs6YBK8FToVb5R47920PVLrh8zuLzdCr9t3w"</span> , <span class="token string">"expires_in"</span> <span class="token builtin class-name">:</span> <span class="token number">3600</span> , <span class="token string">"issued_at"</span> <span class="token builtin class-name">:</span> <span class="token string">"2009-11-10T23:00:00Z"</span> <span class="token comment">#RFC3339</span> <span class="token punctuation">}</span> |
If the authenticate fails, the Token Server will need a 401 - Unauthorized
response to indicate that the credentials
are invalid. In the case of successful authentication, Token Server will need to check the ACL (Access Control List) based on the scope of the request. As in this example, after verifying that the client is the jlhawn
user, the Token Server will need to check with the jlhawn
user to have the pull / push permission in the samalba/my-app
repository stored in the registry server registry.docker.io
.
According to the Docker documentation, in this step, Token Server will determine a list of valid rights in the scope, if in the scope of the request no rights are valid or less, it is also not considered an error. Instead, the token server will create a list that is empty or less than the scope. Then rely on this set of permissions to generate the token
and return it to the Registry Client.
Use Bearer token
Now, on each subsequent request to the Registry server, the client only needs to embed the token in the header as follows to access the resource:
1 2 | Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzI1NiIsImtpZCI6IkJWM0Q6MkFWWjpVQjVaOktJQVA6SU5QTDo1RU42Ok40SjQ6Nk1XTzpEUktFOkJWUUs6M0ZKTDpQT1RMIn0.eyJpc3MiOiJhdXRoLmRvY2tlci5jb20iLCJzdWIiOiJCQ0NZOk9VNlo6UUVKNTpXTjJDOjJBVkM6WTdZRDpBM0xZOjQ1VVc6NE9HRDpLQUxMOkNOSjU6NUlVTCIsImF1ZCI6InJlZ2lzdHJ5LmRvY2tlci5jb20iLCJleHAiOjE0MTUzODczMTUsIm5iZiI6MTQxNTM4NzAxNSwiaWF0IjoxNDE1Mzg3MDE1LCJqdGkiOiJ0WUpDTzFjNmNueXk3a0FuMGM3cktQZ2JWMUgxYkZ3cyIsInNjb3BlIjoiamxoYXduOnJlcG9zaXRvcnk6c2FtYWxiYS9teS1hcHA6cHVzaCxwdWxsIGpsaGF3bjpuYW1lc3BhY2U6c2FtYWxiYTpwdWxsIn0.Y3zZSwaZPqy4y9oRBVRImZyv3m_S9XDHF1tWwN7mL52C_IiA73SJkWVNsvNqpJIn5h7A2F8biv_S2ppQ1lgkbw |
Of course, the Registry server will need a mechanism to decrypt the bearer token and retrieve the claims set and verify the token is valid or not.
References
That’s how to build a Token Server to use for the Private Docker Registry. If you find this post ugly, then please be bold to downvote. In summary, this article of mine has also shown you:
- How to set up the Private Docker Registry with Traefik
- How to authenticate to the Registry using Basic Auth with Traefik
- Introduce to everyone about workflow to create Token Server to overcome the disadvantages of using Basic Auth
You can refer to the documents about Docker Registry and Traefik here: