Pipeline Integration

V1.0 – July 2024

Version Author Description
V1.0 – 2024-07-16 Diogo Hatz d50037923 Initial Version
V1.0 – 2024-07-16 Wisley da Silva Paulo 00830850 Document Review

Objective

This document aims to present the procedures required to integrate a function from Huawei Cloud’s FunctionGraph serverless service running a container image with CI/CD processes through pipelines. This way, it is possible to ensure that the container image of the FunctionGraph function is always up to date.

Considerations

Important: To integrate FunctionGraph with pipelines, simply upload the updated image to Huawei Cloud’s SWR image repository service. However, it is mandatory that both the image name and its tag are the same as those of the image that originated the FunctionGraph function, to ensure that the URL of this image is the same as that configured in the function at the time of its creation. You can also change the URL of the image associated with the function through the following API: https://support.huaweicloud.com/intl/en-us/api-functiongraph/functiongraph_06_0111.html.

There are two approaches to this integration: using pipelines from HWC’s CodeArts Pipelines service or using pipelines from third-party services. Both approaches will be explored below:

  1. CodeArts Pipeline: In the Build task configuration, simply select the “Build Image and Push to SWR” option. The image will be built and then automatically published to the SWR repository;
  2. Third-party pipeline: In the final step of the pipeline, simply add an action to upload the image to the SWR repository.

SWR Organization

Navigate to the SWR service in the Huawei Cloud console. Click the “Organizations” tab and click “Create Organization”. Give the organization a name and click “OK” to confirm the organization creation.

CodeArts

To integrate a FunctionGraph function with a pipeline created in the Huawei Cloud CodeArts Pipelines service, simply create a build task in CodeArts Build according to the steps below.

Service Endpoint

Navigate to the CodeArts Req service in the Huawei Cloud console. If this is your first time logging into the service, you will be prompted to create a CodeArts project. Access the settings of the created CodeArts project by clicking on “Settings” > “General” > “Service Endpoints” > “Create Endpoint”. Select the service where the source code related to the Dockerfile is located.

Select the authentication type with the repository and click on “Authorize and Confirm” to authenticate with the repository.

CodeArts Build

Once the repository endpoint has been created, navigate to the CodeArts Build service and click on “Create Task” to create a Build task. Give the task a name, select the Code Source type where the Dockerfile source code will be located. Configure the Service Endpoint created in item 4.1 of this document, select the repository and the repository branch and click “Next”. Select the “Maven and Container” build template and click “OK”.

Delete the Maven build step from the build process.

Click on the “Build Image and Push to SWR” build step, select the desired Docker version, the image repository as SWR, and the organization created in item 3.0 of this document.

Important: For the image name and tag, both must be fixed, since changes to the image in FunctionGraph will only be reflected for a given name and tag pair. It is recommended to use the latest tag.

CodeArts Pipelines

Navigate to the CodeArts Pipelines service section in the Huawei Cloud console and click “Create Pipeline”.

Set a name for the pipeline, the service where the Dockerfile source code is located, and the endpoint created in item 4.1 of this document.

Also configure the repository where the source code is located and its branch. Click “Next” to proceed and select the “Blank Template” template.

In “Stage_1”, delete the existing “New Job” and create a job for Build. In the Build stage configuration, associate the task created in item 4.2 of this document and the repository where the Dockerfile source code is located.

To add a trigger to the pipeline, click “Task Orchestration” above and select the “Execution Plan” option. Select the trigger type, such as code commit or merge request.

Third-party pipeline

To integrate a FunctionGraph function with a third-party pipeline, simply upload the image after its build to the HWC SWR repository by adding a Docker task in the final stage of the pipeline, according to the commands executed below. Here is the documentation for the Docker “Docker@2” task on Azure as a reference: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/docker-v2?view=azure-pipelines&tabs=yaml.

Getting a Permanent Login to SWR

Generating an AK/SK

To get permanent authentication credentials with SWR, first log in to the Huawei Cloud console and access the Identity and Access Management (IAM) service, click “User Groups” and then “Create User Group”.

Give the group a name and click “OK”.

Click “Authorize” next to the created group, select the “SWR Admin” policy, followed by “OK” and “Finish”.

Now to create a new user, click on “Create User”.

Give the user a name, uncheck the “Management console access” box, and check the “Access key” box. Click “Next” to proceed, add the user to the group created above, and then click “Create”.

Click “OK” to download the generated AK/SK key, which will be used to create authentication credentials with Docker later.

Generating Docker credentials

Log in to any machine running the Linux operating system and type the following command, replacing the “AK” and “SK” fields with the AK/SK key generated in item 5.1.1 of this document.

printf "AK" | openssl dgst -binary -sha256 -hmac "SK" | od -An -vtx1 |
sed 's/\[ \\n\]//g' | sed 'N;s/\\n//'

After executing the command, a key will be generated. Simply replace the key generated in the command below in the “Login key” field and the AK in the “AK” field to obtain the Docker authentication command with the SWR repository.

docker login -u [Regional project name]@[AK] -p [Login key] [Image repository address]

For the Santiago region, for example, the “Regional Project Name” is la-south-2, while the “Image repository address” is swr.la-south-2.myhuaweicloud.com. Here is an example of the authentication command:

docker login -u la-south-2@RVHVMX\*\*\*\*\*\* -p cab4ceab4a1545\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* swr.la-south-2.myhuaweicloud.com

Uploading the image to SWR

After logging in the SWR repository through Docker, copy the following command, changing the fields [image name 1:tag 1], [Image repositor address], [Organization name] and [Image name 2:tag 2] according to the list below:

  • [Image name 1:tag 1]: {name:tag} of the image that will be uploaded;

  • [Image repositor address]: SWR domain. It can be obtained in the login command obtained in item 5.1.2 of this document;

  • [Organization name]: Name of the organization created in item 3.0 of this document;

  • [Image name 2:tag 2]: {name:tag} of the image that will appear in the SWR. Similar to a rename, the same name and tag can be kept.

docker tag {Image name 1:tag 1} {Image repository address} {Organization name} {Image name 2:tag 2}

Example:

docker tag novo:1.0 swr.la-south-2.myhuaweicloud.com/adada/serase

Now just upload the image with the following command:

docker push {Image repository address} {Organization name} {Image name 2:tag 2}

Example:

docker push swr.la-south-2.myhuaweicloud.com/adada/serase

On the SWR service page in the HWC console, you can see that the image was successfully published to the repository:

Creating the pipeline task

The final task of the pipeline itself will consist of a series of commands executed in Docker that will follow the following workflow:

  1. Log in to the Registry where the built image is located;

  2. Pull the image from the original Registry;

  3. Log out of the Registry;

  4. Log in to Huawei’s SWR repository;

  5. Associate a tag with the image in item 2;

  6. Push the image to Huawei’s SWR repository;

  7. Log out of the SWR repository;

FunctionGraph

Having ensured that the name and tag of the image published to SWR via the Pipeline is exactly the same as the image associated with the function in FunctionGraph, any changes made to the image will be reflected in the FunctionGraph function.

Example

In this example, a pipeline was created so that every time a commit is made to the master branch of the specified Test repository, the pipeline will be activated and will build the Docker image according to the Dockerfile in the repository, upload it to the Huawei Cloud SWR repository, and consequently update the FunctionGraph function.

Committing to GitHub repository:

Pipeline triggered:

Build complete, pipeline completed:

You can see in the SWR repository that the image has been updated:

Since the image has the same URL as the image associated with the FunctionGraph function, the changes will be reflected in the desired function.

References