Airflow (Composer)

Seemore natively supports Airflow, which allows you to integrate your Airflow with your Seemore workspace seamlessly.

1. Access airflow API and enable composer API, if not enabled (please select the desired Cloud Composer version - 1 or 2).

2. Create a service account for Seemore

   1. In the GCP console of the desired project, create a new service account

   2. Choose a meaningful name for the service account such as “Seemore service account.”

   3. Click on "Create and continue"

1. In step 2 - add Role: "Composer User” and click on “CONTINUE”

1. In step 3 - click on “Done”, no user grants are required.

2. In the service accounts page:

1. Save the Oauth 2 client ID (you can use the copy button) of the new service account, this will be used in Airflow.

2. Click on the email of the new service account, this should lead you to the following page:

1. Select the 'KEYS' tab

1. Click on the 'ADD KEY' button

2. In the dropdown menu, choose “Create new Key”

3. In the modal that opened, choose “JSON” and click on “Create”

1. A JSON file containing the service account credentials should be immediately downloaded. If you have access to the system, please upload the file to Datamaze. Otherwise save it for an onboarding session.

In Airflow UI

1. Ensure you have an admin role in order to view the security settings.
2. Create a corresponding user with airflow roles for Datamaze service account

3. In the Airflow UI, under the Security menu, click on “List Users”

1. 1. Add a new user by clicking on the “+” button and enter the “Add user” page.

      
2. Fill in the following fields:

   1. Enter a meaningful First name and Last name.

   2. For the user name use the following format: “accounts.google.com:NUMERIC_USER_ID”. The NUMERIC_USER_ID is the Oauth 2 client ID id that you saved earlier.

   3. Check “Is Active”

   4. Specify a unique identifier as the email. You can use any unique string.

   5. Add the “Viewer” Role.
   6. Only if you see the password field, add the Password: “Datamaze” (this is not necessary since we will be authenticating via the service account, but this field is still mandatory so we must fill it.)

   7. Click on the “Save” button, and we are done.

   

   Send the service account & Airflow UI URL (for example: https://96f4e666f43643f588ec8d4144b82079-dot-us-central1.composer.googleusercontent.com) or upload them to Datamaze.\

References:

https://cloud.google.com/composer/docs/composer-2/access-airflow-api#airflow-ui