Manage GPR Templates

This topic describes managing GPU request (GPR) templates. As an admin you can create, update, clone, view, and delete a GPR template in a given workspace.

Create a GPR Template

1. Go to Workspace on the left sidebar.

2. Under All Workspaces, select the workspace you want to manage.

3. In the menu of the selected workspace, click the GPR Templates submenu.

   

4. On the GPU Request Templates page, click the + Create GPR Template button on the right side of the page.

   

5. On the Create Template pane, enter the following information to configure a GPR Template:

   1. For Template Configuration:

      1. Enter a name for the template in the Template Name text box
      2. Select a worker cluster from the Cluster Name drop-down list.

   2. For GPU Configuration:

      1. Choose how GPU Shape and Node Type are selected:

         • Manual Selection:
           • Leave the Auto Select GPU checkbox unselected.
           • Select a memory value from the Memory (GB) per GPU drop-down list. The list displays the available memory (GB) per cluster.
           • Select a GPU shape from the GPU Shape drop-down list.
           • Select a node type from the Node Type drop-down list.
         • Auto Selection:
           • Select the Auto Select GPU checkbox. The GPU shape and node type values are automatically selected based on the memory specified.
           • If you manually enter GPU memory in the Memory (GB) per GPU text box, the Auto Select GPU checkbox is selected automatically. The minimum memory value is 1 GB, and the maximum value is the highest allocated memory available for any worker cluster in the workspace.

      2. Set the GPUs Per Node if you want to change its default value, 1.

      3. Set the GPU Nodes if you want to change its default value, 1.

   3. For Priority Configuration:

      1. GPU User is auto populated.

      2. Set Priority. The default value is Medium (1-200).

         You can change the priority of a GPR in the queue. You can change the priority number (low: 1-100, medium: 101-200, high: 201-300) to move a GPR in the queue. When a GPR is moved to the top of the queue, it is provisioned when the resources are available to provision the GPR.

      3. Set Priority Number. The default value is 101

      4. Specify Reserve for(ddd:hh:mm) duration in Days, Hours, and Minutes.

6. (Optional) Expand Advanced Configuration and:

   1. Set the idle time out to allow the GPU nodes to be used after the configured length of the time that it can be idle. This allows other GPRs to use the unused provisioned GPU nodes.

   2. To make the idle timeout to be effective, the Enforce Idle Timeout is auto selected. If you want to only configure the timeout without enforcing, then unselect this checkbox.

   3. The Requeue on Failure check box is selected by default to queue this GPR in case it fails.

      EGS auto detects issues with one or more GPUs in the provisioned GPR. EGS removes that related GPR from the workspace and re-queues that GPR.

   4. The Evict low priority GPRs checkbox is selected by default that configures auto eviction of a low-priority GPR.

7. Click Add Template. The newly added templates gets added to the list under Manage Templates on the GPU Request Templates page.

View a GPR Template

1. Go to Workspace on the left sidebar.

2. Under All Workspaces, select the workspace you want to manage.

3. In the menu of the selected workspace, click the GPR Templates submenu.

4. Under Workspaces, you see the workspace details. The Assigned Templates lists the GPU request templates assigned to the workspace. Displays No Templates Assigned if none are configured, and the Auto GPR filed shows whether automatic GPU provisioning is enabled for this workspace.

5. Under Manage Templates, you see the list of managed templates

   

6. Click the vertical ellipsis icon that is to the right of the template that you want to view, and click View from the menu.

7. View the template details.

   

Edit a GPR Template

1. Go to Workspace on the left sidebar.

2. Under All Workspaces, select the workspace you want to manage.

3. In the menu of the selected workspace, click the GPR Templates submenu.

4. On the GPU Request Templates page, go to Manage Templates.

5. Under Manage Templates, click the vertical ellipsis for the template that you want to edit. and click Edit from the menu.

   

6. On the Update Template pane, except template name, you can change the configuration of other fields.

   info

   The updated configuration is effective to the new GPRs that are created from this updated template.

   

7. After updating the template, click Save Changes for the latest configuration to be effective.

Clone a GPR Template

1. Go to Workspace on the left sidebar.

2. Under All Workspaces, select the workspace you want to manage.

3. In the menu of the selected workspace, click the GPR Templates submenu.

4. On the GPU Request Templates page, go to Manage Templates.

5. Under Manage Templates, click the vertical ellipsis for the template that you want to clone. and click Clone Template from the menu.

   

6. On the Clone Template page, the name of the template contains a suffix -clone added to the cloned parent template's name. You can change the following configuration:

   • Name of the template
   • Priority Number
   • Reserve for duration
   • Enforce Idle Timeout
   • Requeuing GPRs on Failure
   • Eviction low priority GPRs

5. Click Add Template to complete cloning the template.

Enable Auto GPR on a Template

Auto GPR feature allows you to create the GPR without any human intervention. After auto GPR is enabled, you can deploy AI workloads in the application namespace. The GPR is created based on the workload's GPU requirement. You can bind a GPR template to the workspace. After a template is attached to the workspace, an admin or users can create AI workloads. Based on those AI workloads, the GPR is created automatically.

info

An auto GPR requires a default template to automatically create a GPR. So, ensure that you set a default template for auto GPR to work.

To enable auto GPR on the default template ( template assigned to a workspace ):

1. On the GPU Request Templates page, Under Workspace, click the Edit Assigned Templates button.

2. On the Workspace Templates pane, select the template to turn on the Auto GPR toggle button.

3. Click the Save Changes button.

Set a Default Template

An auto GPR uses a default template to create a GPR. So, it's required to set a default template when you enable auto GPR on a workspace.

To set a default template:

1. On the GPU Request Templates page, under Workspace, click the Edit Assigned Template button.

2. On the Workspace Templates pane, you can see the message to select a default template.

3. Select a template that you want to set as default for that workspace, and also click the Star symbol under Default.

   

4. Click Save Changes to set the default template.

   

5. On the GPU Request Templates page, under Workspace, the newly set default is visible in the Assigned Templates field.

Change the Default Template

1. On the GPU Request Templates page, under Workspace, click the Edit Assigned Template button.
2. On the Workspace Templates pane, unselect the default template.
3. Select a new template and click the Star symbol under Default.
4. Click Save Changes to set the new default template.

Unset a Default Template

1. On the GPU Request Templates page, under Workspace, click the Edit Assigned Template button.
2. On the Workspace Templates pane, unselect the default template.
3. Click Save Changes to unset the default template.

Disable Auto GPR on a Template

1. On the GPU Request Templates page, under Workspace, click the Edit Assigned Template button.
2. On the Workspace Templates pane, turn off the Auto GPR toggle button.
3. Click Save Changes to disable Auto GPR on that workspace.

Delete a GPR Template

2. On the GPU Request Templates page, under Manage Templates, click the vertical ellipsis for the template that you want to edit, and click Delete from the menu.

3. On the delete confirmation dialog, type the name of the template and click Delete.