Manage GPR Templates

This topic describes managing GPU request (GPR) templates. As an admin you can create, update, clone, view, and delete a GPR template in a given account.

info

Across our documentation, we refer to the workspace as the slice workspace. The two terms are used interchangeably.

View a GPR Template

1. Go to GPR Templates on the left sidebar.

   

   Under Workspaces, the templates assigned to the workspace is under Assigned, and the Auto GPR indicates if it is enabled on a given workspace.

2. On the GPU Request Templates page, go to Manage Templates.

   

3. Click the vertical ellipsis icon that is to the right of the template that you want to view, and click View from the menu.

   

4. View the template details.

   

Create a GPR Template

1. Go to GPR Templates on the left sidebar.

2. On the GPU Request Templates page, click Add Template in the right side of the page.

   

3. For Template Config:

   1. Enter a name for the template in the Template Name text box
   2. Select a worker cluster from the Cluster Name drop-down list.

4. For GPU Configuration:

   1. Select Node Type from its drop-down list. The GPU Shape is auto populated.

   2. For a Multi-Instance GPU (MIG) node, Memory per GPU contains a drop-down list with memory profiles with a default node profile. Based on the number of GPU nodes and GPUs per node, select the MIG node memory profile.

      For a regular (non-MIG) GPU node, Memory per GPU is auto-populated.

   3. Set the GPU Per Node if you want to change its default value, 1.

   4. Set the GPU Nodes if you want to change its default value, 1.

5. For Priority Configuration:

   1. User is auto populated.

   2. Set Priority. The default value is Medium (101-200).

      You can change the priority of a GPR in the queue. You can change the priority number (low: 1-100, medium: 101-200, high: 201-300) to move a GPR in the queue. When a GPR is moved to the top of the queue, it is provisioned when the resources are available to provision the GPR.

   3. Set Priority Number. The default value is 101

   4. Specify Reserve for duration in Days, Hrs, and Mns.

6. Expand Advanced Configuration and:

   1. (Optional) Set the idle time out to allow the GPU nodes to be used after the configured length of the time that it can be idle. This allows other GPRs to use the unused provisioned GPU nodes.

   2. (Optional) To make the idle timeout to be effective, the Enforce Idle Timeout is auto selected. If you want to only configure the timeout without enforcing, then unselect this checkbox.

   3. (Optional) The Requeue on Failure check box is selected by default to queue this GPR in case it fails.

      EGS auto detects issues with one or more GPUs in the provisioned GPR. EGS removes that related GPR from the workspace and re-queues that GPR.

   4. (Optional) The Evict low priority GPRs checkbox is selected by default that configures auto eviction of a low-priority GPR.

      

7. Click Add Template. The newly added templates gets added to the list under Manage Templates on the GPU Request Templates page.

Edit a GPR Template

1. Go to GPR Templates on the left sidebar.

2. On the GPU Request Templates page, go to Manage Templates.

3. Under Manage Templates, click the vertical ellipsis for the template that you want to edit. and click Edit from the menu.

   

4. On the Update Template page, except template name, you can change the configuration of other fields.

   info

   The updated configuration is effective to the new GPRs that are created from this updated template.

   

5. After updating the template, click Save Changes for the latest configuration to be effective.

Clone a GPR Template

1. Go to GPR Templates on the left sidebar.

2. On the GPU Request Templates page, go to Manage Templates.

3. Under Manage Templates, click the vertical ellipsis for the template that you want to edit. and click Clone Template from the menu.

   

4. On the Clone Template page, the name of the template contains a suffix -clone added to the cloned parent template's name. You can change the following configuration:

   • Name of the template
   • Priority Number
   • Reserve for duration
   • Enforce Idle Timeout
   • Requeuing GPRs on Failure
   • Eviction low priority GPRs

5. Click Add Template to complete cloning the template.

Enable Auto GPR on a Template

Auto GPR feature allows you to create the GPR without any human intervention. After auto GPR is enabled, you can deploy AI workloads in the application namespace. The GPR is created based on the workload's GPU requirement. You can bind a GPR template to the workspace. After a template is attached to the workspace, an admin or users can create AI workloads. Based on those AI workloads, the GPR is created automatically.

To enable auto GPR:

1. On the GPU Request Templates page, click the workspace on which you want to enable auto GPR.

2. On the Slice Workspace Templates pane, turn on the Auto GPR toggle button.

   

3. Click Save Changes to enable Auto GPR on that workspace.

info

An auto GPR requires a default template to automatically create a GPR. So, ensure that you set a default template for auto GPR to work.

Set a Default Template

An auto GPR uses a default template to create a GPR. So, it's required to set a default template when you enable auto GPR on a workspace.

To set a default template:

1. On the GPU Request Templates page, click the workspace for which you want to set a default template.

2. On the Slice Workspace Templates pane, you can see the message to select a default template.

3. Select a template that you want to set as default for that workspace, and also click the Star symbol under Default.

   

4. Click Save Changes to set the default template.

   

5. On the GPU Request Templates page, under Slice Workspace, the newly set default is visible in the Assigned column.

Change the Default Template

1. On the GPU Request Templates page, click the workspace for which you want to change the default template.
2. On the Slice Workspace Templates pane, unselect the default template.
3. Select a new template and click the Star symbol under Default.
4. Click Save Changes to set the new default template.

Unset a Default Template

1. On the GPU Request Templates page, click the workspace for which you want to change the default template.
2. On the Slice Workspace Templates pane, unselect the default template.
3. Click Save Changes to unset the default template.

Disable Auto GPR on a Template

1. On the GPU Request Templates page, click the workspace on which you want to disable auto GPR.

2. On the Slice Workspace Templates pane, turn off the Auto GPR toggle button.

   

3. Click Save Changes to disable Auto GPR on that workspace.

Delete a GPR Template

1. Go to GPR Templates on the left sidebar.

2. On the GPU Request Templates page, go to Manage Templates.

3. Under Manage Templates, click the vertical ellipsis for the template that you want to edit. and click Delete from the menu.

4. On the delete confirmation dialog, type the name of the template and click Delete.